Method, device and equipment for determining target thrown by unmanned aerial vehicle and storage medium

Abstract

The application discloses a method, device and equipment for determining a target thrown by a UAV (Unmanned Aerial Vehicle) and a storage medium. The method comprises the following steps: acquiring a to-be-thrown scene image collected by a UAV, performing filtering processing on the to-be-thrown scene image to obtain a filtered to-be-thrown scene image; performing interpolation processing and amplification processing on the filtered to-be-thrown scene image to obtain a to-be-extracted feature image, extracting a target image feature in the to-be-extracted feature image, performing contrast enhancement processing on the target image feature to obtain a to-be-identified region image; identifying a target throwing region in the to-be-identified region image, determining a target throwing position based on the target throwing region, and controlling the UAV to throw a target object to the target throwing position. The accuracy of the UAV throwing is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, and in particular to a method and device for determining a target for throwing by a UAV, equipment and a storage medium. BACKGROUND

[0002] With the rapid development of UAV technology, UAVs are increasingly widely used in the field of image acquisition and processing, and have great potential in fields such as logistics, agriculture and environmental monitoring.

[0003] However, due to the complex and variable operating environment of UAVs, the image data obtained often has problems such as noise, low resolution and difficulty in identifying targets, which results in higher requirements for real-time identification and positioning accuracy of throwing tasks; traditional image processing methods cannot meet the needs of specific tasks of UAVs, such as accurately identifying a throwing target during flight. SUMMARY

[0004] The present application provides a method and device for determining a target for throwing by a UAV, equipment and a storage medium to solve the problem that a UAV cannot accurately identify a throwing target during flight.

[0005] According to an aspect of the present application, a method for determining a target for throwing by a UAV is provided, the method comprising:

[0006] obtaining a scene image to be thrown collected by a UAV, and performing filtering processing on the scene image to be thrown to obtain a filtered scene image to be thrown;

[0007] performing interpolation processing and enlargement processing on the filtered scene image to be thrown to obtain a feature image to be extracted, extracting a target image feature in the feature image to be extracted, and performing contrast enhancement processing on the target image feature to obtain a region image to be identified;

[0008] identifying a target throwing region in the region image to be identified, determining a target throwing position based on the target throwing region, and controlling the UAV to throw a target object to the target throwing position.

[0009] According to another aspect of the present application, a device for determining a target for throwing by a UAV is provided, the device comprising:

[0010] an image acquisition module configured to obtain a scene image to be thrown collected by a UAV, and perform filtering processing on the scene image to be thrown to obtain a filtered scene image to be thrown;

[0011] An image processing module is configured to perform interpolation processing and magnification processing on the filtered to-be-thrown scene image to obtain a to-be-extracted feature image, extract a target image feature from the to-be-extracted feature image, and perform contrast enhancement processing on the target image feature to obtain a to-be-recognized region image.

[0012] A throwing position determination module is configured to identify a target throwing region in the to-be-recognized region image, determine a target throwing position based on the target throwing region, and control the unmanned aerial vehicle to throw a target object to the target throwing position.

[0013] According to another aspect of the present application, an electronic device is provided, which comprises:

[0014] at least one processor; and

[0015] a memory connected to the at least one processor in communication; wherein

[0016] The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to execute the method for determining a throwing target of an unmanned aerial vehicle according to any one of the embodiments of the present application.

[0017] According to another aspect of the present application, a computer readable storage medium is provided, which stores computer instructions for enabling a processor to implement the method for determining a throwing target of an unmanned aerial vehicle according to any one of the embodiments of the present application when executed by the processor.

[0018] The technical scheme of the embodiments of the present application, by acquiring a to-be-thrown scene image collected by an unmanned aerial vehicle, performing filtering processing on the to-be-thrown scene image to obtain a filtered to-be-thrown scene image, improving the quality of the to-be-thrown scene image, then performing interpolation processing and magnification processing on the filtered to-be-thrown scene image to obtain a to-be-extracted feature image, extracting a target image feature from the to-be-extracted feature image, performing contrast enhancement processing on the target image feature to obtain a to-be-recognized region image, significantly reducing the amount of image data and highlighting the features of the image, making the originally low-contrast throwing target region more easily recognizable after processing, finally identifying a target throwing region in the to-be-recognized region image, determining a target throwing position based on the target throwing region, and controlling the unmanned aerial vehicle to throw a target object to the target throwing position, solves the problem that the unmanned aerial vehicle cannot accurately identify a throwing target during flight, and achieves the beneficial effect of improving the accuracy of unmanned aerial vehicle throwing.

[0019] It is to be understood that the details set forth herein do not limit the scope of the application to the one embodiment described but rather serve as a description of certain embodiments only, and that modifications to certain features of the one embodiment can be undertaken without departing from the scope of the application. BRIEF DESCRIPTION OF DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings needed in the embodiments description. Obviously, the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor.

[0021] Figure 1 is a flow chart of a method for determining a target thrown by a UAV according to an embodiment of the present application;

[0022] Figure 2a is a flow chart of a method for determining a target thrown by a UAV according to an embodiment of the present application;

[0023] Figure 2b is a sample diagram of image filtering enhancement according to an optional example of a method for determining a target thrown by a UAV according to an embodiment of the present application;

[0024] Figure 2c is a sample diagram of region identification and segmentation according to an optional example of a method for determining a target thrown by a UAV according to an embodiment of the present application;

[0025] Figure 2d is a sample effect diagram of target throwing position according to an optional example of a method for determining a target thrown by a UAV according to an embodiment of the present application;

[0026] Figure 3 is a structural diagram of a device for determining a target thrown by a UAV according to an embodiment of the present application;

[0027] Figure 4 is a structural diagram of an electronic device for implementing a method for determining a target thrown by a UAV according to an embodiment of the present application. DETAILED DESCRIPTION

[0028] In order to make the technical personnel in the art better understand the present application scheme, the following will combine the drawings in the embodiments of the present application, and the technical solutions in the embodiments of the present application will be described clearly and completely. Obviously, the described embodiments are only some embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor should be within the scope of protection of the present application.

[0029] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and in the above drawings are used to distinguish similar objects, and do not necessarily have to be used to describe a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the application described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device including a series of steps or units does not have to be limited to only those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0030] Embodiment one

[0031] Figure 1 A flowchart of a method for determining a target thrown by a UAV is provided for the first embodiment of the present application. The present embodiment can be applicable to the case where a UAV performs a throwing task. The method can be performed by a UAV target determination device, which can be realized in the form of hardware and / or software, and can be configured in an electronic device. As shown in the figure, the method comprises: Figure 1

[0032] S110, acquiring a to-be-thrown scene image collected by a UAV, and performing filtering processing on the to-be-thrown scene image to obtain a filtered to-be-thrown scene image.

[0033] The to-be-thrown scene image can be understood as an image of a scene that is about to perform a throwing action captured by a UAV.

[0034] Specifically, the image of the to-be-thrown scene is acquired by a camera or other image acquisition device carried on the UAV. The to-be-thrown scene image is filtered to eliminate noise in the image and ensure that the image for subsequent processing is clearer and more reliable.

[0035] Optionally, the filtering processing on the to-be-thrown scene image to obtain a filtered to-be-thrown scene image comprises: performing denoising processing on the to-be-thrown scene image by a pre-constructed noise model, and performing filter boundary processing on the to-be-thrown scene image after the denoising processing to obtain a filtered to-be-thrown scene image.

[0036] The noise model is expressed by the following formula:

[0037] s(x,y)=u(x,y)+n(x,y);

[0038] ​Among them, s(x,y) is the image of the scene to be thrown, u(x,y) is the noise-free image of the scene to be thrown, and n(x,y) is Gaussian noise.

[0039] Specifically, we choose to use zero-mean Gaussian noise and set the noise model to lay the foundation for subsequent filtering processing, providing a clear reference for noise parameters, thereby improving the denoising effect. Then, we design the window parameters of the filter. The filter window can be set to 3×3, which is expressed by the following formula:

[0040]

[0041] in, is the pixel value after filtering, Ω x,y is the window area centered at (x, y), the size is 3×3, and (i, j) is the pixel value.

[0042] The designed filter can significantly reduce the impact of noise on image quality and improve the visual clarity of the image. The 3x3 window size is moderate, which can effectively remove noise without causing excessive smoothing.

[0043] Finally, filter boundary processing is performed to fill the boundary area with mirrored pixels at the edge of the image, which can be expressed as follows:

[0044] S padded (x,y)=s(reflect(x),reflect(y));

[0045]

[0046]

[0047] Among them, x edge and y edge are the boundary coordinates of the image in the x and y directions.

[0048] Specifically, the mirror filling ensures the stability of pixels in the boundary area, so that the filtering process can be carried out smoothly in the entire image domain, thereby preventing boundary pixels from being processed incorrectly or ignored.

[0049] S120. Interpolate and amplify the filtered image of the scene to be thrown to obtain a feature image to be extracted, extract target image features from the feature image to be extracted, and perform contrast enhancement on the target image features to obtain an image of the area to be identified.

[0050] Specifically, the filtered image of the throwing scene is interpolated and amplified to achieve higher resolution, which can match the throwing targets at different distances and viewing angles, and provide more detailed information for subsequent feature extraction and segmentation.

[0051] Optionally, the filtered image to be thrown scene is interpolated by the following formula:

[0052]

[0053] Wherein, a ij is the interpolation coefficient, (x', y') is the new pixel point inserted, x' and y' are image coordinates, (x'-x) and (y'-y) are the relative positions of the interpolated pixels.

[0054] Specifically, the image edge can be effectively smoothed by the interpolation method, so that the enlarged image is more natural and clear in vision, which is helpful for subsequent image analysis and feature extraction. The interpolation coefficient is calculated in the specific process, using the surrounding pixel values and their first derivatives in the horizontal, vertical and diagonal directions, denoted as f x , f y and f xy . The formula for calculating the corresponding pixel (x, y) is as follows:

[0055]

[0056] Exemplarily, a pixel point has 16 interpolation matrices A, and the new interpolation pixel value can be calculated by the following matrix formula:

[0057]

[0058] Wherein, the matrix The matrix A is composed of eigenvectors.

[0059] Optionally, the target image feature in the feature image to be extracted includes: determining the covariance matrix of the feature image to be extracted, and performing eigenvalue decomposition on the covariance matrix to obtain the target image feature.

[0060] Exemplarily, the eigenvalue decomposition of the covariance matrix to obtain the target image feature is represented by the following formula:

[0061]

[0062] K=A(U-μ);

[0063] Wherein, C x is the covariance matrix, y i is the column vector of the feature image to be extracted, μ is the mean vector, K represents the image containing the target image feature, A is the matrix, and U is the feature image to be extracted. The matrix A is used to convert the feature image to be extracted U into the feature image K containing the target image feature.

[0064] Optionally, the contrast enhancement processing is performed on the target image feature based on a cumulative distribution function to obtain the to-be-identified region image.

[0065] Optionally, the contrast enhancement processing is performed on the target image feature based on a cumulative distribution function according to the following formula:

[0066]

[0067] wherein T(r k ) is a cumulative distribution function, p(r i ) is a probability of a pixel with intensity r i , and h(r j ) is a pixel intensity frequency.

[0068] In this embodiment, by extracting the target image feature, the image data amount is significantly reduced and the target point feature is highlighted, the contrast enhancement processing is performed on the target image feature based on a cumulative distribution function, so that the target image feature is more recognizable in the feature space, and the target point is separated from the surrounding region. The contrast enhancement part enhances the image contrast through the optimized histogram equalization, so that the target image feature is more obvious in the image, and the image brightness distribution is adjusted, so that the originally low-contrast throwing region is more easily identified after processing.

[0069] S130, identifying a target throwing region in the to-be-identified region image, determining a target throwing position based on the target throwing region, and controlling the unmanned aerial vehicle to throw a target object to the target throwing position.

[0070] The target throwing region can be understood as a specific region or place where the unmanned aerial vehicle needs to throw the object. The target object can be understood as an object that the unmanned aerial vehicle needs to carry and throw.

[0071] Specifically, at least one throwing position is determined in the identified target throwing region. This can be determined based on the center, edge and specific marker point of the region, etc. The throwing position coordinates in the image are converted into actual coordinates in the unmanned aerial vehicle navigation system, and the flight path of the unmanned aerial vehicle is planned according to the actual coordinates. When the unmanned aerial vehicle approaches the target throwing position, the flight height, speed and direction of the unmanned aerial vehicle are controlled to ensure accurate throwing of the target object. At the appropriate time, the throwing mechanism of the unmanned aerial vehicle is triggered to throw the target object to the target position. Optionally, during the throwing process, the state of the unmanned aerial vehicle and the accuracy of the target position are continuously monitored. When deviation or abnormality is found in the throwing, the flight path or throwing strategy of the unmanned aerial vehicle is adjusted in time.

[0072] The technical scheme of the embodiment of the present application comprises the following steps: acquiring a to-be-throwing scene image collected by a UAV, performing filtering processing on the to-be-throwing scene image to obtain a to-be-throwing scene image after filtering processing, improving the quality of the to-be-throwing scene image, performing interpolation processing and amplification processing on the to-be-throwing scene image after filtering processing to obtain a to-be-extracted feature image, extracting a target image feature in the to-be-extracted feature image, performing contrast enhancement processing on the target image feature to obtain a to-be-identified region image, significantly reducing the image data amount and highlighting the features of the image, making the originally low-contrast throwing target region more easily identified after processing, identifying a target throwing region in the to-be-identified region image, determining a target throwing position based on the target throwing region, and controlling the UAV to throw a target object to the target throwing position, thereby solving the problem that the UAV cannot accurately identify a throwing target during flight, and achieving the beneficial effect of improving the accuracy of UAV throwing.

[0073] Embodiment two

[0074] Figure 2a A flowchart of a method for determining a UAV throwing target provided by the second embodiment of the present application, which is a further refinement of how to identify a target throwing region in the to-be-identified region image in the above-mentioned embodiments. Optionally, the identification of the target throwing region in the to-be-identified region image comprises: performing gradient calculation on the to-be-identified region image to obtain a gradient image corresponding to the to-be-identified region image, wherein the gradient image comprises a gradient amplitude value of each pixel; determining at least one candidate throwing region based on the minimum gradient amplitude value and the maximum gradient amplitude value in a preset region in the to-be-identified region image, performing image segmentation on at least one candidate throwing region based on the image segmentation method to obtain a segmented candidate throwing region; for each segmented candidate throwing region, detecting the candidate throwing region to obtain a region identification index, and determining the target throwing region based on at least one candidate throwing region and the region identification index corresponding to the candidate throwing region.

[0075] As Figure 2a shown, the method comprises:

[0076] In S210, a to-be-throwing scene image collected by a UAV is acquired, and filtering processing is performed on the to-be-throwing scene image to obtain a to-be-throwing scene image after filtering processing.

[0077] In S220, interpolation processing and amplification processing are performed on the to-be-throwing scene image after filtering processing to obtain a to-be-extracted feature image, a target image feature in the to-be-extracted feature image is extracted, and contrast enhancement processing is performed on the target image feature to obtain a to-be-identified region image.

[0078] S230, gradient calculation is performed on the to-be-identified region image to obtain a gradient image corresponding to the to-be-identified region image, wherein the gradient image comprises a gradient amplitude value of each pixel.

[0079] The gradient amplitude value can be understood as a gradient amplitude, which is a measure of the gradient size and is used to quantify the local change rate of the pixel value in the image. For example, the greater the gradient amplitude value, the more intense the change of the pixel value at the position, which may be an edge, contour or texture feature significant region in the image.

[0080] Specifically, the gradient calculation on the to-be-identified region image is represented by the following formula:

[0081]

[0082] Wherein G(x,y) is the gradient amplitude of each pixel.

[0083] In the embodiment of the application, the gradient image generated by the gradient calculation provides edge information, which helps better region segmentation and target identification.

[0084] S240, determining at least one candidate throwing region based on the minimum gradient amplitude value and the maximum gradient amplitude value in the preset region in the to-be-identified region image, and performing image segmentation on the at least one candidate throwing region based on the image segmentation method to obtain a segmented candidate throwing region.

[0085] Specifically, local minimum and maximum values are found on the gradient image to mark at least one candidate throwing region, and morphological operation is used to generate markers of foreground and background; the gradient image is regarded as a terrain through target point conversion, wherein the brightness value represents the terrain height, and the candidate throwing region and other regions are segmented.

[0086] Specifically, the image segmentation on the at least one candidate throwing region based on the image segmentation method is represented by the following formula:

[0087]

[0088] Wherein CB i (G) represents the i-th candidate throwing region, represents the neighborhood of the pixel point (x,y), G(x,y) is the gradient amplitude of each pixel, and G(x',y') is the gradient amplitude of the neighborhood of the pixel point (x,y).

[0089] S250, for each segmented candidate throwing region, detecting the candidate throwing region to obtain a region identification index, and determining the target throwing region based on at least one candidate throwing region and the region identification index corresponding to the candidate throwing region.

[0090] Specifically, for each segmented candidate throwing area, the candidate throwing area is detected, the gradient intensity of the candidate throwing area is calculated, the mean and standard deviation of the gradient intensity are calculated, and the gradient intensity matching degree index is calculated according to the mean and standard deviation. The contour of the real target (which may be a template obtained through a preprocessing step) is obtained. The contour of the candidate area is calculated. The shape matching algorithm (such as Hausdorff distance, shape context, etc.) is used to calculate the matching degree between the contour of the candidate area and the contour of the real target. The contour matching degree index is obtained. The gradient intensity matching degree index and the contour matching degree index are normalized. The two normalized indexes are added to obtain the regional recognition index of each candidate area. The candidate area with the highest regional recognition index is selected as the target throwing area.

[0091] S260, determine a target throwing position based on the target throwing area, and control the unmanned aerial vehicle to throw the target object to the target throwing position.

[0092] The technical scheme of the embodiment of the application calculates the gradient of the to-be-recognized region image to obtain a gradient image corresponding to the to-be-recognized region image, wherein the gradient image includes a gradient amplitude value of each pixel; at least one candidate throwing area is determined based on a minimum gradient amplitude value and a maximum gradient amplitude value in a preset region in the to-be-recognized region image; the image segmentation method is used to perform image segmentation on at least one candidate throwing area to obtain a segmented candidate throwing area; for each segmented candidate throwing area, the candidate throwing area is detected to obtain a regional recognition index; and the target throwing area is determined based on at least one candidate throwing area and the regional recognition index corresponding to the candidate throwing area. By calculating the gradient image of the to-be-recognized region image, the speed of brightness or color change in the image can be captured, and the gradient amplitude value is used to determine the candidate throwing area, so that regions with no obvious gradient change and not matching the target feature can be excluded, thereby improving the accuracy of target throwing area recognition.

[0093] As an optional example of the embodiment of the application, the method for determining the unmanned aerial vehicle throwing target of the embodiment is applied to an unmanned aerial vehicle throwing target detection system, and specifically includes the following steps:

[0094] S1. An image data set containing an unmanned aerial vehicle throwing operation scene is collected and made, including throwing behavior images and throwing environment image information.

[0095] S2, construct an unmanned aerial vehicle throwing target image filtering UAVIF module, and perform filtering processing on the to-be-thrown scene image to obtain a filtered to-be-thrown scene image.

[0096] S3, construct a UAV throwing target image zoom BICI module, interpolate and enlarge the filtered scene image to be thrown to obtain a feature image to be extracted, and extract the target image features in the feature image to be extracted.

[0097] S4, construct a UAV throwing target feature extraction and contrast enhancement FECE module, and perform contrast enhancement processing on the target image features to obtain a region image to be identified.

[0098] S5, construct a UAV throwing target final detection THDET module to identify the target throwing region in the region image to be identified.

[0099] The various modules of the entire image processing flow are integrated into a complete UAV throwing target real-time detection system.

[0100] The UAVIF module, BICI module, FECE module and THDET module are integrated into a UAV throwing target detection system, so that the system can efficiently and accurately identify and locate the throwing target.

[0101] Further, the image data set of the UAV throwing operation scene collected in step S1 includes target images before throwing, images after accurate throwing, and image data of throwing deviation; The specific production method is to build multiple throwing targets on the test site, collect images and label information through the image transmission device of the UAV, and train the throwing target model through the subsequent designed feature extraction module.

[0102] Further, Figure 2b An image filtering enhanced sample schematic diagram of an optional example of a UAV throwing target determination method is provided. As shown in Figure 2b , Figure 2b (a) is a filtered scene image to be thrown, (b) is a feature image to be extracted, and (c) is a region image to be identified. The a image filtering part in the image filtering enhanced result schematic diagram is generated by the UAVIF module constructed in step S2, which eliminates noise in the image to ensure that the image for subsequent processing is clearer and more reliable. The process is realized through Python code, and the specific operation steps are as follows:

[0103] The noise model is pre-constructed to perform denoising processing on the scene image to be thrown, and filter boundary processing is performed on the denoised scene image to be thrown to obtain a filtered scene image to be thrown.

[0104] The noise model is expressed by the following formula:

[0105] s(x,y)=u(x,y)+n(x,y);

[0106] Among them, s(x,y) is the image of the scene to be thrown, u(x,y) is the noise-free image of the scene to be thrown, and n(x,y) is Gaussian noise.

[0107] Specifically, we choose to use zero-mean Gaussian noise and set the noise model to lay the foundation for subsequent filtering processing, providing a clear reference for noise parameters, thereby improving the denoising effect. Then, we design the window parameters of the filter. The filter window can be set to 3×3, which is expressed by the following formula:

[0108]

[0109] in, is the pixel value after filtering, Ω x,y is the window area centered at (x, y), the size is 3×3, and (i, j) is the pixel value.

[0110] The designed filter can significantly reduce the impact of noise on image quality and improve the visual clarity of the image. The 3x3 window size is moderate, which can effectively remove noise without causing excessive smoothing.

[0111] Finally, filter boundary processing is performed to fill the boundary area with mirrored pixels at the edge of the image, which can be expressed as follows:

[0112] S padded (x,y)=s(reflect(x),reflect(y));

[0113]

[0114] Among them, x edge and y edge are the boundary coordinates of the image in the x and y directions.

[0115] Specifically, the mirror filling ensures the stability of pixels in the boundary area, so that the filtering process can be carried out smoothly in the entire image domain, thereby preventing boundary pixels from being processed incorrectly or ignored.

[0116] Further, such as Figure 2b As shown, Figure 2b Image (b) is obtained by the BICI image scaling module constructed in step S3. It mainly uses interpolation algorithms to magnify the image to achieve higher resolution. This allows it to match the power grid throwing target at different distances and viewing angles, providing more detailed information for subsequent feature extraction and segmentation. The specific operation logic is as follows:

[0117] The filtered image is subjected to interpolation processing and magnification processing to obtain a feature image to be extracted, target image features in the feature image to be extracted are extracted, and the target image features are subjected to contrast enhancement processing to obtain a region image to be recognized.

[0118] Specifically, the filtered image to be thrown is subjected to interpolation processing and magnification processing to achieve higher resolution, which can match the throwing target under different distances and angles of view, and provide more detailed information for subsequent feature extraction and segmentation.

[0119] Specifically, given the original image U, new pixel points need to be inserted in the enlarged image U'.

[0120] Optionally, the filtered image to be thrown is subjected to interpolation processing by the following formula:

[0121]

[0122] wherein a ij is an interpolation coefficient, (x', y') is a new pixel point inserted, x' and y' are image coordinates, and (x'-x) and (y'-y) are relative positions of the interpolation pixel.

[0123] Specifically, the interpolation method can effectively smooth the image edge, so that the enlarged image is more natural and clear in vision, which is helpful for subsequent image analysis and feature extraction. In the specific calculation process of the interpolation coefficient, the surrounding pixel values and their first-order derivatives in the horizontal, vertical and diagonal directions are used, denoted as f x , f y and f xy . The formula for calculating the corresponding pixel (x, y) is as follows:

[0124]

[0125] Illustratively, a pixel point has 16 interpolation matrices A, and the new interpolation pixel value can be calculated by the following matrix formula:

[0126]

[0127] wherein the matrix The matrix A is composed of eigenvectors.

[0128] Optionally, the target image features in the feature image to be extracted are extracted, including determining a covariance matrix of the feature image to be extracted, and performing eigenvalue decomposition on the covariance matrix to obtain the target image features.

[0129] Exemplarily, eigenvalue decomposition is performed on the covariance matrix to obtain the target image feature, which is expressed by the following formula:

[0130]

[0131] K = A (U - μ) ;

[0132] wherein C x is a covariance matrix, y i is a column vector of an image to be extracted, μ is a mean vector, K represents an image including a target image feature, A is a matrix, and U is an image to be extracted. The matrix A is used to convert the image to be extracted U into the image K including the target image feature.

[0133] Optionally, the target image feature is subjected to a contrast enhancement process to obtain a region image to be recognized, including: performing a contrast enhancement process on the target image feature based on a cumulative distribution function to obtain the region image to be recognized.

[0134] Optionally, the target image feature is subjected to a contrast enhancement process based on a cumulative distribution function by the following formula:

[0135]

[0136] wherein T(r k ) is a cumulative distribution function, p(r i ) is a probability of a pixel with intensity r i , and h(r j ) is a pixel intensity frequency.

[0137] Further, Figure 2c An example of a sample schematic diagram of region identification and segmentation of a method for determining a target thrown by a UAV is provided. As Figure 2c shown, Figure 2c the target throwing region detection and segmentation results shown in two parts (a) and (b) in the figure, (c) is a schematic diagram of the target throwing region detection result. Figure 2c The target throwing region detection and segmentation results shown in two parts (a) and (b) in the figure are generated by the THDET module constructed in step S5, a high-adaptability algorithm is designed for the target difference in the power grid environment, and the signal region of the throwing target is accurately identified from the real-time image after the pre-processing; the specific operation is as follows:

[0138] Gradient calculation is performed on the region image to be recognized to obtain a gradient map corresponding to the region image to be recognized, wherein the gradient map includes a gradient amplitude value of each pixel.

[0139] The gradient amplitude value can be understood as a gradient amplitude, which is a measure of the gradient size, and is used to quantify the local change rate of the pixel value in the image. For example, the greater the gradient amplitude value, the more intense the change of the pixel value at the position, which can be an edge, contour or texture feature significant area in the image.

[0140] Specifically, the gradient calculation of the to-be-identified region image is represented by the following formula:

[0141]

[0142] Wherein, G(x,y) is the gradient amplitude of each pixel.

[0143] In the embodiment of the application, the gradient image generated by the gradient calculation provides edge information, which helps better region segmentation and target identification.

[0144] S240, determining at least one candidate throwing region based on the minimum gradient amplitude value and the maximum gradient amplitude value in the preset region in the to-be-identified region image, and performing image segmentation on at least one candidate throwing region based on the image segmentation method to obtain a segmented candidate throwing region.

[0145] Specifically, local minimum and maximum values are found on the gradient image to mark at least one candidate throwing region, and morphological operations are used to generate markers of foreground and background; the gradient image is regarded as a terrain through target point conversion, wherein the brightness value represents the terrain height, and the candidate throwing region and other regions are segmented.

[0146] For example, the image segmentation of at least one candidate throwing region based on the image segmentation method is represented by the following formula:

[0147]

[0148] Wherein, CB i (G) represents the i-th candidate throwing region, represents the neighborhood of the pixel point (x,y), and G(x,y) is the gradient amplitude of each pixel. G(x',y') is the gradient amplitude of the neighborhood of the pixel point (x,y).

[0149] S250, for each segmented candidate throwing region, detecting the candidate throwing region to obtain a region identification index, and determining the target throwing region based on at least one candidate throwing region and the region identification index corresponding to the candidate throwing region.

[0150] Figure 2d A sample effect schematic diagram of the target throwing position of an optional example of a method for determining a UAV throwing target is provided. As shown in Figure 2d the figure,Figure 2d (a) is a scene image to be thrown collected by a UAV, and (b) is an output image after a target throwing area in the image to be recognized is recognized.

[0151] The technical scheme of the embodiment of the present application has significant advantages in real-time, precision and robustness of UAV image processing. The modular design includes multiple steps such as noise filtering, image scaling, feature extraction and contrast enhancement, which ensures clear identification of the throwing target from a complex background. The recognition degree of the target feature is effectively improved, and the defect of poor recognition accuracy of the traditional method under low resolution and low contrast is overcome. The target throwing area is accurately segmented by using the watershed transformation, so that the UAV can accurately locate the target throwing position in a complex scene, and provides reliable technical support for automatic throwing and material delivery tasks.

[0152] Embodiment three

[0153] Figure 3 A structure diagram of a UAV throwing target determination device provided by the third embodiment of the present application is shown in FIG. 3. Figure 3 As shown in the figure, the device includes an image acquisition module 310, an image processing module 320 and a throwing position determination module 330.

[0154] The image acquisition module 310 is configured to acquire a scene image to be thrown collected by a UAV, and perform filtering processing on the scene image to be thrown to obtain a filtered scene image to be thrown. The image processing module 320 is configured to perform interpolation processing and magnification processing on the filtered scene image to be thrown to obtain a feature image to be extracted, extract a target image feature in the feature image to be extracted, and perform contrast enhancement processing on the target image feature to obtain a region image to be recognized. The throwing position determination module 330 is configured to recognize a target throwing area in the region image to be recognized, determine a target throwing position based on the target throwing area, and control the UAV to throw a target object to the target throwing position.

[0155] The technical scheme of the embodiment of the present application comprises the following steps: acquiring a to-be-launched scene image collected by a UAV, performing filtering processing on the to-be-launched scene image to obtain a to-be-launched scene image after filtering processing, improving the quality of the to-be-launched scene image, performing interpolation processing and magnification processing on the to-be-launched scene image after filtering processing to obtain a to-be-extracted feature image, extracting a target image feature in the to-be-extracted feature image, performing contrast enhancement processing on the target image feature to obtain a to-be-recognized region image, significantly reducing the amount of image data and highlighting the features of the image, making the originally low-contrast to-be-launched target region more easily recognized after processing, finally, recognizing a target to-be-launched region in the to-be-recognized region image, determining a target to-be-launched position based on the target to-be-launched region, and controlling the UAV to launch a target object to the target to-be-launched position, thereby solving the problem that the UAV cannot accurately recognize a to-be-launched target during flight, and achieving the beneficial effect of improving the accuracy of UAV launching.

[0156] Optionally, the image acquisition module comprises:

[0157] a filtering unit, configured to perform denoising processing on the to-be-launched scene image by using a pre-constructed noise model, and perform filter boundary processing on the to-be-launched scene image after denoising processing to obtain a to-be-launched scene image after filtering processing;

[0158] wherein the noise model is expressed by the following formula:

[0159] s(x, y) = u(x, y) + n(x, y);

[0160] wherein s(x, y) is the to-be-launched scene image, u(x, y) is a to-be-launched scene image without noise, and n(x, y) is Gaussian noise.

[0161] Optionally, the filtering unit is specifically configured to:

[0162] perform interpolation processing on the to-be-launched scene image after filtering processing by using the following formula:

[0163]

[0164] wherein a ij is an interpolation coefficient, (x', y') is a new pixel point inserted, x' and y' are image coordinates, and (x'-x) and (y'-y) are relative positions of the interpolation pixel.

[0165] Optionally, the image processing module is specifically configured to:

[0166] determine a covariance matrix of the to-be-extracted feature image, perform eigenvalue decomposition on the covariance matrix to obtain a target image feature.

[0167] Optionally, the image processing module is specifically used for:

[0168] performing contrast enhancement processing on the target image feature based on a cumulative distribution function to obtain the image of the to-be-identified region.

[0169] Optionally, the image processing module is specifically used for:

[0170] performing contrast enhancement processing on the target image feature based on a cumulative distribution function through the following formula:

[0171]

[0172] wherein, T(r k ) is a cumulative distribution function, p(r i ) is a probability of a pixel with intensity r i , and h(r j ) is a pixel intensity frequency.

[0173] Optionally, the throwing position determination module comprises:

[0174] a gradient calculation unit configured to perform gradient calculation on the image of the to-be-identified region to obtain a gradient map corresponding to the image of the to-be-identified region, wherein the gradient map comprises a gradient amplitude value of each pixel;

[0175] a candidate region determination unit configured to determine at least one candidate throwing region based on a minimum gradient amplitude value and a maximum gradient amplitude value in a preset region in the image of the to-be-identified region, and perform image segmentation on the at least one candidate throwing region based on the image segmentation method to obtain a segmented candidate throwing region;

[0176] a target region determination unit configured to, for each segmented candidate throwing region, perform detection on the candidate throwing region to obtain a region identification index, and determine the target throwing region based on the at least one candidate throwing region and the region identification index corresponding to the candidate throwing region.

[0177] The unmanned aerial vehicle throwing target determination device provided in the embodiments of the present application can execute the unmanned aerial vehicle throwing target determination method provided in any of the embodiments of the present application, and has the corresponding function modules and beneficial effects of the execution method.

[0178] Embodiment Four

[0179] Figure 4A schematic diagram of the structure of an electronic device 10 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or claimed herein.

[0180] like Figure 4 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., which is communicatively connected to the at least one processor 11. The memory stores a computer program that can be executed by the at least one processor. The processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 into the random access memory (RAM) 13. Various programs and data required for the operation of the electronic device 10 can also be stored in the RAM 13. The processor 11, ROM 12, and RAM 13 are connected to each other via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0181] Multiple components in the electronic device 10 are connected to the I / O interface 15, including an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a magnetic disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0182] The processor 11 can be any general-purpose and / or specialized processing component with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as the method for determining a drone drop target.

[0183] In some embodiments, the method of determining a drone drop-off target can be implemented as a computer program tangibly embodied in a computer readable storage medium, e.g., storage unit 18. In some embodiments, parts or all of the computer program can be loaded and / or installed onto electronic device 10 via, e.g., ROM 12 and / or communication unit 19. When the computer program is loaded onto RAM 13 and executed by processor 11, one or more steps of the method of determining a drone drop-off target as described above can be performed. Alternatively, in other embodiments, processor 11 can be configured to perform the method of determining a drone drop-off target by other means, e.g., with the aid of firmware.

[0184] Various implementations of the systems and techniques described above can be realized in digital electronic circuitry, integrated circuitry, a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on a chip (SOC), a programmable logic device (PLD), a computer hardware, firmware, software, and / or combinations thereof. These various implementations can include implementation in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.

[0185] Computer programs used to implement the processes of the present application can be written in any combination of one or more programming languages. These computer programs can be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the computer program, when executed by the processor, implements the functions / acts specified in the flowcharts and / or block diagrams. The computer program can be executed entirely on a machine, partially on a machine, partially on a machine as part of a standalone software package, partially on a machine and partially on a remote machine or entirely on a remote machine or server.

[0186] In the context of the present application, a computer-readable storage medium can be a tangible medium that can contain or store a computer program for use by or in connection with an instruction execution system, apparatus, or device. A computer-readable storage medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium can be a machine-readable signal medium. More specific examples of a machine-readable storage medium will include one or more lines of a program of instructions in a transitory signal, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0187] To provide for interaction with a service acquirer, the systems and techniques described here can be implemented on an electronic device having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the service acquirer and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the service acquirer can provide input to the electronic device. Other kinds of devices can be used to provide for interaction with a service acquirer; for example, feedback provided to the service acquirer can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the service acquirer can be received in any form, including acoustic, speech, or tactile input.

[0188] The systems and techniques described here can be implemented in a computing system that includes a back end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front end component (e.g., a client computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0189] The computing system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system, to solve the defects of large management difficulty and weak business scalability in traditional physical host and VPS service.

[0190] It should be understood that the various forms of flow shown above can be used to reorder, add or delete steps. For example, each step described in the present application can be executed in parallel, sequentially or in a different order, as long as the desired results of the technical solutions of the present application can be achieved, which is not limited herein.

[0191] The above detailed description does not constitute a limitation on the protection scope of the present application. Those skilled in the art should understand that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modifications, equivalent replacements and improvements made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A method for determining a drone throw target, characterized in that, The method comprises the following steps: acquire a to-be-dropped scene image collected by a UAV, and perform filtering processing on the to-be-dropped scene image to obtain a filtered to-be-dropped scene image; perform interpolation processing and magnification processing on the filtered to-be-dropped scene image to obtain a to-be-extracted feature image, extract a target image feature in the to-be-extracted feature image, and perform contrast enhancement processing on the target image feature to obtain a to-be-recognized region image; recognize a target dropping region in the to-be-recognized region image, determine a target dropping position based on the target dropping region, and control the UAV to drop a target object to the target dropping position; the step of recognizing the target dropping region in the to-be-recognized region image comprises: perform gradient calculation on the to-be-recognized region image to obtain a gradient image corresponding to the to-be-recognized region image, wherein the gradient image comprises a gradient amplitude value of each pixel; determine at least one candidate dropping region based on a minimum gradient amplitude value and a maximum gradient amplitude value in a preset region in the to-be-recognized region image, and perform image segmentation on at least one candidate dropping region based on an image segmentation method to obtain a segmented candidate dropping region; for each segmented candidate dropping region, perform detection on the candidate dropping region to obtain a region recognition index, and determine the target dropping region based on at least one candidate dropping region and the region recognition index corresponding to the candidate dropping region.

2. The method of claim 1, wherein, the step of performing filtering processing on the to-be-dropped scene image to obtain a filtered to-be-dropped scene image comprises: perform denoising processing on the to-be-dropped scene image through a pre-constructed noise model, and perform filter boundary processing on the to-be-dropped scene image after denoising processing to obtain a filtered to-be-dropped scene image; wherein the noise model is expressed by the following formula: ； in, is the scene image to be thrown, is a noise-free image of the scene to be thrown, is Gaussian noise.

3. The method of claim 1, wherein, perform interpolation processing on the filtered to-be-dropped scene image through the following formula: ； wherein, is an interpolation coefficient, is a new pixel point inserted, and is an image coordinate, and is a relative position of the interpolated pixel.

4. The method of claim 1, wherein, the step of extracting a target image feature in the to-be-extracted feature image comprises: determine a covariance matrix corresponding to the to-be-extracted feature image, perform eigenvalue decomposition on the covariance matrix to obtain a target image feature. the step of performing contrast enhancement processing on the target image feature to obtain a to-be-recognized region image comprises:

5. The method of claim 1, wherein, perform contrast enhancement processing on the target image feature based on a cumulative distribution function to obtain the to-be-recognized region image. perform contrast enhancement processing on the target image feature based on a cumulative distribution function through the following formula:

6. The method of claim 5, wherein, comprise: ； ； wherein, is the cumulative distribution function, is the probability of a pixel having intensity of 0, is the pixel intensity frequency.

7. A drone throw target determination apparatus, comprising: an image acquisition module, configured to acquire a to-be-dropped scene image collected by a UAV, and perform filtering processing on the to-be-dropped scene image to obtain a filtered to-be-dropped scene image; an image processing module, configured to perform interpolation processing and magnification processing on the filtered to-be-dropped scene image to obtain a to-be-extracted feature image, extract a target image feature in the to-be-extracted feature image, and perform contrast enhancement processing on the target image feature to obtain a to-be-recognized region image; ​ The throwing position determination module is configured to identify a target throwing area in the image of the to-be-identified area, determine a target throwing position based on the target throwing area, and control the UAV to throw a target object to the target throwing position. The throwing position determination module comprises: a gradient calculation unit configured to perform gradient calculation on the image of the to-be-identified area to obtain a gradient map corresponding to the image of the to-be-identified area, wherein the gradient map comprises a gradient amplitude value of each pixel; a candidate area determination unit configured to determine at least one candidate throwing area based on a minimum gradient amplitude value and a maximum gradient amplitude value in a preset area in the image of the to-be-identified area, and perform image segmentation on the at least one candidate throwing area based on an image segmentation method to obtain a segmented candidate throwing area; a target area determination unit configured to, for each segmented candidate throwing area, perform detection on the candidate throwing area to obtain an area recognition index, and determine the target throwing area based on the at least one candidate throwing area and the area recognition index corresponding to the candidate throwing area.

8. An electronic device, comprising: comprise: at least one processor; and a memory connected to the at least one processor in communication; wherein the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to execute the method for determining the throwing target of the UAV according to any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer readable storage medium stores computer instructions, and the computer instructions are used to enable the processor to implement the method for determining the throwing target of the UAV according to any one of claims 1-6 when executed.

Images