Method, device and equipment for determining throwing target of unmanned aerial vehicle, and storage medium

Abstract

The invention discloses an unmanned aerial vehicle throwing target determination method and device, equipment and a storage medium. The method comprises the steps of obtaining a to-be-thrown scene image collected by an unmanned aerial vehicle, and performing filtering processing on the to-be-thrown scene image to obtain the to-be-thrown scene image after filtering processing; performing interpolation processing and amplification processing on the to-be-thrown scene image after filtering processing to obtain a to-be-extracted feature image, extracting a target image feature in the to-be-extracted feature image, and performing contrast enhancement processing on the target image feature to obtain a to-be-recognized region image; and identifying a target throwing area in the to-be-identified area image, determining a target throwing position based on the target throwing area, and controlling the unmanned aerial vehicle to throw a target object to the target throwing position. And the throwing accuracy of the unmanned aerial vehicle is improved.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a method, device, equipment and storage medium for determining a target for a drone. Background Art

[0002] With the rapid development of drone technology, drones are increasingly used in image acquisition and processing, showing great potential in logistics, agriculture, and environmental monitoring.

[0003] However, due to the complex and changeable operating environment of UAVs, the acquired image data often has problems such as noise, low resolution, and difficulty in target identification, which leads to higher requirements for real-time recognition and positioning accuracy of throwing tasks; traditional image processing methods cannot meet the needs of UAV-specific tasks, such as accurately identifying throwing targets during flight. Summary of the invention

[0004] The present invention provides a method, device, equipment and storage medium for determining a throwing target of an unmanned aerial vehicle, so as to solve the problem that the unmanned aerial vehicle cannot accurately identify the throwing target during flight.

[0005] According to one aspect of the present invention, a method for determining a target for a drone is provided, the method comprising:

[0006] Acquire the image of the scene to be thrown collected by the drone, and perform filtering processing on the image of the scene to be thrown to obtain the filtered image of the scene to be thrown;

[0007] Performing interpolation and amplification processing on the filtered image of the scene to be thrown to obtain a feature image to be extracted, extracting target image features from the feature image to be extracted, and performing contrast enhancement processing on the target image features to obtain an image of the area to be identified;

[0008] A target throwing area in the image of the area to be identified is identified, a target throwing position is determined based on the target throwing area, and the drone is controlled to throw the target object to the target throwing position.

[0009] According to another aspect of the present invention, a device for determining a target for a drone is provided, the device comprising:

[0010] An image acquisition module is used to acquire an image of a scene to be thrown collected by a drone, and filter the image of the scene to be thrown to obtain a filtered image of the scene to be thrown;

[0011] An image processing module is used to perform interpolation processing and amplification processing on the image of the scene to be thrown after filtering to obtain a feature image to be extracted, extract the target image features in the feature image to be extracted, and perform contrast enhancement processing on the target image features to obtain an image of the area to be identified;

[0012] The throwing position determination module is used to identify the target throwing area in the image of the area to be identified, determine the target throwing position based on the target throwing area, and control the drone to throw the target object to the target throwing position.

[0013] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0014] at least one processor; and

[0015] a memory communicatively connected to the at least one processor; wherein,

[0016] The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the method for determining the drone throwing target described in any embodiment of the present invention.

[0017] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the method for determining a drone throwing target as described in any embodiment of the present invention when executed.

[0018] The technical solution of the embodiment of the present invention is to obtain the image of the scene to be thrown collected by the drone, filter the image of the scene to be thrown to obtain the filtered image of the scene to be thrown; improve the quality of the image of the scene to be thrown, and then interpolate and amplify the filtered image of the scene to be thrown to obtain the feature image to be extracted, extract the target image features in the feature image to be extracted, and perform contrast enhancement on the target image features to obtain the image of the area to be identified; significantly reduce the amount of image data and highlight the features of the image, so that the originally low-contrast throwing target area is easier to identify after processing, and finally, identify the target throwing area in the image of the area to be identified, determine the target throwing position based on the target throwing area, and control the drone to throw the target object to the target throwing position, thereby solving the problem that the drone cannot accurately identify the throwing target during flight and achieving the beneficial effect of improving the accuracy of drone throwing.

[0019] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0021] Figure 1 is a flow chart of a method for determining a target for a drone drop according to Embodiment 1 of the present invention;

[0022] Figure 2a is a flow chart of a method for determining a target for throwing a drone provided according to Embodiment 2 of the present invention;

[0023] Figure 2b is a schematic diagram of samples after image filtering enhancement according to an optional example of a method for determining a target for throwing a drone provided in Embodiment 2 of the present invention;

[0024] Figure 2c It is a sample schematic diagram of region recognition and segmentation of an optional example of a method for determining a target for throwing a drone provided in Embodiment 2 of the present invention;

[0025] Figure 2d is a schematic diagram of sample effects of target throwing positions of an optional example of a method for determining a target for throwing a drone provided in Embodiment 2 of the present invention;

[0026] Figure 3 2 is a schematic diagram of the structure of a device for determining a target for throwing a drone provided in accordance with Embodiment 3 of the present invention;

[0027] Figure 4 It is a structural schematic diagram of an electronic device for implementing the method for determining a target for throwing a drone according to an embodiment of the present invention. DETAILED DESCRIPTION

[0028] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0029] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0030] Embodiment 1

[0031] Figure 1 A flowchart of a method for determining a target for a drone throwing is provided for the first embodiment of the present invention. This embodiment is applicable to the case where a drone is performing a throwing task. The method can be performed by a device for determining a target for a drone throwing. The device for determining a target for a drone throwing can be implemented in the form of hardware and / or software. The device for determining a target for a drone throwing can be configured in an electronic device. Figure 1 As shown, the method includes:

[0032] S110, acquiring an image of a scene to be thrown collected by a drone, and performing filtering processing on the image of the scene to be thrown to obtain a filtered image of the scene to be thrown.

[0033] The to-be-thrown scene image can be understood as an image of a scene in which a throwing action is about to be performed captured by a drone.

[0034] Specifically, the image of the scene to be thrown is obtained through a camera or other image acquisition device carried by the drone, and the image of the scene to be thrown is filtered to eliminate noise in the image, ensuring that the image processed subsequently is clearer and more reliable.

[0035] Optionally, the filtering process of the scene image to be thrown to obtain the filtered scene image to be thrown includes: denoising the scene image to be thrown by using a pre-constructed noise model, and performing filter boundary processing on the denoised scene image to be thrown to obtain the filtered scene image to be thrown;

[0036] The noise model is expressed as follows:

[0037] s(x,y)=u(x,y)+n(x,y);

[0038] Among them, s(x, y) is the image of the scene to be thrown, u(x, y) is the noise-free image of the scene to be thrown, and n(x, y) is Gaussian noise.

[0039] Specifically, we choose to use zero-mean Gaussian noise and set the noise model to lay the foundation for subsequent filtering processing, so as to provide a clear reference for noise parameters, thereby improving the denoising effect. Then, we design the window parameters of the filter. The filter window can be set to 3×3, which can be expressed by the following formula:

[0040]

[0041] in, is the pixel value after filtering, Ω x,y is the window area centered at (x, y), the size is 3×3, and (i, j) is the pixel value.

[0042] The designed filter can significantly reduce the impact of noise on image quality and improve the visual clarity of the image. The 3x3 window size is moderate, which can effectively remove noise without causing excessive smoothing.

[0043] Finally, filter boundary processing is performed to fill the boundary area with mirrored pixels at the edge of the image, which can be expressed as follows:

[0044] S padded (x,y)=s(reflect(x),reflect(y));

[0045]

[0046]

[0047] Among them, x edge and edge are the boundary coordinates of the image in the x and y directions.

[0048] Specifically, the mirror filling ensures the stability of the pixels in the boundary area, so that the filtering process can be carried out smoothly in the entire image domain, thereby preventing the boundary pixels from being processed incorrectly or ignored.

[0049] S120, interpolating and amplifying the filtered image of the scene to be thrown to obtain a feature image to be extracted, extracting target image features from the feature image to be extracted, and contrast enhancing the target image features to obtain an image of the area to be identified.

[0050] Specifically, the filtered image of the scene to be thrown is interpolated and enlarged to achieve a higher resolution, which can match the throwing targets at different distances and viewing angles and provide more detailed information for subsequent feature extraction and segmentation.

[0051] Optionally, the filtered image of the scene to be thrown is interpolated using the following formula:

[0052]

[0053] Among them, a ij is the interpolation coefficient, (x′, y′) is the new pixel to be inserted, x′ and y′ are the image coordinates, (x′-x) and (y′-y) are the relative positions of the interpolated pixels.

[0054] Specifically, the interpolation method can effectively smooth the image edge, making the enlarged image more natural and clear visually, which is helpful for subsequent image analysis and feature extraction. In the specific calculation process of the interpolation coefficient, the surrounding pixel values ​​and their first-order derivatives in the horizontal, vertical and diagonal directions are first used, denoted by f x 、f y and f xy The specific corresponding pixel (x, y) calculation formula is as follows:

[0055]

[0056] For example, a pixel point has 16 interpolation matrix A, and the new interpolation pixel value can be calculated by the following matrix formula:

[0057]

[0058] Among them, the matrix The matrix A consists of eigenvectors.

[0059] Optionally, extracting the target image features from the feature image to be extracted includes: determining a covariance matrix with the feature image to be extracted, and performing eigenvalue decomposition on the covariance matrix to obtain the target image features.

[0060] Exemplarily, the covariance matrix is ​​subjected to eigenvalue decomposition to obtain target image features expressed by the following formula:

[0061]

[0062] K = A(U-μ);

[0063] Among them, C x is the covariance matrix, y i is the column vector of the feature image to be extracted, μ is the mean vector, K represents the image including the features of the target image, A is a matrix, and U is the feature image to be extracted. Matrix A is used to transform the feature image U to be extracted into the feature image K containing the features of the target image.

[0064] Optionally, performing contrast enhancement processing on the target image features to obtain the image of the area to be identified includes: performing contrast enhancement processing on the target image features based on a cumulative distribution function to obtain the image of the area to be identified.

[0065] Optionally, contrast enhancement processing is performed on the target image feature based on the cumulative distribution function using the following formula:

[0066]

[0067] Among them, T(r k ) is the cumulative distribution function, p(r i ) is the strength r i The probability of a pixel, h(r j ) is the pixel intensity frequency.

[0068] In this embodiment, by extracting the target image features, the image data volume is significantly reduced and the target point features are highlighted. The target image features are contrast enhanced by the cumulative distribution function, making the target image features more recognizable in the feature space, and facilitating the separation of the target point from the surrounding area. The contrast enhancement part enhances the image contrast by optimizing the histogram equalization, making the target image features more obvious in the image, and adjusting the image brightness distribution, making the originally low-contrast throwing area easier to identify after processing.

[0069] S130, identifying a target throwing area in the image of the area to be identified, determining a target throwing position based on the target throwing area, and controlling the drone to throw the target object to the target throwing position.

[0070] The target throwing area can be understood as a specific area or location where the drone needs to throw the object. The target object can be understood as an object that the drone needs to carry and throw.

[0071] Specifically, at least one throwing position is determined within the identified target throwing area. This can be determined based on the center, edge, and specific marking points of the area. The throwing position coordinates in the image are converted into actual coordinates in the drone navigation system, and the flight path of the drone is planned according to the actual coordinates. When the drone approaches the target throwing position, the flight altitude, speed, and direction of the drone are controlled to ensure accurate throwing of the target object. The throwing mechanism of the drone is triggered at the appropriate time to throw the target object to the target position. Optionally, during the throwing process, the status of the drone and the accuracy of the target position are continuously monitored. If deviations or abnormalities are found in the throwing, the flight path or throwing strategy of the drone is adjusted in time.

[0072] The technical solution of the embodiment of the present invention is to obtain the image of the scene to be thrown collected by the drone, filter the image of the scene to be thrown to obtain the filtered image of the scene to be thrown; improve the quality of the image of the scene to be thrown, and then interpolate and amplify the filtered image of the scene to be thrown to obtain the feature image to be extracted, extract the target image features in the feature image to be extracted, and perform contrast enhancement on the target image features to obtain the image of the area to be identified; significantly reduce the amount of image data and highlight the features of the image, so that the originally low-contrast throwing target area is easier to identify after processing, and finally, identify the target throwing area in the image of the area to be identified, determine the target throwing position based on the target throwing area, and control the drone to throw the target object to the target throwing position, thereby solving the problem that the drone cannot accurately identify the throwing target during flight and achieving the beneficial effect of improving the accuracy of drone throwing.

[0073] Embodiment 2

[0074] Figure 2a A flowchart of a method for determining a target for drone throwing provided in the second embodiment of the present invention, this embodiment is a further refinement of how to identify the target throwing area in the image of the area to be identified in the above embodiment. Optionally, the identification of the target throwing area in the image of the area to be identified includes: performing gradient calculation on the image of the area to be identified to obtain a gradient map corresponding to the image of the area to be identified, wherein the gradient map includes a gradient amplitude value for each pixel; determining at least one candidate throwing area based on the minimum gradient amplitude value and the maximum gradient amplitude value in a preset area in the image of the area to be identified, performing image segmentation on at least one of the candidate throwing areas based on the image segmentation method to obtain a segmented candidate throwing area; for each segmented candidate throwing area, detecting the candidate throwing area to obtain an area recognition index, and determining the target throwing area based on at least one of the candidate throwing areas and an area recognition index corresponding to the candidate throwing area.

[0075] like Figure 2a As shown, the method includes:

[0076] S210, acquiring an image of the scene to be thrown collected by a drone, and performing filtering processing on the image of the scene to be thrown to obtain a filtered image of the scene to be thrown.

[0077] S220, interpolating and amplifying the filtered image of the scene to be thrown to obtain a feature image to be extracted, extracting target image features from the feature image to be extracted, and contrast enhancing the target image features to obtain an image of the area to be identified.

[0078] S230, performing gradient calculation on the image of the area to be identified to obtain a gradient map corresponding to the image of the area to be identified, wherein the gradient map includes a gradient amplitude value of each pixel.

[0079] The gradient magnitude value can be understood as the gradient magnitude, which is a measure of the gradient size and is used to quantify the local rate of change of pixel values ​​in an image. For example, the larger the gradient magnitude value, the more dramatic the change in pixel values ​​at that location, which may be an area with significant edge, contour or texture features in the image.

[0080] Specifically, the gradient calculation of the image of the area to be identified is expressed by the following formula:

[0081]

[0082] Among them, G(x,y) is the gradient magnitude of each pixel.

[0083] In the embodiment of the present invention, the gradient image generated by gradient calculation provides edge information, which helps to better perform region segmentation and target recognition.

[0084] S240. Determine at least one candidate throwing area based on the minimum gradient amplitude value and the maximum gradient amplitude value in a preset area in the image of the area to be identified, and perform image segmentation on at least one of the candidate throwing areas based on the image segmentation method to obtain a segmented candidate throwing area.

[0085] Specifically, local minima and maxima are found on the gradient map to mark at least one candidate throwing area, and morphological operations are used to generate foreground and background markers; the gradient image is regarded as terrain through target point transformation, where the brightness value represents the terrain height, and the candidate throwing area and other areas are segmented.

[0086] Exemplarily, image segmentation of at least one of the candidate throwing areas based on the image segmentation method is expressed by the following formula:

[0087]

[0088] Among them, CB i (G) represents the i-th candidate throwing area, represents the neighborhood of the pixel point (x, y), G(x, y) is the gradient magnitude of each pixel, and G(x′, y′) is the gradient magnitude of the neighborhood of the pixel point (x, y).

[0089] S250. For each segmented candidate throwing area, detect the candidate throwing area to obtain an area recognition index, and determine the target throwing area based on at least one of the candidate throwing areas and the area recognition index corresponding to the candidate throwing area.

[0090] Specifically, for each segmented candidate throwing area, the candidate throwing area is detected, the gradient strength of the candidate throwing area is calculated, the mean and standard deviation of the gradient strength are calculated, and the gradient strength matching index is calculated according to the mean and standard deviation. The contour of the real target is obtained (it may be a template obtained through the preprocessing step). The contour of the candidate area is calculated. A shape matching algorithm (such as Hausdorff distance, shape context, etc.) is used to calculate the matching degree between the candidate area contour and the real target contour. In order to obtain the contour matching index, the gradient strength matching index and the contour matching index are normalized. The two normalized indices are added to obtain the highest region recognition index for each candidate area. The candidate area with the highest region recognition index is selected as the target throwing area.

[0091] S260: Determine a target throwing position based on the target throwing area, and control the drone to throw the target object to the target throwing position.

[0092] The technical solution of the embodiment of the present invention performs gradient calculation on the image of the area to be identified to obtain a gradient map corresponding to the image of the area to be identified, wherein the gradient map includes the gradient amplitude value of each pixel; at least one candidate throwing area is determined based on the minimum gradient amplitude value and the maximum gradient amplitude value in the preset area of ​​the image of the area to be identified, and at least one of the candidate throwing areas is image segmented based on the image segmentation method to obtain the segmented candidate throwing area; for each segmented candidate throwing area, the candidate throwing area is detected to obtain a region identification index, and the target throwing area is determined based on at least one candidate throwing area and the region identification index corresponding to the candidate throwing area. By calculating the gradient map of the image of the area to be identified, the speed of brightness or color changes in the image can be captured, and the candidate throwing area is determined by using the gradient amplitude value, and those areas with unclear gradient changes and inconsistent with the target features can be excluded, thereby improving the accuracy of identifying the target throwing area.

[0093] As an optional example of the first embodiment of the present invention, the method for determining a drone throwing target in this embodiment is applied to a drone throwing target detection system, and the method for determining a drone throwing target specifically includes the following steps:

[0094] S1. Collect and create an image dataset containing UAV throwing operation scenes, including throwing behavior images and throwing environment image information.

[0095] S2. Constructing a UAV throwing target image filtering UAVIF module to filter the to-be-thrown scene image to obtain a filtered to-be-thrown scene image.

[0096] S3. Construct a BICI module for scaling the target image of a drone, perform interpolation and amplification processing on the filtered image of the scene to be thrown to obtain a feature image to be extracted, and extract the target image features in the feature image to be extracted.

[0097] S4. Construct a UAV target feature extraction and contrast enhancement FECE module to perform contrast enhancement processing on the target image features to obtain an image of the area to be identified.

[0098] S5. Construct a UAV throwing target final detection THDET module to identify the target throwing area in the image of the area to be identified.

[0099] The various modules of the entire image processing process are integrated into a complete real-time detection system for drone dropped targets.

[0100] The UAVIF module, BICI module, FECE module and THDET module are integrated into a UAV throwing target detection system, so that the system can efficiently and accurately identify and locate the throwing targets.

[0101] Furthermore, the image data set of the drone throwing operation scene collected in step S1 includes the target image before throwing, the image after accurate throwing, and the image data of throwing deviation; the specific production method is to build multiple throwing targets on site in the test site, collect images and annotate information through the drone's own image transmission equipment, and train the throwing target model through the feature extraction module designed subsequently.

[0102] Furthermore, Figure 2b A sample schematic diagram of image filtering enhancement is provided for an optional example of a method for determining a target for a drone drop. Figure 2b As shown, Figure 2b (a) is the image of the scene to be thrown after filtering, (b) is the feature image to be extracted, and (c) is the image of the area to be identified. The filtering part of the image a after the image filtering and enhancement is generated by the UAVIF module constructed in step S2. By eliminating the noise in the image, the image to be processed later is clearer and more reliable. The process is implemented through Python code. The specific steps are as follows:

[0103] De-noising the image of the scene to be thrown by using a pre-constructed noise model, and performing filter boundary processing on the de-noised image of the scene to be thrown to obtain a filtered image of the scene to be thrown;

[0104] The noise model is expressed as follows:

[0105] s(x,y)=u(x,y)+n(x,y);

[0106] Among them, s(x, y) is the image of the scene to be thrown, u(x, y) is the noise-free image of the scene to be thrown, and n(x, y) is Gaussian noise.

[0107] Specifically, we choose to use zero-mean Gaussian noise and set the noise model to lay the foundation for subsequent filtering processing, so as to provide a clear reference for noise parameters, thereby improving the denoising effect. Then, we design the window parameters of the filter. The filter window can be set to 3×3, which can be expressed by the following formula:

[0108]

[0109] in, is the pixel value after filtering, Ω x,y is the window area centered at (x, y), the size is 3×3, and (i, j) is the pixel value.

[0110] The designed filter can significantly reduce the impact of noise on image quality and improve the visual clarity of the image. The 3x3 window size is moderate, which can effectively remove noise without causing excessive smoothing.

[0111] Finally, filter boundary processing is performed to fill the boundary area with mirrored pixels at the edge of the image, which can be expressed as follows:

[0112] S padded (x,y)=s(reflect(x),reflect(y));

[0113]

[0114] Among them, x edge and edge are the boundary coordinates of the image in the x and y directions.

[0115] Specifically, the mirror filling ensures the stability of the pixels in the boundary area, so that the filtering process can be carried out smoothly in the entire image domain, thereby preventing the boundary pixels from being processed incorrectly or ignored.

[0116] Further, such as Figure 2b As shown, Figure 2b The middle image (b) is obtained by the BICI image scaling module constructed in step S3. The image is enlarged mainly through the interpolation algorithm to achieve a higher resolution, which can match the power grid throwing targets at different distances and viewing angles, and provide more detailed information for subsequent feature extraction and segmentation. The specific operation logic is as follows:

[0117] The filtered image of the scene to be thrown is interpolated and amplified to obtain a feature image to be extracted, and the target image features in the feature image to be extracted are extracted. The target image features are contrast enhanced to obtain an image of the area to be identified.

[0118] Specifically, the filtered image of the scene to be thrown is interpolated and enlarged to achieve a higher resolution, which can match the throwing targets at different distances and viewing angles and provide more detailed information for subsequent feature extraction and segmentation.

[0119] Specifically, given an original image U, new pixels need to be inserted into the enlarged image U'.

[0120] Optionally, the filtered image of the scene to be thrown is interpolated using the following formula:

[0121]

[0122] Among them, a ij is the interpolation coefficient, (x′, y′) is the new pixel to be inserted, x′ and y′ are the image coordinates, (x′-x) and (y′-y) are the relative positions of the interpolated pixels.

[0123] Specifically, the interpolation method can effectively smooth the image edge, making the enlarged image more natural and clear visually, which is helpful for subsequent image analysis and feature extraction. In the specific calculation process of the interpolation coefficient, the surrounding pixel values ​​and their first-order derivatives in the horizontal, vertical and diagonal directions are first used, denoted by f x 、f y and f xy The specific corresponding pixel (x, y) calculation formula is as follows:

[0124]

[0125] For example, a pixel point has 16 interpolation matrix A, and the new interpolation pixel value can be calculated by the following matrix formula:

[0126]

[0127] Among them, the matrix The matrix A consists of eigenvectors.

[0128] Optionally, extracting the target image features from the feature image to be extracted includes: determining a covariance matrix with the feature image to be extracted, and performing eigenvalue decomposition on the covariance matrix to obtain the target image features.

[0129] Exemplarily, the covariance matrix is ​​subjected to eigenvalue decomposition to obtain target image features expressed by the following formula:

[0130]

[0131] K = A(U-μ);

[0132] Among them, C x is the covariance matrix, y i is the column vector of the feature image to be extracted, μ is the mean vector, K represents the image including the features of the target image, A is a matrix, and U is the feature image to be extracted. Matrix A is used to transform the feature image U to be extracted into the feature image K containing the features of the target image.

[0133] Optionally, performing contrast enhancement processing on the target image features to obtain the image of the area to be identified includes: performing contrast enhancement processing on the target image features based on a cumulative distribution function to obtain the image of the area to be identified.

[0134] Optionally, contrast enhancement processing is performed on the target image feature based on the cumulative distribution function using the following formula:

[0135]

[0136] Among them, T(r k ) is the cumulative distribution function, p(r i ) is the intensity r i The probability of a pixel, h(r j ) is the pixel intensity frequency.

[0137] Furthermore, Figure 2c A sample schematic diagram of region recognition and segmentation of an optional example of a method for determining a target for a drone is provided. Figure 2c As shown, Figure 2c Parts (a) and (b) show the target throwing area detection and segmentation results, and part (c) is a schematic diagram of the target throwing area detection results. Figure 2c The target throwing area detection and segmentation results shown in parts (a) and (b) are generated by the THDET module constructed in step S5. A highly adaptable algorithm is designed for the target differences in the power grid environment to accurately identify the signal area of ​​the throwing target from the real-time image after the previous processing; the specific operations are:

[0138] A gradient calculation is performed on the image of the area to be identified to obtain a gradient map corresponding to the image of the area to be identified, wherein the gradient map includes a gradient amplitude value of each pixel.

[0139] The gradient magnitude value can be understood as the gradient magnitude, which is a measure of the gradient size and is used to quantify the local rate of change of pixel values ​​in an image. For example, the larger the gradient magnitude value, the more dramatic the change in pixel values ​​at that location, which may be an area with significant edge, contour or texture features in the image.

[0140] Specifically, the gradient calculation of the image of the area to be identified is expressed by the following formula:

[0141]

[0142] Among them, G(x,y) is the gradient magnitude of each pixel.

[0143] In the embodiment of the present invention, the gradient image generated by gradient calculation provides edge information, which helps to better perform region segmentation and target recognition.

[0144] S240. Determine at least one candidate throwing area based on the minimum gradient amplitude value and the maximum gradient amplitude value in a preset area in the image of the area to be identified, and perform image segmentation on at least one of the candidate throwing areas based on the image segmentation method to obtain a segmented candidate throwing area.

[0145] Specifically, local minima and maxima are found on the gradient map to mark at least one candidate throwing area, and morphological operations are used to generate foreground and background markers; the gradient image is regarded as terrain through target point transformation, where the brightness value represents the terrain height, and the candidate throwing area and other areas are segmented.

[0146] Exemplarily, image segmentation of at least one of the candidate throwing areas based on the image segmentation method is expressed by the following formula:

[0147]

[0148] Among them, CB i (G) represents the i-th candidate throwing area, represents the neighborhood of the pixel point (x, y), G(x, y) is the gradient magnitude of each pixel, and G(x′, y′) is the gradient magnitude of the neighborhood of the pixel point (x, y).

[0149] S250. For each segmented candidate throwing area, detect the candidate throwing area to obtain an area recognition index, and determine the target throwing area based on at least one of the candidate throwing areas and the area recognition index corresponding to the candidate throwing area.

[0150] Figure 2d A sample effect diagram of the target throwing position of an optional example of a method for determining a target for a drone throwing is provided. Figure 2d As shown, Figure 2d (a) is an image of the scene to be thrown captured by the drone, and (b) is an output image after identifying the target throwing area in the image of the area to be identified.

[0151] The technical solution of the embodiment of the present invention has significant advantages in the real-time, accuracy and robustness of drone image processing. Its modular design includes multiple steps such as noise filtering, image scaling, feature extraction and contrast enhancement, which ensures clear identification of throwing targets from complex backgrounds; effectively improves the recognition of target features, and overcomes the defects of traditional methods with poor recognition accuracy under low resolution and low contrast conditions; uses watershed transformation to accurately segment the target throwing area, so that the drone can accurately locate the target throwing position in complex scenes, providing reliable technical support for tasks such as automated throwing and material delivery.

[0152] Embodiment 3

[0153] Figure 3 This is a schematic diagram of the structure of a device for determining a target for a drone drop provided by Embodiment 3 of the present invention. Figure 3 As shown, the device includes: an image acquisition module 310, an image processing module 320 and a throwing position determination module 330.

[0154] Among them, the image acquisition module 310 is used to acquire the image of the scene to be thrown collected by the drone, and filter the image of the scene to be thrown to obtain the filtered image of the scene to be thrown; the image processing module 320 is used to interpolate and amplify the filtered image of the scene to be thrown to obtain a feature image to be extracted, extract the target image features in the feature image to be extracted, and perform contrast enhancement on the target image features to obtain an image of the area to be identified; the throwing position determination module 330 is used to identify the target throwing area in the image of the area to be identified, determine the target throwing position based on the target throwing area, and control the drone to throw the target object to the target throwing position.

[0155] The technical solution of the embodiment of the present invention is to obtain the image of the scene to be thrown collected by the drone, filter the image of the scene to be thrown to obtain the filtered image of the scene to be thrown; improve the quality of the image of the scene to be thrown, and then interpolate and amplify the filtered image of the scene to be thrown to obtain the feature image to be extracted, extract the target image features in the feature image to be extracted, and perform contrast enhancement on the target image features to obtain the image of the area to be identified; significantly reduce the amount of image data and highlight the features of the image, so that the originally low-contrast throwing target area is easier to identify after processing, and finally, identify the target throwing area in the image of the area to be identified, determine the target throwing position based on the target throwing area, and control the drone to throw the target object to the target throwing position, thereby solving the problem that the drone cannot accurately identify the throwing target during flight and achieving the beneficial effect of improving the accuracy of drone throwing.

[0156] Optionally, the image acquisition module includes:

[0157] A filtering unit, used for performing denoising processing on the to-be-thrown scene image by using a pre-constructed noise model, and performing filter boundary processing on the to-be-thrown scene image after denoising processing, so as to obtain the to-be-thrown scene image after filtering processing;

[0158] The noise model is expressed as follows:

[0159] s(x,y)=u(x,y)+n(x,y);

[0160] Among them, s(x, y) is the image of the scene to be thrown, u(x, y) is the noise-free image of the scene to be thrown, and n(x, y) is Gaussian noise.

[0161] Optionally, the filtering unit is specifically used for:

[0162] The filtered image of the scene to be thrown is interpolated using the following formula:

[0163]

[0164] Among them, a ij is the interpolation coefficient, (x′, y′) is the new pixel to be inserted, x′ and y′ are the image coordinates, (x′-x) and (y′-y) are the relative positions of the interpolated pixels.

[0165] Optionally, the image processing module is specifically used to:

[0166] Determine a covariance matrix with the feature image to be extracted, and perform eigenvalue decomposition on the covariance matrix to obtain target image features.

[0167] Optionally, the image processing module is specifically used to:

[0168] The target image features are subjected to contrast enhancement processing based on a cumulative distribution function to obtain the image of the area to be identified.

[0169] Optionally, the image processing module is specifically used to:

[0170] The contrast enhancement process of the target image feature is performed based on the cumulative distribution function by the following formula:

[0171]

[0172] Among them, T(r k ) is the cumulative distribution function, p(r i ) is the intensity r i The probability of a pixel, h(r j ) is the pixel intensity frequency.

[0173] Optionally, the throwing position determination module includes:

[0174] A gradient calculation unit, used for performing gradient calculation on the image of the area to be identified to obtain a gradient map corresponding to the image of the area to be identified, wherein the gradient map includes a gradient amplitude value of each pixel;

[0175] A candidate area determination unit, configured to determine at least one candidate throwing area based on the minimum gradient amplitude value and the maximum gradient amplitude value in a preset area in the image of the area to be identified, and perform image segmentation on at least one of the candidate throwing areas based on the image segmentation method to obtain a segmented candidate throwing area;

[0176] The target area determination unit is used to detect each segmented candidate throwing area to obtain an area recognition index, and determine the target throwing area based on at least one candidate throwing area and the area recognition index corresponding to the candidate throwing area.

[0177] The device for determining a target for throwing a drone provided in an embodiment of the present invention can execute the method for determining a target for throwing a drone provided in any embodiment of the present invention, and has functional modules and beneficial effects corresponding to the execution method.

[0178] Embodiment 4

[0179] Figure 4A schematic diagram of the structure of an electronic device 10 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0180] like Figure 4 As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0181] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0182] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The processor 11 executes the various methods and processes described above, such as the determination of the method for the drone to drop a target.

[0183] In some embodiments, the method for determining the target of a drone drop can be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the method for determining the target of a drone drop described above can be performed. Alternatively, in other embodiments, the processor 11 can be configured to perform the method for determining the target of a drone drop by any other suitable means (e.g., by means of firmware).

[0184] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0185] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0186] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0187] To provide interaction with a service acquirer, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the service acquirer; and a keyboard and a pointing device (e.g., a mouse or a trackball), through which the service acquirer can provide input to the electronic device. Other types of devices may also be used to provide interaction with a service acquirer; for example, the feedback provided to the service acquirer may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the service acquirer may be received in any form (including acoustic input, voice input, or tactile input).

[0188] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a service acquirer computer having a graphical service acquirer interface or a web browser through which the service acquirer can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0189] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0190] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0191] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for determining a target for a drone, characterized in that: include: Acquire the image of the scene to be thrown collected by the drone, and perform filtering processing on the image of the scene to be thrown to obtain the filtered image of the scene to be thrown; Performing interpolation and amplification processing on the filtered image of the scene to be thrown to obtain a feature image to be extracted, extracting target image features from the feature image to be extracted, and performing contrast enhancement processing on the target image features to obtain an image of the area to be identified; A target throwing area in the image of the area to be identified is identified, a target throwing position is determined based on the target throwing area, and the drone is controlled to throw the target object to the target throwing position.

2. The method according to claim 1, characterized in that The filtering process is performed on the to-be-thrown scene image to obtain the filtered to-be-thrown scene image, comprising: De-noising the image of the scene to be thrown by using a pre-constructed noise model, and performing filter boundary processing on the de-noised image of the scene to be thrown to obtain a filtered image of the scene to be thrown; The noise model is expressed as follows: s(x,y)=u(x,y)+n(x,y); Among them, s(x, y) is the image of the scene to be thrown, u(x, y) is the noise-free image of the scene to be thrown, and n(x, y) is Gaussian noise.

3. The method according to claim 1, characterized in that The filtered image of the scene to be thrown is interpolated using the following formula: Among them, a ij is the interpolation coefficient, (x′, y′) is the new pixel to be inserted, x′ and y′ are the image coordinates, (x′-x) and (y′-y) are the relative positions of the interpolated pixels.

4. The method according to claim 1, characterized in that The step of extracting target image features from the feature image to be extracted comprises: Determine a covariance matrix with the feature image to be extracted, and perform eigenvalue decomposition on the covariance matrix to obtain target image features.

5. The method according to claim 1, characterized in that The step of performing contrast enhancement processing on the target image features to obtain an image of the area to be identified includes: The target image features are subjected to contrast enhancement processing based on a cumulative distribution function to obtain the image of the area to be identified.

6. The method according to claim 5, characterized in that The contrast enhancement process of the target image feature is performed based on the cumulative distribution function by the following formula: Among them, T(r k ) is the cumulative distribution function, p(r i ) is the intensity r i The probability of a pixel, h(r j ) is the pixel intensity frequency.

7. The method according to claim 1, characterized in that The identifying the target throwing area in the to-be-identified area image includes: Performing gradient calculation on the image of the area to be identified to obtain a gradient map corresponding to the image of the area to be identified, wherein the gradient map includes a gradient amplitude value of each pixel; Determine at least one candidate throwing area based on the minimum gradient amplitude value and the maximum gradient amplitude value in a preset area in the image of the area to be identified, and perform image segmentation on at least one of the candidate throwing areas based on the image segmentation method to obtain a segmented candidate throwing area; For each segmented candidate throwing area, the candidate throwing area is detected to obtain an area recognition index, and the target throwing area is determined based on at least one of the candidate throwing areas and the area recognition index corresponding to the candidate throwing area.

8. A device for determining a target for a drone, characterized in that: include: An image acquisition module is used to acquire an image of a scene to be thrown collected by a drone, and filter the image of the scene to be thrown to obtain a filtered image of the scene to be thrown; An image processing module is used to perform interpolation processing and amplification processing on the image of the scene to be thrown after filtering to obtain a feature image to be extracted, extract the target image features in the feature image to be extracted, and perform contrast enhancement processing on the target image features to obtain an image of the area to be identified; The throwing position determination module is used to identify the target throwing area in the image of the area to be identified, determine the target throwing position based on the target throwing area, and control the drone to throw the target object to the target throwing position.

9. An electronic device, characterized in that: include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the method for determining a target for dropping a drone according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the method for determining a drone throwing target according to any one of claims 1 to 8 when executed.

Images