A method for joint recognition of images and sounds of birds invading power transmission lines

Abstract

The present invention relates to the field of image recognition technology, and in particular to a method for jointly identifying images and sounds of birds invading power transmission lines. Transmission line image data and sound signals are obtained, and the edges of the transmission lines are screened out in the image data by the length of the edge lines and the degree of regularity. Next, image blocks with dense edge pixels are identified on the edges of the transmission lines as suspected bird areas, which are more likely to represent the locations where birds stay. The color concentration of the suspected bird areas and the degree of proximity of the locations to the visual center are considered to determine the significant values ​​of their bird features. In order to more accurately capture the characteristics of birds, the images are processed at multiple scales, and the feature weights of each suspected bird area are obtained to optimize the target detection algorithm, and then target detection is performed on real-time image data. Finally, the image detection results are combined with the sound signals, and the sound features are used to further verify and confirm the bird species, thereby improving the comprehensiveness and accuracy of bird recognition.

Description

Technical Field

[0001] The present invention relates to the technical field of image recognition, and in particular to a method for jointly recognizing images and sounds of birds invading power transmission lines. Background Art

[0002] With the widespread deployment of power networks, transmission lines have become an important infrastructure for maintaining power supply in modern society. However, transmission lines often become a place for birds to roost or migrate. Bird activities can cause problems such as short circuits and disconnections in transmission lines, which will seriously affect the stable operation of the power system. Therefore, it is necessary to drive away birds that stay on transmission lines. Since different birds often respond differently to driving signals, it is particularly important to classify and identify birds that stay on transmission lines in order to improve the effectiveness of driving.

[0003] When classifying and identifying birds on transmission lines, target detection algorithms are usually used to identify targets on image data. During the training process of the target detection algorithm, regularization techniques are used on the model weights to suppress overfitting of the model, thereby improving the generalization of the model. However, in the process of identifying birds in images, due to the complexity of the information in the images, although traditional regularization methods can effectively prevent overfitting, they cannot fully utilize the differential contributions of different regions in the image to the bird identification task. Therefore, if the weights of the corresponding features of different regions are not dynamically adjusted, important regions will not be highlighted, resulting in low recognition accuracy. Summary of the invention

[0004] In order to solve the problem of complex information in images, although the traditional regularization method can effectively prevent overfitting, it cannot fully utilize the differential contributions of different regions in the image to the bird recognition task. Therefore, if the weights of the corresponding features of different regions are not dynamically adjusted, important regions will not be highlighted, resulting in a technical problem of low recognition accuracy. The purpose of the present invention is to provide a method for joint recognition of images and sounds of birds invading power lines. The technical scheme adopted is as follows:

[0005] Acquire a historical RGB image and a real-time RGB image of the power transmission line; determine the edge line of the power transmission line in the historical RGB image;

[0006] Determine the image block corresponding to each edge pixel point on the edge line of each transmission line; based on the difference in the number of edge pixels in different image blocks, screen out suspected bird areas in all image blocks;

[0007] In each suspected bird area, the bird feature significance value of the suspected bird area is determined based on the position distribution of the pixel points in the historical RGB image and the color distribution disorder; based on the bird feature significance value of each suspected bird area, the historical RGB image is multi-scaled and compared with the images at different scales, and the bird feature significance value of each suspected bird area is combined to determine the feature weight corresponding to each suspected bird area;

[0008] The loss function in the target detection algorithm is optimized based on the feature weights of the suspected bird area in the historical RGB image to obtain an optimized target detection algorithm; target detection is performed on the real-time RGB image based on the optimized target detection algorithm to obtain the target area; and bird recognition is performed based on the target area and the sound classification result.

[0009] Further, the determining the edge line of the power transmission line in the historical RGB image includes:

[0010] Processing the grayscale image of the historical RGB image based on the Canny operator to obtain edge lines in the historical RGB image;

[0011] Based on the number of edge pixels on the edge line and the regularity of the edge line, the edge line of the transmission line is screened from all the edge lines.

[0012] Furthermore, the step of screening the edge line of the power transmission line from all edge lines based on the number of edge pixels on the edge line and the regularity of the edge line includes:

[0013] Fitting the position coordinates of edge pixels on each edge line based on the least squares method to obtain a fitting curve;

[0014] On the fitting curve, a tangent slope value corresponding to each position coordinate is obtained, and based on the difference between the tangent slope values ​​of adjacent position coordinates, an edge fluctuation factor of the edge line corresponding to the fitting curve is determined;

[0015] Determine an edge rule characteristic value of each edge line based on an edge fluctuation factor of each edge line and the number of edge pixels on each edge line, wherein the edge fluctuation factor is negatively correlated with the edge rule characteristic value, and the number of edge pixels is positively correlated with the edge rule characteristic value;

[0016] Among all edge lines, the edge lines whose edge rule characteristic values ​​are greater than the preset rule threshold are regarded as transmission line edge lines.

[0017] Furthermore, the step of determining the image block corresponding to each edge pixel point on each edge line of the power transmission line includes:

[0018] In the historical RGB image, on each edge line of the power transmission line, a neighborhood of a preset size is constructed with each edge pixel point as the center, as an image block corresponding to each edge pixel point.

[0019] Furthermore, the method for obtaining the suspected bird area includes:

[0020] On each transmission line edge line, the mean of the number of edge pixels in the image blocks corresponding to all edge pixels is taken as the mean eigenvalue;

[0021] On each transmission line edge line, the difference between the number of edge pixels in each image block and the corresponding mean eigenvalue is used as the first characteristic factor of each image block; the neighborhood block of each image block is determined, and the second characteristic factor of each image block is determined based on the difference between the number of edge pixels in each image block and the number of edge pixels in the neighborhood block;

[0022] Normalizing the sum of the first characteristic factor and the second characteristic factor to obtain a value that serves as a characteristic degree value of each image block;

[0023] Among all the image blocks, the image blocks with feature degree values ​​greater than the preset feature threshold are regarded as suspected bird areas.

[0024] Furthermore, the method for obtaining the significant value of the bird feature includes:

[0025] Taking the central pixel point of the historical RGB image as a reference point;

[0026] In each suspected bird area, determining a distance factor of the suspected bird area based on the Euclidean distance between the pixel point and the reference point position;

[0027] In each suspected bird region, the color deviation value of the suspected bird region is determined based on the Euclidean distance of the color values ​​between different pixels in all color channels;

[0028] Based on the distance factor and color deviation value corresponding to each suspected bird area, the bird characteristic significance value of each suspected bird area is determined, and the distance factor and color deviation value are negatively correlated with the bird characteristic significance value, and the bird characteristic significance value is a normalized value.

[0029] Furthermore, the multi-scale processing of the historical RGB images based on the bird feature saliency value of each suspected bird area and the comparison of images at different scales are performed, and the feature weight corresponding to each suspected bird area is determined by combining the bird feature saliency value of each suspected bird area, including:

[0030] In the historical RGB image, the bird feature saliency value of each suspected bird region is used as the sampling weight corresponding to each suspected bird region, and the sampling weight of the image block other than the suspected bird region is set to a preset value;

[0031] In the historical RGB image, upsampling the historical RGB image a preset number of times based on the sampling weights of all image blocks to obtain different sampled images;

[0032] Based on the difference in the number of pixels of each suspected bird area in different sampling images and in the historical RGB images, as well as the significant value of the bird features of each suspected bird area, the feature weight corresponding to each suspected bird area is determined.

[0033] Furthermore, the method for obtaining the feature weight includes:

[0034] The difference between the number of pixels in each suspected bird area in different sampling images and the number of pixels in the historical RGB image is used as the difference factor corresponding to each sampling image;

[0035] The product of the mean of the difference factor of each suspected bird region in all sampled images and the bird feature significance value is normalized and used as the feature weight corresponding to each suspected bird region.

[0036] Furthermore, the loss function in the target detection algorithm is optimized based on the feature weights of the suspected bird areas in the historical RGB images, including:

[0037] ;

[0038] in, Represents the optimized loss function; represents the original loss function; represents a hyperparameter; Indicates the first The feature weight of the suspected bird area, Indicates the number of suspected bird areas in the historical RGB image.

[0039] Furthermore, the bird identification based on the target area and the sound classification result includes:

[0040] Get real-time sound signals at the transmission lines;

[0041] Classify all target areas based on the trained neural network model to obtain image classification results;

[0042] Using the real-time sound signal as input of a trained neural network model and outputting a sound classification result;

[0043] The image classification result and the sound classification result are combined to determine the category of the bird in the real-time RGB image.

[0044] The present invention has the following beneficial effects:

[0045] Obtaining the historical RGB image and real-time RGB image of the transmission line, and determining the edge line of the transmission line in the historical RGB image, can reduce the scope of subsequent processing. Furthermore, since birds stay on the transmission line, some parts of the edge line will be raised. Therefore, after determining the local image block where each edge pixel point is located on each transmission line edge line, the suspected bird area is screened out based on the number of edge pixels in the image block. At this time, the suspected bird area is more likely to represent the presence of birds in the area of ​​the image. Next, for the suspected bird area, its salience in the global historical RGB image should be analyzed. Since the salience can be evaluated by whether the color is concentrated and whether the position is close to the visual center, the position distribution and color distribution of each suspected bird area in the historical RGB image are analyzed to determine the salience value of the bird features in the suspected bird area. Furthermore, since the areas occupied by birds on the transmission line are of different sizes, in order to more accurately determine the boundaries and shapes of each target area in the subsequent target detection process, the historical RGB images are multi-scale processed based on the bird feature saliency values ​​of the suspected bird area to capture the features of different scales in the image, and then the images at different scales are compared, and the feature weights of each suspected bird area are determined by combining the bird feature saliency values ​​of each suspected bird area. Finally, the loss function in the target detection algorithm is optimized using the feature weights to obtain the optimized target detection algorithm. Finally, target detection is performed on the real-time RGB image based on the optimized target detection algorithm, which can make the detection process more focused on important bird feature areas and obtain more accurate target areas, which are the identified bird areas. Finally, bird recognition is performed based on the target area, making the recognition results more accurate, greatly improving the pertinence and effectiveness of subsequent bird expulsion. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0047] Figure 1 A method flow chart of a method for jointly identifying images and sounds of birds invading power transmission lines provided by an embodiment of the present invention;

[0048] Figure 2 A method flow chart of a method for obtaining a significant value of a bird feature provided by one embodiment of the present invention;

[0049] Figure 3 A system block diagram of a system for jointly identifying images and sounds of birds invading power transmission lines provided by one embodiment of the present invention;

[0050] Figure 4 A schematic diagram of the system structure of a system for jointly identifying images and sounds of birds invading power transmission lines provided by one embodiment of the present invention;

[0051] Figure 5 A schematic diagram of a computer-readable storage medium provided for one embodiment of the present invention. DETAILED DESCRIPTION

[0052] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the following is a detailed description of the specific implementation method, structure, features and effects of a method for joint recognition of images and sounds of birds invading power lines proposed by the present invention in combination with the accompanying drawings and preferred embodiments. In the following description, different "one embodiment" or "another embodiment" does not necessarily refer to the same embodiment. In addition, specific features, structures or characteristics in one or more embodiments may be combined in any suitable form.

[0053] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

[0054] The following is a detailed description of a method for jointly identifying images and sounds of birds invading power transmission lines provided by the present invention with reference to the accompanying drawings.

[0055] See also Figure 1 , which shows a method flow chart of a method for jointly identifying images and sounds of birds invading power transmission lines provided by an embodiment of the present invention, the method comprising the following steps;

[0056] Step S1: Acquire a historical RGB image and a real-time RGB image of a power transmission line; determine the edge line of the power transmission line in the historical RGB image.

[0057] Bird intrusion is a common but potentially very harmful problem in power systems. Bird activities on power lines and facilities can cause a variety of problems, including short circuits, power outages, equipment damage, and even fires, which not only cause economic losses to power infrastructure, but may also cause harm to the birds themselves. Therefore, it is necessary to drive away birds on transmission lines. Given that different birds respond differently to drive-away signals, birds on transmission lines should be identified so that they can be driven away more specifically. Therefore, the accuracy of bird identification is particularly important.

[0058] In the embodiment of the present invention, the image data is combined with the sound data, and the morphological characteristics of the birds can be analyzed through image recognition technology, so as to identify the species; at the same time, the sound data includes the chirping of birds, etc., and these characteristics can also be used as an important basis for identifying birds. Therefore, the camera installed on the transmission line tower can be used to shoot the transmission line to obtain the historical RGB image and real-time RGB image of the transmission line; while shooting, the real-time sound signal at the transmission line is obtained through the microphone or sound sensor installed near the camera.

[0059] It should be noted that since the image data and sound data collected outdoors may contain noise, the historical RGB images, real-time RGB images and real-time sound signals can be denoised. The specific denoising method can use mean filtering, etc., which is a technical means well known to technical personnel in this field and will not be elaborated here; the specific image data and sound data acquisition device can be adjusted according to the implementation scenario and is not limited here.

[0060] At this point, the historical RGB image, real-time RGB image and real-time sound signal required in the embodiment of the present invention can be obtained.

[0061] Because the transmission lines are long and linear with relatively regular features, and the lines in the background of the image are more complex and irregular, the edge lines of the transmission lines in the historical RGB image can be determined, and the area where the edge lines of the transmission lines are located can accurately represent the transmission line area.

[0062] Preferably, in one embodiment of the present invention, determining the edge line of the power transmission line in the historical RGB image includes:

[0063] First, the grayscale image corresponding to the historical RGB image is obtained. Then, in the grayscale image, the edge line is obtained based on the Canny operator, and then it is mapped back to the historical RGB image, that is, the edge pixel points at the corresponding position of the edge line in the grayscale image in the historical RGB image are used as the edge pixel points in the historical RGB image, thereby obtaining the edge line in the historical RGB image.

[0064] It should be noted that the method for obtaining the grayscale image can adopt methods such as average grayscale, maximum value method or weighted grayscale, which are all technical means well known to technical personnel in this field and are not limited or elaborated here; the method of obtaining edge lines by the Canny operator is a well-known technology and is not elaborated here.

[0065] In historical RGB images, since power transmission lines are usually long and continuous, the edge lines formed are usually longer and have more obvious regular features. In contrast, the lines in the background of historical RGB images are often more complex and changeable, without fixed rules, such as the branches and leaves of trees, the outlines of buildings, etc. The edge lines formed by these background lines in the image are often messy. The length of the edge line can be measured by the number of edge pixels. Therefore, based on the number of edge pixels on the edge pixels and the regularity of the edge lines, the edge lines of the power transmission lines are screened out from all the edge lines. The edge lines of the power transmission lines can help to quickly locate and identify the power transmission line areas in historical RGB images.

[0066] Therefore, the position coordinates of the edge pixel points on each edge line are fitted based on the least squares method to obtain a fitting curve. At this time, the fitting curve can characterize the shape of each edge line.

[0067] Then, on the fitting curve, the tangent slope value corresponding to each position coordinate is obtained, and based on the difference between the tangent slope values ​​of adjacent position coordinates, where the difference is specifically characterized by the absolute value of the difference, the edge fluctuation factor of the edge line corresponding to the fitting curve is determined. The formula model of the edge fluctuation factor can be, for example, as follows:

[0068] ;

[0069] in, Indicates the edge fluctuation factor corresponding to each edge line; Indicates the number of position coordinates on the fitting curve corresponding to each edge line; Indicates that on the fitting curve corresponding to each edge line, The tangent slope value of the position coordinate; Indicates that on the fitting curve corresponding to each edge line, The tangent slope value corresponding to the position coordinates.

[0070] In the formula model of the edge fluctuation factor, the tangent slope value corresponding to each position coordinate is obtained on the fitting curve corresponding to the edge line. The tangent slope value can characterize the change trend of the fitting curve, and then the regularity of the edge line can be quantified by analyzing the difference between the change trends of adjacent position coordinates. When the difference is larger, it means that the trend of the fitting curve has changed. Then it can be considered that the edge line has a more complex shape at this time, and it is more likely to be the edge line of the background area. On the contrary, when the absolute value of the difference is The smaller the value, the more consistent the trend of the fitting curve at different positions is, which is more likely to be the edge line corresponding to the transmission line. Therefore, on the fitting curve corresponding to the edge line, the average of the absolute values ​​of the difference between the tangent slope values ​​of all adjacent position coordinates is used as the edge fluctuation factor. The larger the edge fluctuation factor value, the less likely the edge line is to be the edge line corresponding to the transmission line area.

[0071] Then, the edge fluctuation factor of each edge line and the number of edge pixels on each edge line are combined to determine the edge rule characteristic value of each edge line. The formula model of the edge rule characteristic value can be specifically, for example:

[0072] ;

[0073] in, represents the edge rule feature value of each edge line; Indicates the number of edge pixels on each edge line; Indicates the edge fluctuation factor corresponding to each edge line; Expressed as a natural constant An exponential function with base ; Represents the normalization function.

[0074] In the formula model of edge rule eigenvalue, when the number of pixel points on the edge line is more, it is considered that it is more likely to be the edge line corresponding to the transmission line rather than the edge line in the background area; when the edge fluctuation factor corresponding to the edge line is larger, it is considered that it is more likely to be the edge line in the background area, and vice versa, when the edge fluctuation factor corresponding to the edge line is smaller, it is considered that it is more likely to be the edge line corresponding to the transmission line. Therefore, the edge fluctuation factor is negatively correlated and normalized to achieve logical relationship correction, and then multiplied by the number of edge pixels on the edge line, and the obtained product is normalized to obtain the edge rule eigenvalue corresponding to each edge line. The value range of the edge rule eigenvalue is 0-1, and the larger the value, the more likely it is the edge line corresponding to the transmission line, that is, the edge line of the transmission line.

[0075] Finally, among all edge lines, the edge lines whose edge rule feature values ​​are greater than the preset rule threshold are taken as transmission line edge lines.

[0076] It should be noted that the method of obtaining the fitting curve by the least squares method is a technical means well known to those skilled in the art and will not be elaborated here; the method for calculating the tangent slope value is a well-known technology and will not be elaborated here; the preset rule threshold is set to 0.7, and the specific value can be adjusted according to the implementation scenario and is not limited here; when analyzing the coordinates, the lower left corner of the historical RGB image can be taken as the origin, the horizontal right direction is the x-axis, and the vertical upward direction is the y-axis to establish a coordinate system. At this time, each pixel point in the historical RGB image has a two-dimensional coordinate, and the specific coordinate system construction process can be adjusted according to the actual scenario and is not limited here; the method for obtaining the tangent slope value is a well-known technology and will not be elaborated here.

[0077] At this point, the edge lines of the transmission lines in the historical RGB images, that is, the edge lines corresponding to the transmission lines, can be screened out. In the subsequent process, the edge lines of the transmission lines can be further analyzed to identify the birds.

[0078] Step S2: determine the image block corresponding to each edge pixel point on the edge line of each transmission line; based on the difference in the number of edge pixels in different image blocks, screen out suspected bird areas in all image blocks.

[0079] When birds land on the transmission line, a bulge will appear on the edge line where the transmission line is located. The number of edge pixels in the area where the bulge is located will be greater than that in other areas. Therefore, based on local considerations, the image block corresponding to each edge pixel on the edge line of the transmission line is first determined.

[0080] Preferably, determining the image block corresponding to each edge pixel point on each transmission line edge line includes:

[0081] In the historical RGB image, on each transmission line edge line, a neighborhood of a preset size is constructed with each edge pixel as the center, as the image block corresponding to each edge pixel. It should be noted that the size of the neighborhood is set to 5×5, and the specific size can be adjusted according to the implementation scenario and is not limited here.

[0082] Then, the number distribution of edge pixels in different image blocks is analyzed, and suspected bird areas are screened out from all image blocks. The locations of the suspected bird areas are very likely to be areas where birds exist.

[0083] Preferably, in one embodiment of the present invention, the method for obtaining the suspected bird area includes:

[0084] On each transmission line edge line, the mean of the number of edge pixels in the image blocks corresponding to all edge pixels is taken as the mean eigenvalue corresponding to the image blocks on each transmission line edge line; by calculating the mean eigenvalue, the influence of the extreme value of the number of edge pixels in individual image blocks on the overall analysis can be reduced, making the subsequent calculation results more stable and accurate.

[0085] Then, on each transmission line edge line, the difference between the number of edge pixels in each image block and the corresponding mean eigenvalue is used as the first eigenfactor of each image block; the first eigenfactor quantifies the extent to which the number of edge pixels in each image block is higher than the overall average level.

[0086] Next, considering that birds on the transmission line will cause the edge line to form a convex hull, the number of edge pixels in the area where the convex hull is located should be greater than that in other areas. Therefore, the second characteristic factor of each image block is determined by analyzing the difference in the number of edge pixels between each image block and the neighborhood block. The difference here refers to the difference value, and the neighborhood block can be set to the image block corresponding to the neighborhood edge pixel of the central edge pixel of the image block (the neighborhood edge pixel is the other edge pixel in the image block except the central edge pixel). The neighborhood block set in this way is closer to the image block and has a higher reference value. The second characteristic factor takes into account the change in the number of edge pixels in image blocks with similar distances, which is more conducive to identifying the area where pixel mutations occur on the edge line, that is, it is more conducive to identifying the area where bulges appear on the edge line where the transmission line is located.

[0087] The first characteristic factor and the second characteristic factor are combined, and the sum of the first characteristic factor and the second characteristic factor is normalized to obtain the value as the characteristic degree value of each image block. The formula model of the characteristic degree value includes:

[0088] ;

[0089] in, Indicates the characteristic degree value of each image block; Indicates the number of edge pixels in each image block; Represents the mean eigenvalue corresponding to each image block; represents the number of neighborhood blocks for each suspected bird area; Represents the first The number of edge pixels in a neighborhood block; Represents the normalization function.

[0090] In the formula model of the characteristic degree value, in the overall field of view, the difference between the number of edge pixels in each image block and the corresponding mean characteristic value is taken as the first characteristic factor , the larger the value of the first characteristic factor, the higher the average number of edge pixels in the image block is, and the image block is more likely to be in the convex hull area on the edge of the transmission line, which is more likely to be an area where birds exist. At the same time, from a more detailed local analysis, the difference between the number of edge pixels in each image block and the number of edge pixels in each neighborhood block is calculated. , when the difference is larger, it means that the number of edge pixels in the image block is more prominent than that of the surrounding image blocks, which is more likely to be an area where birds exist on the transmission line. Therefore, the difference between the number of edge pixels in each image block and all neighboring blocks is averaged to obtain the second characteristic factor , the larger the value of the second eigenfactor is, the more likely it is that there are birds in the corresponding image block, and therefore it needs more attention in the future. Finally, the first eigenvalue and the second eigenvalue are combined and added together. Since both the first and second eigenfactors may be negative, The function is monotonic, so we use The function normalizes it so that the sum value ranges from 0 to 1, and the closer it is to 1, the more the image block represents the edge of the area where birds may exist.

[0091] Finally, among all the image blocks, the image blocks with feature degree values ​​greater than the preset feature threshold are regarded as suspected bird areas.

[0092] It should be noted that the value of the preset feature threshold is set to 0.65, and the specific value can be adjusted according to the implementation scenario and is not limited here.

[0093] Step S3: In each suspected bird area, the bird feature significance value of the suspected bird area is determined based on the position distribution of the pixel points in the historical RGB image and the color distribution chaos; based on the bird feature significance value of each suspected bird area, the historical RGB image is multi-scale processed and the images at different scales are compared, and the bird feature significance value of each suspected bird area is combined to determine the feature weight corresponding to each suspected bird area.

[0094] Based on the above steps, a suspected bird area that may represent the edge of the area where birds are located on the transmission line can be obtained. Therefore, each pixel point in the suspected bird area may represent certain characteristics of the bird. Therefore, in this step, all pixel points in the suspected bird area can be analyzed, rather than just the edge pixels.

[0095] Since different suspected bird regions may represent different features, and different features play different roles in bird identification. And since the color distribution and position distribution of pixel points are basic features in the image, they play an important role in identifying objects in the image. Therefore, by analyzing the position distribution and color distribution chaos of edge pixels in the suspected bird region, the bird feature saliency value of the suspected bird region is obtained, which can more effectively determine the key area in the image. At the same time, since a single-scale image may not be able to capture all the important features in the image, and multi-scale processing can make up for this defect, the bird feature saliency value of each suspected bird region is used to perform multi-scale processing on the historical RGB image, and by comparing images at different scales and combining the bird feature saliency value, more comprehensive feature information is extracted, and the feature weight of each suspected bird region is determined.

[0096] Color is an important visual feature in an image. In the task of identifying birds on power lines, when the colors of the pixels in the suspected bird area are relatively consistent, the part will be more prominent visually. Similarly, when the position distribution of the pixels in the suspected bird area is farther away from the visual center, the degree of prominence will be lower. Therefore, in each suspected bird area, the bird feature saliency value of the suspected bird area is determined based on the position distribution of the edge pixels in the historical RGB image and the color distribution chaos. The calculation of this bird feature saliency value helps to give different attention to these suspected bird areas in subsequent image processing.

[0097] Preferably, in one embodiment of the present invention, the method for obtaining the saliency value of bird features includes:

[0098] See also Figure 2 , which shows a method flow chart of a method for obtaining a significant value of a bird feature in one embodiment of the present invention, the method comprising the following steps:

[0099] Step S301: Determine a reference point and determine a distance factor for each suspected bird region based on a position deviation between a pixel point in each suspected bird region and the reference point.

[0100] The central pixel of the historical RGB image is used as the reference point. The position of the reference point can represent the position of the visual center in the image. Since the positional relationship between the suspected bird area and the visual center can be represented by the Euclidean distance between each pixel in the suspected bird area and the reference point, in each suspected bird area, the mean of the Euclidean distances between all pixels and the reference point is used as the distance factor of the suspected bird area, denoted as The distance factor represents the deviation between the pixel points in the suspected bird area and the visual center. The larger the distance factor value is, the farther the suspected bird area is from the visual center, and the smaller the prominence will be. Conversely, when the distance factor value is smaller, it means that the suspected bird area is closer to the visual center, and the greater the prominence will be.

[0101] Step S302: determining a color deviation value of each suspected bird region based on differences between color values ​​of different pixels in each suspected bird region under all color channels.

[0102] The color distribution disorder of the pixels in the suspected bird area can be characterized by the color difference between the pixels. Therefore, in each suspected bird area, all the pixels are combined in pairs to obtain all non-repeating pixel combinations. In view of the historical RGB image in the embodiment of the present invention, each pixel has a color value in the red, green and blue channels. Therefore, in each pixel combination, the Euclidean distance between the color values ​​of the two pixels in all color channels is used as the distance parameter. Then, the mean of the distance parameters corresponding to all pixel combinations is used as the color deviation value of the suspected bird area, which is recorded as The color deviation value represents the color difference between the pixels in the suspected bird area, that is, the color distribution disorder. The larger the color deviation value, the more disordered the color distribution of the pixels in the suspected bird area. The more inconsistent the colors of the pixels in the suspected bird area are, and the lower the degree of prominence will be. Conversely, when the color deviation value is smaller, the more consistent the color distribution of the pixels in the suspected bird area is, then on the basis that the suspected bird area represents the area where birds exist, it is also possible to represent the feathers, eyes and other characteristics of birds. In the subsequent analysis process, the degree of prominence should be higher.

[0103] Step S303: determining the bird feature saliency value of each suspected bird region based on the distance factor and the color deviation value corresponding to each suspected bird region.

[0104] Based on the distance factor and color deviation value corresponding to each suspected bird area, the bird feature significance value of each suspected bird area is determined, and the distance factor and color deviation value are negatively correlated with the bird feature significance value, and the bird feature significance value is a normalized value. The formula model of the bird feature significance value can be, for example:

[0105] ;

[0106] in, Indicates the bird feature significance value of each suspected bird area; represents the distance factor for each suspected bird area; Indicates the color deviation value of each suspected bird area; Expressed as a natural constant An exponential function with base .

[0107] In the formula model of the bird feature saliency value, based on the analysis in the aforementioned steps S301 and S302, it can be known that the larger the distance factor value is, the farther the suspected bird area is from the visual center, and the smaller the prominence will be. On the contrary, when the distance factor value is smaller, the closer the suspected bird area is to the visual center, the greater the prominence will be. The smaller the color deviation value is, the more consistent the color distribution of pixels in a suspected bird area is. On the basis that the suspected bird area represents the area where birds exist, it is also possible to characterize the feathers, eyes and other features of birds, so the prominence should be higher in the subsequent analysis process. Therefore, the distance factor and the color deviation value are both inversely related to the prominence degree, so the sum of the distance factor and the color deviation value is negatively correlated and normalized to achieve logical relationship correction, and the bird feature saliency value of each suspected bird area is obtained. At this time, the larger the bird feature saliency value is, the higher the attention should be given.

[0108] At this point, the bird feature significance value of each suspected bird area can be obtained, which is used to quantify the criticality of different suspected bird areas, and the feature weight of each suspected bird area can be further adjusted in the subsequent process.

[0109] When identifying birds on transmission lines, birds may appear in the power history RGB image at different sizes and distances. Therefore, the suspected bird area that may represent birds may correspond to birds of different scales. Therefore, if the weight of the suspected bird area is determined only at a single scale, the information in the suspected bird area will be inaccurate. Multi-scale processing can reveal information at different scales in the image, including detail information and global information. In the aforementioned process, the image degree value can characterize the importance of each suspected bird area. Therefore, in this embodiment of the present invention, the historical RGB image is multi-scale processed based on the bird feature significance value of each suspected bird area and the images at different scales are compared to promote feature fusion, thereby obtaining the feature weight corresponding to each suspected bird area.

[0110] Preferably, in one embodiment of the present invention, the method for obtaining feature weights includes:

[0111] Since the bird feature saliency value can be used as a quantitative indicator to measure the importance of each suspected bird area, in the historical RGB image, the bird feature saliency value of each suspected bird area is used as the sampling weight corresponding to each suspected bird area, and the sampling weight of the image block except the suspected bird area is set to a preset value.

[0112] Since the purpose of the present invention is to identify birds on transmission lines, more attention should be paid to detail information. Therefore, in the historical RGB image, the historical RGB image is upsampled a preset number of times based on the Gaussian pyramid algorithm and the sampling weights of the image blocks to obtain different sampling images.

[0113] Since the difference in the number of pixels in each suspected bird region in different sampling images and the original image is of great significance for evaluating the characteristics of the suspected bird region, the difference in the number of pixels in each suspected bird region in each sampling image and the historical RGB image is calculated as the difference factor corresponding to each sampling image, and then the mean of the difference factor corresponding to each suspected bird region in all sampling images and the product of the bird feature significance value of each suspected bird region are normalized to obtain the feature weight corresponding to each suspected bird region. The formula model of the feature weight includes:

[0114] ;

[0115] in, represents the feature weight of each suspected bird area; Indicates the bird feature significance value of each suspected bird area; Indicates the number of sampled images; Indicates the number of pixels in each suspected bird area in the historical RGB image; Indicates that each suspected bird area is in the The number of pixels in the sampled image; Represents the normalization function.

[0116] In the formula model of feature weight, since upsampling can restore some lost details in the suspected bird area, if the number of pixels in the suspected bird area increases under different sampling weights, it means that more details are restored, so the feature weight should be increased. Therefore, in each upsampled sampled image, the number of pixels in the suspected bird area in the sampled image is subtracted from the number of pixels in the historical RGB image to obtain the difference factor of the suspected bird area in each sampled image. , the larger the difference factor, the more detailed information there is in the suspected bird area, so the suspected bird area should have a larger weight in the electrical history RGB image, so the difference factors corresponding to the suspected bird area in all sampled images are averaged , the mean value combines the details of the suspected bird area in all sampled images, and the larger the value, the greater the weight of the suspected bird area should be. Since the larger the bird feature significance value of the suspected bird area is, the higher the attention should be given, so the product of the mean value of all difference factors in the suspected bird area and the bird feature significance value is normalized as the feature weight of the suspected bird area, and the larger the feature weight is, the more important and significant the feature of the suspected bird area is.

[0117] It should be noted that, in the embodiment of the present invention, only the feature weight analysis is performed on the suspected bird area, and since the sampling weight of the suspected bird area ranges from 0 to 1, the sampling weight of other image blocks except the suspected bird area is set to 1, that is, the preset value is set to 1; the preset number of times is set to 5, and the specific number of upsampling can be adjusted according to the implementation scenario, which is not limited here; the process of upsampling the image based on the Gaussian pyramid algorithm is a technical process well known to those skilled in the art and will not be elaborated here.

[0118] At this point, the feature weights of all suspected bird areas in the historical RGB images can be obtained. The larger the feature weight, the more information related to the bird identification task is contained in the suspected bird area, which will play a greater role in the subsequent process.

[0119] Step S4: Optimize the loss function in the target detection algorithm based on the feature weights of the suspected bird area in the historical RGB image to obtain an optimized target detection algorithm; perform target detection on the real-time RGB image based on the optimized target detection algorithm to obtain the target area; and identify the bird based on the target area and the sound classification result.

[0120] Object detection algorithms can quickly identify potential target areas from an image. This is particularly important for bird recognition because it can narrow the focus to a specific part of the image, reducing the complexity and computation of subsequent processing. In object detection algorithms, weight_decay is applied by regularizing the model weights during training, specifically by adding a term proportional to the model weight to the loss function ( , represents the hyperparameter, Represents the model weight), this regularization technique is intended to reduce the risk of overfitting, thereby improving the generalization ability of the model. However, based on the analysis and calculation of the aforementioned steps, it can be seen that in the complex bird recognition task process, different regions have different significance levels. Therefore, in an embodiment of the present invention, by using feature weights for regularization, it can be ensured that while reducing overfitting, the importance of different regions in the bird recognition task can be more finely controlled, and the recognition accuracy can be improved, so as to better adapt to the needs of specific tasks. Therefore, the target detection algorithm is combined with the feature weight of each suspected bird area, and the feature weight of the suspected bird area in the historical RGB image is used to optimize the loss function in the target detection algorithm, thereby obtaining an optimized target detection algorithm. Finally, the optimized target detection algorithm is used to perform target detection on the historical RGB image to obtain the target area. And bird recognition is performed based on the target area to obtain a more accurate bird recognition effect.

[0121] In an embodiment of the present invention, the target detection algorithm uses the YOLOv5 algorithm. Regularization and feature weights have a synergistic effect in optimizing model performance. Regularization is used to prevent overfitting, while feature weights improve recognition accuracy by finely controlling the importance of different regions. Combining these two methods can further improve the performance of the model in complex bird recognition tasks. Therefore, the feature weights of each suspected bird region are applied to the YOLOv5 algorithm, the loss function is optimized, and the optimized target detection algorithm is obtained.

[0122] Preferably, in one embodiment of the present invention, the loss function in the target detection algorithm is optimized based on the feature weights of the suspected bird regions in the historical RGB images, including:

[0123] First, in the regularization process, the feature weights of the suspected bird areas are used instead of the model weights to optimize the original loss function. The formula model of the optimized loss function includes:

[0124] ;

[0125] in, Represents the optimized loss function; represents the original loss function; represents a hyperparameter; Indicates the first The feature weight of the suspected bird area, Indicates the number of suspected bird areas in the historical RGB image.

[0126] In the formula model of the optimized loss function, the feature weights of the suspected bird areas replace the model weights. At this time, the optimized loss function can reduce the risk of overfitting while also integrating those areas that have an important impact on the bird recognition results, effectively improving the generalization ability and recognition accuracy.

[0127] It should be noted that the hyperparameter Set to 4. The specific value can be adjusted according to the implementation scenario and is not limited here.

[0128] After optimizing the loss function, an optimized target detection algorithm, that is, an optimized YOLOv5 algorithm, can be obtained after training. It should be noted that the specific training process is a process well known to those skilled in the art and will not be described in detail here.

[0129] Finally, target detection can be performed on the real-time RGB image based on the optimized YOLOv5 algorithm, and the bounding box of each bird can be output. These bounding boxes directly mark the position of the bird in the real-time RGB image, that is, the target area is obtained.

[0130] It should be noted that the method of obtaining the target area based on the target detection algorithm is a well-known technology and will not be described in detail here.

[0131] Bird identification not only relies on visual information, but is also influenced by sound signals, such as chirping. Different species of birds often have unique differences in visual and sound characteristics, and these differences can be better reflected and utilized in the fusion of multimodal information. Therefore, by combining image and sound information, the accuracy and robustness of identification can be improved.

[0132] Therefore, preferably, in one embodiment of the present invention, bird identification is performed based on the target area and the sound classification result, including:

[0133] YOLOv5 is mainly used for real-time identification and counting of birds near power transmission lines. Therefore, after obtaining the target area, the trained neural network can be used to classify all target areas to obtain image classification results. At the same time, the real-time sound signal is used as the input of the trained neural network model to output the sound classification result. Finally, the image classification results and sound classification results are combined. When there are birds in the sound classification results that are consistent with the image classification results, it is judged that there are such birds on the transmission line at this time. Then, the microphone set on the transmission line tower can be used to play the repelling signal that the birds are sensitive to to drive them away.

[0134] It should be noted that the neural network model used in classifying the target area can adopt the ViT model. The ViT model focuses more on high-level understanding and classification of images. By training the ViT model to identify different bird species, it can more accurately analyze and identify birds; and the neural network model used in classifying sound signals can adopt the ESResNet model. In other embodiments of the present invention, other neural network models can also be used to obtain image classification results and sound classification results. The specific use of the neural network model is not limited here, and since the training process of the neural network model is a process well known to those skilled in the art, the training process will not be repeated here.

[0135] In other embodiments of the present invention, bird identification may also be performed based on the target area in the following manners, including:

[0136] All target areas are classified based on the trained neural network model to obtain image classification results; real-time sound signals are used as input of the trained neural network model to output sound classification results.

[0137] Then, since the image processing process is optimized in the embodiment of the present invention, the accuracy of the image classification results will be higher, so the influence weight of each bird classification in the classification results is set to 0.7, and the influence weight of each bird classification in the sound classification results is set to 0.3. The number of each bird classification is counted in the image classification results and the sound classification results respectively. For example, in the image classification results, there are 5 areas identified as red-headed gulls, so the number of the red-headed gull bird classification is 5.

[0138] Finally, multimodal fusion is performed. In the image classification results and sound classification results, the number of the same bird classification is weighted based on the influence weight (if a certain bird classification only exists in the image classification results, the number of this bird classification in the sound classification results can be recorded as 0, and vice versa), so as to obtain the existence probability of each bird classification. Finally, the bird classifications are arranged in descending order according to the existence probability to obtain the bird classification arrangement sequence, and the repelling signals that the birds of this type are sensitive to are played in sequence according to the order in the bird classification arrangement sequence to repel them.

[0139] It should be noted that, in order to facilitate calculation, all indicator data involved in the calculation in the embodiment of the present invention are preprocessed to eliminate the dimension effect. The specific means of eliminating the dimension effect are technical means well known to those skilled in the art and are not limited here.

[0140] In summary, the historical RGB images, real-time RGB images and real-time sound signals of the transmission line are obtained. The image data reflects the appearance characteristics of the birds, and the sound data reflects the acoustic characteristics such as their calls. The combination of the two can more comprehensively characterize the characteristics of the birds and provide a rich source of information for subsequent bird identification. Since the transmission line is in the shape of a long line and has a more regular shape compared to the background, the edge line in the historical RGB image is determined, and the edge line of the transmission line is screened out based on the number of edge pixels on the edge line and the regularity of the edge line. By determining the edge line of the transmission line, the scope of subsequent processing can be narrowed. Furthermore, since birds stay on the transmission line, some parts of the edge line will be convex. Therefore, after determining the local image block where each edge pixel point is located on each edge line of the transmission line, the suspected bird area is screened out based on the number of edge pixels in the image block. At this time, the suspected bird area is more likely to represent the presence of birds in the area in the image. Next, for the suspected bird area, its prominence in the global historical RGB image should be analyzed. Since the prominence can be evaluated by whether the color is concentrated and whether the position is close to the visual center, the position distribution and color distribution of each suspected bird area in the historical RGB image are analyzed to determine the bird feature prominence value of the suspected bird area. Furthermore, since the areas occupied by birds on the transmission line are of different sizes, in order to more accurately determine the boundaries and shapes of each target area in the subsequent target detection process, the historical RGB image is multi-scaled based on the bird feature prominence value of the suspected bird area to capture the features of different scales in the image, and then the images at different scales are compared, and the feature weight of each suspected bird area is determined by combining the bird feature prominence value of each suspected bird area. Finally, the loss function in the YOLOv5 algorithm is optimized using the feature weight, and the optimized YOLOv5 algorithm is obtained to perform target detection on the real-time RGB image, which can make the detection process more focused on important bird feature areas and obtain more accurate target areas. Finally, bird recognition is performed by combining the target area and sound signal, and the sound data is combined with the image data to make the recognition result more comprehensive and accurate, greatly improving the pertinence and effectiveness of subsequent bird expulsion.

[0141] This embodiment also provides a system for jointly identifying images and sounds of birds invading power lines. Figure 3 , which shows a system block diagram in one embodiment of the present invention, including:

[0142] The data acquisition and edge analysis module 401 is used to implement step S1 in the above method; the suspected bird area screening module 402 is used to implement step S2 in the above method; the feature weight analysis module 403 is used to implement step S3 in the above method; the bird identification module 404 is used to implement step S4 in the above method.

[0143] It should be noted that: the system provided in the above embodiment is only illustrated by the division of the above functional modules. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the computer device is divided into different functional modules to complete all or part of the functions described above. In addition, the power transmission line intrusion bird image and sound recognition system provided in the above embodiment and the power transmission line intrusion bird image and sound recognition method embodiment belong to the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0144] See also Figure 4 , which shows a system structure diagram of a power transmission line intrusion bird image and sound joint recognition system provided by an embodiment of the present invention, including a processor 500, a memory 501, a bus 502 and a communication interface 503, wherein the processor 500, the communication interface 503 and the memory 501 are connected via the bus 502; wherein the memory 501 may include a high-speed random access memory, the bus 502 may be an ISA bus, a PCI bus or an EISA bus, etc., and the processor 500 may be an integrated circuit chip with signal processing capabilities; the memory 501 stores at least one instruction, at least one program, a code set or an instruction set, and when the at least one instruction, at least one program, a code set or an instruction set is loaded and executed by the processor, the steps in a power transmission line intrusion bird image and sound recognition method are implemented.

[0145] The present invention also provides a computer-readable storage medium corresponding to the method provided in the above-mentioned embodiment. Figure 5 , the storage medium shown is a CD, on which a computer program (ie, a program product) is stored. When the computer program is run by a processor, it will execute the method provided by any of the aforementioned embodiments.

[0146] It should be noted that examples of the computer-readable storage medium may also include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), read-only memory (ROM), and other optical and magnetic storage media, which are not listed here one by one.

[0147] It should be noted that the sequence of the above embodiments of the present invention is only for description and does not represent the advantages and disadvantages of the embodiments. The processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0148] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referenced to each other, and each embodiment focuses on the differences from other embodiments.

Claims

1. A method for joint recognition of images and sounds of birds invading power transmission lines, characterized in that: The method comprises: Acquire a historical RGB image and a real-time RGB image of the power transmission line; determine the edge line of the power transmission line in the historical RGB image; Determine the image block corresponding to each edge pixel point on the edge line of each transmission line; based on the difference in the number of edge pixels in different image blocks, screen out suspected bird areas in all image blocks; In each suspected bird area, the bird feature significance value of the suspected bird area is determined based on the position distribution of the pixel points in the historical RGB image and the color distribution disorder; based on the bird feature significance value of each suspected bird area, the historical RGB image is multi-scaled and compared with the images at different scales, and the bird feature significance value of each suspected bird area is combined to determine the feature weight corresponding to each suspected bird area; The loss function in the target detection algorithm is optimized based on the feature weights of the suspected bird area in the historical RGB image to obtain an optimized target detection algorithm; target detection is performed on the real-time RGB image based on the optimized target detection algorithm to obtain the target area; and bird recognition is performed based on the target area and the sound classification result.

2. The method for joint recognition of images and sounds of birds invading power transmission lines according to claim 1 is characterized in that: The step of determining the edge line of the power transmission line in the historical RGB image comprises: Processing the grayscale image of the historical RGB image based on the Canny operator to obtain edge lines in the historical RGB image; Based on the number of edge pixels on the edge line and the regularity of the edge line, the edge line of the transmission line is screened from all the edge lines.

3. The method for joint recognition of images and sounds of birds invading power transmission lines according to claim 2 is characterized in that: The step of screening the edge line of the power transmission line from all edge lines based on the number of edge pixels on the edge line and the regularity of the edge line includes: Fitting the position coordinates of edge pixels on each edge line based on the least squares method to obtain a fitting curve; On the fitting curve, a tangent slope value corresponding to each position coordinate is obtained, and based on the difference between the tangent slope values ​​of adjacent position coordinates, an edge fluctuation factor of the edge line corresponding to the fitting curve is determined; Determine an edge rule characteristic value of each edge line based on an edge fluctuation factor of each edge line and the number of edge pixels on each edge line, wherein the edge fluctuation factor is negatively correlated with the edge rule characteristic value, and the number of edge pixels is positively correlated with the edge rule characteristic value; Among all edge lines, the edge lines whose edge rule characteristic values ​​are greater than the preset rule threshold are regarded as transmission line edge lines.

4. The method for joint recognition of images and sounds of birds invading power lines according to claim 1 is characterized in that: The step of determining the image block corresponding to each edge pixel point on each edge line of the power transmission line comprises: In the historical RGB image, on each edge line of the power transmission line, a neighborhood of a preset size is constructed with each edge pixel point as the center, as an image block corresponding to each edge pixel point.

5. The method for joint recognition of images and sounds of birds invading power transmission lines according to claim 1 is characterized in that: The method for obtaining the suspected bird area includes: On each transmission line edge line, the mean of the number of edge pixels in the image blocks corresponding to all edge pixels is taken as the mean eigenvalue; On each transmission line edge line, the difference between the number of edge pixels in each image block and the corresponding mean eigenvalue is used as the first characteristic factor of each image block; the neighborhood block of each image block is determined, and the second characteristic factor of each image block is determined based on the difference between the number of edge pixels in each image block and the number of edge pixels in the neighborhood block; Normalizing the sum of the first characteristic factor and the second characteristic factor to obtain a value that serves as a characteristic degree value of each image block; Among all the image blocks, the image blocks with feature degree values ​​greater than the preset feature threshold are regarded as suspected bird areas.

6. The method for joint recognition of images and sounds of birds invading power lines according to claim 1 is characterized in that: The method for obtaining the significant value of the bird feature comprises: Taking the central pixel point of the historical RGB image as a reference point; In each suspected bird area, determining a distance factor of the suspected bird area based on the Euclidean distance between the pixel point and the reference point position; In each suspected bird region, the color deviation value of the suspected bird region is determined based on the Euclidean distance of the color values ​​between different pixels in all color channels; Based on the distance factor and color deviation value corresponding to each suspected bird area, the bird characteristic significance value of each suspected bird area is determined, and the distance factor and color deviation value are negatively correlated with the bird characteristic significance value, and the bird characteristic significance value is a normalized value.

7. The method for joint recognition of images and sounds of birds invading power transmission lines according to claim 1 is characterized in that: The method of performing multi-scale processing on the historical RGB images based on the bird feature saliency value of each suspected bird area and comparing the images at different scales, and combining the bird feature saliency value of each suspected bird area, thereby determining the feature weight corresponding to each suspected bird area, includes: In the historical RGB image, the bird feature significance value of each suspected bird area is used as the sampling weight corresponding to each suspected bird area, and the sampling weight of the image block other than the suspected bird area is set to a preset value; In the historical RGB image, upsampling the historical RGB image a preset number of times based on the sampling weights of all image blocks to obtain different sampled images; Based on the difference in the number of pixels of each suspected bird area in different sampling images and in the historical RGB images, as well as the significant value of the bird features of each suspected bird area, the feature weight corresponding to each suspected bird area is determined.

8. The method for joint recognition of images and sounds of birds invading power transmission lines according to claim 7 is characterized in that: The method for obtaining the feature weight includes: The difference between the number of pixels in each suspected bird area in different sampling images and the number of pixels in the historical RGB image is used as the difference factor corresponding to each sampling image; The product of the mean of the difference factor of each suspected bird region in all sampled images and the bird feature significance value is normalized and used as the feature weight corresponding to each suspected bird region.

9. The method for joint recognition of images and sounds of birds invading power transmission lines according to claim 1 is characterized in that: The loss function in the target detection algorithm is optimized based on the feature weights of the suspected bird areas in the historical RGB images, including: ； in, Represents the optimized loss function; represents the original loss function; represents a hyperparameter; Indicates the first The feature weight of the suspected bird area, Indicates the number of suspected bird areas in the historical RGB image.

10. The method for joint recognition of images and sounds of birds invading power lines according to claim 1, characterized in that: The bird identification based on the target area and the sound classification result includes: Get real-time sound signals at the transmission lines; Classify all target areas based on the trained neural network model to obtain image classification results; Using the real-time sound signal as input of a trained neural network model and outputting a sound classification result; The image classification result and the sound classification result are combined to determine the category of the bird in the real-time RGB image.

Images