Power line protection zone image recognition warning method, device, terminal and storage medium

Abstract

The present invention relates to the field of power line protection technology, and in particular to a method, device, terminal and storage medium for image recognition and early warning of a power line protection zone. The method of the present invention first acquires a plurality of first images, identifies a moving target and the position of the moving target based on the plurality of first images, and then accurately focuses on the moving target based on the position of the moving target, and then acquires focus parameters, and obtains the moving parameters of the moving target through the focus parameters; finally, an early warning message is issued according to the moving parameters. It can be seen that based on the analysis of the image, it is possible to quickly determine whether there is a moving target and the position of the moving target in the power line protection zone, and focus on the moving target through the position to obtain the moving parameters of the moving target. The image recognition method is relatively fast, and can intelligently give the moving parameters of the moving target, so as to facilitate targeted countermeasures based on the moving parameters of the moving target.

Description

Technical Field

[0001] The present invention relates to the technical field of power line protection, and in particular to a method, device, terminal and storage medium for early warning of power line protection zone image recognition. Background Art

[0002] The power line protection zone refers to the protection area planned to ensure the safety of the power supply line.

[0003] Taking China as an example, in order to ensure the smooth progress of power production and construction and effectively maintain the public and personal safety around the lines, in accordance with the relevant requirements of Article 10 of the "Regulations on the Protection of Power Facilities" on the protection zone of power lines, the area within the two parallel planes formed by the horizontal extension of the conductor edge line to the outside and perpendicular to the ground, in general areas, the extension distance of the edge line of the conductor at each voltage level is as follows: 5 meters for 1-10 kV, 10 meters for 35-110 kV, 15 meters for 154-330 kV, and 20 meters for 500 kV. In densely populated areas such as factories, mines, and towns, the area of ​​the overhead power line protection zone can be slightly smaller than the above regulations. However, the extension distance of the edge line of the conductor at each voltage level should not be less than the sum of the horizontal distance of the conductor edge line after the maximum calculated sag and the maximum calculated wind deviation and the safe distance from the building after the wind deviation.

[0004] It can be seen that with the progress of the times, the planning of power line protection areas is becoming increasingly stretched. For example, in terms of the protection of drones, drones fly at a high speed and are very likely to break through the protection area at a high speed, causing interference to the power lines.

[0005] Therefore, it is necessary to develop new protection and early warning methods based on existing technologies, so as to timely and proactively discover safety hazards in power supply lines. Summary of the invention

[0006] The embodiments of the present invention provide a method, device, terminal and storage medium for early warning of power line protection zone image recognition, which are used to solve the problem of lack of timely and active discovery of potential safety hazards of power supply lines in the prior art.

[0007] In a first aspect, an embodiment of the present invention provides a method for image recognition and early warning of a power line protection zone, comprising:

[0008] Acquire a plurality of first images, wherein the plurality of first images are images acquired at different times for a predetermined field of view in the protection area;

[0009] Based on the plurality of first images, identifying a moving object and a position of the moving object;

[0010] Based on the position of the moving target, accurately focusing the moving target;

[0011] Acquiring a focus parameter, and obtaining a movement parameter of the moving target through the focus parameter;

[0012] An early warning message is issued according to the movement parameters.

[0013] In a possible implementation manner, identifying the moving target based on the multiple first images includes:

[0014] Acquire a picture taking frame and select two images from the plurality of first images, wherein the picture taking frame is a matrix having a plurality of rows and columns;

[0015] Taking the image extraction frame as a mask, extracting a first selected area and a second selected area corresponding to the upper left corners of the two images respectively, wherein the first selected area and the second selected area correspond to the two images respectively;

[0016] Difference matrix calculation step: calculating the difference between the first selected area and the second selected area as a first difference matrix;

[0017] The step of calculating the selection kernel is as follows: calculating the sum of the elements in the first difference matrix as the first selection kernel;

[0018] If the image taking frame position does not reach the lower right corner, the image taking frame is moved according to a predetermined moving distance, a first selection area and a second selection area are selected from the two images, and the process jumps to the difference matrix calculation step, wherein the first selection area and the second selection area correspond to the two images respectively;

[0019] Selecting two largest first selection area cores from the multiple first selection area cores obtained in the selection area core calculation step;

[0020] If the two largest first selection area cores both exceed the threshold, then there is a moving target in the predetermined field of view.

[0021] In a possible implementation manner, the two images include a latest image, and identifying the position of the moving target based on the multiple first images includes:

[0022] Obtain two positions of the image acquisition frame corresponding to the two largest first selection area cores in the latest image, respectively as a first position and a second position;

[0023] Taking the image extraction frame as a mask, extracting a third selected area corresponding to the first position from the latest image, and extracting a fourth selected area corresponding to the second position from the other of the two images;

[0024] Calculate the absolute value of the difference between the third selected area and the fourth selected area as a second difference matrix;

[0025] Calculate the sum of each element in the second difference matrix as the second selection area kernel;

[0026] If the second selection area kernel is smaller than the threshold, the position of the moving target is the position of the first position;

[0027] Otherwise, the position of the moving target is the position of the second position.

[0028] In a possible implementation manner, the accurately focusing the moving target based on the position of the moving target includes:

[0029] Based on the position of the moving target, acquiring a second image, wherein the second image includes an image of the moving target;

[0030] identifying a main body of the moving target according to the second image;

[0031] Based on the subject, accurate focusing on the moving target is achieved.

[0032] In a possible implementation manner, identifying the main body of the moving target according to the second image includes:

[0033] Acquire a recognition model, wherein the recognition model is an artificial neural network trained with moving target images;

[0034] The second image is input into the recognition model to obtain the category of the moving target and the position of the image of the moving target in the second image.

[0035] In a possible implementation, obtaining the recognition model includes:

[0036] Acquire a plurality of image samples, wherein the plurality of image samples include a moving target image;

[0037] Setting a label for identifying a moving target for each image in the plurality of image samples, the label comprising a moving target category label and a label for a position of the moving target in the image;

[0038] Input the plurality of image samples into the recognition model, wherein the recognition model is a neural network having multiple convolutional layers, pooling layers, and fully connected layers, wherein the multiple convolutional layers are used to extract image features, the pooling layers are used to further extract image features to obtain main features, and the fully connected layers are used to classify images based on the main features;

[0039] According to the label of each image in the multiple image samples and the output of the recognition model, the parameters of the recognition model are adjusted until the deviation between the output of the recognition model and the label of the image in the multiple image samples is less than a threshold.

[0040] In a possible implementation manner, the achieving accurate focusing on the moving target based on the subject includes:

[0041] According to the position of the image of the moving target in the second image, shifting the field of view so that the image of the moving target is located at the center of the second image;

[0042] The focus is adjusted based on the subject to obtain a clear image of the subject.

[0043] In a second aspect, an embodiment of the present invention provides a power line protection zone image recognition and early warning device, comprising:

[0044] An image acquisition module, used to acquire a plurality of first images, wherein the plurality of first images are images acquired at different times for a predetermined field of view within the protection area;

[0045] A target recognition module, configured to recognize a moving target and a position of the moving target based on the plurality of first images;

[0046] A focusing module, used for accurately focusing on the moving target based on the position of the moving target;

[0047] A focus parameter acquisition module, used to acquire focus parameters, and obtain movement parameters of the moving target through the focus parameters; and

[0048] The warning output module is used to issue a warning message according to the movement parameters.

[0049] In a third aspect, an embodiment of the present invention provides a terminal, comprising a memory and a processor, wherein the memory stores a computer program that can be executed on the processor, and when the processor executes the computer program, it implements the steps of the method described in the first aspect or any possible implementation method of the first aspect.

[0050] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method described in the first aspect or any possible implementation of the first aspect are implemented.

[0051] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0052] The embodiment of the present invention discloses a method for image recognition and early warning of a power line protection zone, which first acquires a plurality of first images, and based on the plurality of first images, identifies a moving target and the position of the moving target, and then based on the position of the moving target, accurately focuses on the moving target, then acquires focus parameters, and obtains the moving parameters of the moving target through the focus parameters; finally, issues an early warning message according to the moving parameters. It can be seen that based on the analysis of the image, it is possible to quickly determine whether there is a moving target and the position of the moving target in the power line protection zone, and focus on the moving target through the position to obtain the moving parameters of the moving target. The image recognition method is relatively fast, and can intelligently give the moving parameters of the moving target, so as to facilitate targeted countermeasures based on the moving parameters of the moving target. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0054] Figure 1 It is a flow chart of a method for image recognition and early warning of a power line protection zone provided by an embodiment of the present invention;

[0055] Figure 2 is a schematic diagram of a process of performing image processing based on a first picture provided by an embodiment of the present invention;

[0056] Figure 3 is a schematic diagram of a process of performing image processing based on a second picture provided by an embodiment of the present invention;

[0057] Figure 4 It is a convolutional neural network topology diagram provided by an embodiment of the present invention;

[0058] Figure 5 It is a functional block diagram of a power line protection zone image recognition and early warning device provided by an embodiment of the present invention;

[0059] Figure 6 It is a functional block diagram of a terminal provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0060] In the following description, specific details such as specific system structures and technologies are provided for the purpose of illustration rather than limitation so as to provide a thorough understanding of the embodiments of the present invention. However, it should be clear to those skilled in the art that the present invention may be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, and methods are omitted so as not to obscure the description of the present invention with unnecessary details.

[0061] In order to make the purpose, technical solutions and advantages of the present invention more clear, a specific implementation method will be described below in conjunction with the accompanying drawings.

[0062] The following is a detailed description of an embodiment of the present invention. This example is implemented based on the technical solution of the present invention, and provides a detailed implementation method and a specific operation process, but the protection scope of the present invention is not limited to the following embodiments.

[0063] Figure 1 This is a flow chart of a power line protection zone image recognition and early warning method provided in an embodiment of the present invention.

[0064] like Figure 1 As shown, it shows a flowchart of the implementation of the power line protection zone image recognition and early warning method provided by an embodiment of the present invention, which is described in detail as follows:

[0065] In step 101, a plurality of first images are acquired, wherein the plurality of first images are images acquired at different times for a predetermined field of view in a protection area.

[0066] In step 102, a moving object and a position of the moving object are identified based on the plurality of first images.

[0067] In some implementations, step 102 includes:

[0068] Acquire a picture taking frame and select two images from the plurality of first images, wherein the picture taking frame is a matrix having a plurality of rows and columns;

[0069] Taking the image extraction frame as a mask, extracting a first selected area and a second selected area corresponding to the upper left corners of the two images respectively, wherein the first selected area and the second selected area correspond to the two images respectively;

[0070] Difference matrix calculation step: calculating the absolute value of the difference between the third selected area and the fourth selected area as a first difference matrix;

[0071] The step of calculating the selection kernel is as follows: calculating the sum of the elements in the first difference matrix as the first selection kernel;

[0072] If the image taking frame position does not reach the lower right corner, the image taking frame is moved according to a predetermined moving distance, a first selection area and a second selection area are selected from the two images, and the process jumps to the difference matrix calculation step, wherein the first selection area and the second selection area correspond to the two images respectively;

[0073] Selecting two largest first selection area cores from the multiple first selection area cores obtained in the selection area core calculation step;

[0074] If the two largest first selection area cores both exceed the threshold, then there is a moving target in the predetermined field of view.

[0075] For example, in one application scenario, a wide-angle camera is provided in the protection zone to capture images in the protection zone. The wide-angle camera has a larger field of view. Multiple images acquired by the wide-angle camera at different times are used to determine whether there is a moving target in the protection zone. If there is a moving target, the position of the moving target can also be determined.

[0076] In one application scenario, based on two images, it can be determined whether there is a moving target in the protection area. Figure 2 as well as Figure 3 As shown in the figure, the two figures show the process of image processing based on the first image and the second image. Each dotted rectangle in the figure represents a pixel. As we know, a digital image is expressed by multiple pixels, and each pixel corresponds to one or more numbers. For a rectangular image, the digital structure used to express the image is a matrix of multiple rows and columns. For the two images taken at different times, we first obtain a frame ( Figure 2 FIG. 2 shows that the image acquisition frame is set on the first image 201, and the first selection area 202 is acquired. Figure 3 FIG. 3 shows that a picture taking frame is set on the first image 301 to obtain a first selected area 302), and the picture taking frame is set on the image to obtain selected areas corresponding to the two images.

[0077] Then, subtract the corresponding elements in the two selections and take the absolute value to obtain a difference matrix. Finally, calculate the sum of each element of the difference matrix.

[0078] After completing the above operations, translate the frame a predetermined distance and perform the selection operation again. Figure 2 The operation of shifting the image frame from the upper left corner to the right by two pixel units is shown in FIG. After the shift, the above subtraction and summing operations are repeated until the image frame is shifted to the lower right corner.

[0079] The above operation is essentially to divide the two images into blocks and compare each block. The greater the difference between the blocks, the more non-zero elements the difference between the two blocks contains, and the larger the sum obtained by the solution.

[0080] Therefore, by determining the sum of each selected element, the difference between each image block in the two images can be compared. Figure 2 The first image 201 and Figure 3 In the second image 301, we can find that the pixel with an × in the upper left corner of the first image 201 has moved to the lower right corner of the second image 301, while the rest of the pixels have not changed. Based on the above process, we can know that there is a moving target in the image, and it has moved from the upper left corner of the image to the lower right corner. The rest is the background image, so there is no change when comparing the two images.

[0081] In some implementations, the two images include a latest image, and step 102 includes:

[0082] Obtain two positions of the image acquisition frame corresponding to the two largest first selection area cores in the latest image, respectively as a first position and a second position;

[0083] Taking the image extraction frame as a mask, extracting a third selected area corresponding to the first position from the latest image, and extracting a fourth selected area corresponding to the second position from the other of the two images;

[0084] Calculate the absolute value of the difference between the third selected area and the fourth selected area as a second difference matrix;

[0085] Calculate the sum of each element in the second difference matrix as the second selection area kernel;

[0086] If the second selection area kernel is smaller than the threshold, the position of the moving target is the position of the first position;

[0087] Otherwise, the position of the moving target is the position of the second position.

[0088] For example, to identify the position of a target, we usually analyze the latest image to obtain the latest position of the moving target.

[0089] In one application scenario, analysis is performed based on the positions of the frames corresponding to the two largest selection kernels. As we know, the position of the largest selection kernel corresponds to the position where the moving target appears in the previous and next pictures.

[0090] Still Figure 2 and Figure 3For example, the two maximum selected kernels correspond to the upper left corner and the lower right corner of the image respectively. For the first image 201, the image block in the upper left corner is the image of the moving target, and the image block in the lower right corner is the background image. For the second image 301, the image block in the upper left corner is the background image, and the image block in the lower right corner is the image of the moving target. If the image block in the upper left corner of the first image 201 is compared with the image block in the lower right corner of the second image 301, we can see that the similarity of the two images is high (because both include moving targets). If the second image 301 is the latest image, we can determine that the position of the moving target is the lower right corner. On the contrary, if we take the lower right corner of the first image 201 and the upper left corner of the second image for comparison, the two similarities are low (both are background images, but different parts of the background images are quite different). Then, if the second image 301 is the latest image, we can determine that the position of the moving target is the lower right corner.

[0091] In step 103, the moving target is precisely focused based on the position of the moving target.

[0092] In some embodiments, step 103 includes:

[0093] Based on the position of the moving target, acquiring a second image, wherein the second image includes an image of the moving target;

[0094] identifying a main body of the moving target according to the second image;

[0095] Based on the subject, accurate focusing on the moving target is achieved.

[0096] In some implementations, identifying the main body of the moving target according to the second image includes:

[0097] Acquire a recognition model, wherein the recognition model is an artificial neural network trained with moving target images;

[0098] The second image is input into the recognition model to obtain the category of the moving target and the position of the image of the moving target in the second image.

[0099] In some embodiments, obtaining the recognition model includes:

[0100] Acquire a plurality of image samples, wherein the plurality of image samples include a moving target image;

[0101] Setting a label for identifying a moving target for each image in the plurality of image samples, the label comprising a moving target category label and a label for a position of the moving target in the image;

[0102] Input the plurality of image samples into the recognition model, wherein the recognition model is a neural network having multiple convolutional layers, pooling layers, and fully connected layers, wherein the multiple convolutional layers are used to extract image features, the pooling layers are used to further extract image features to obtain main features, and the fully connected layers are used to classify images based on the main features;

[0103] According to the label of each image in the multiple image samples and the output of the recognition model, the parameters of the recognition model are adjusted until the deviation between the output of the recognition model and the label of the image in the multiple image samples is less than a threshold.

[0104] For example, in some application scenarios, a telephoto camera is also provided, which can rotate horizontally and tilt, so that a farther position in the protection area can be zoomed in to take pictures. The images taken in close up have more details and are easier to distinguish the category of moving targets.

[0105] The image taken by the telephoto camera is used as the second image. When capturing the image, there may be problems such as incomplete capture, inaccurate focus, and the moving target image is not in the center of the second image, which leads to a large deviation in the positioning of the moving target.

[0106] Therefore, after taking a second image of a moving target, the second image should be sent to a recognition model. After the subject is identified, the position of the subject in the second image is analyzed, and the rotation angle and pitch angle of the telephoto camera are adjusted according to the position.

[0107] like Figure 4 As shown, Figure 4 A typical convolutional neural network topology diagram is shown in FIG. For the model used in image recognition, in one application scenario, the neural network model used can be specifically a CNN (Convolutional Neural Networks) neural network model. The model includes a convolution layer, a pooling layer, and a fully connected layer. They are used to extract image features, further extract the main features based on the features, and then classify according to the main features.

[0108] In some applications, convolutional neural networks include an input layer that can process multidimensional data. Commonly, the input layer of a one-dimensional convolutional neural network receives a one-dimensional or two-dimensional array, where the one-dimensional array is usually time or spectrum sampling; the two-dimensional array may contain multiple channels; the input layer of a two-dimensional convolutional neural network receives a two-dimensional or three-dimensional array; the input layer of a three-dimensional convolutional neural network receives a four-dimensional array. Since convolutional neural networks are widely used in the field of computer vision, many studies presuppose three-dimensional input data when introducing their structure, that is, two-dimensional pixels and RGB channels on a plane.

[0109] Similar to other neural network algorithms, since the gradient descent algorithm is used for learning, the input features of the convolutional neural network need to be standardized. Specifically, before inputting the learning data into the convolutional neural network, the input data needs to be normalized in the channel or time / frequency dimension. If the input data is pixels, the original pixel values ​​distributed in it can also be normalized to a predetermined interval. Standardization of input features is conducive to improving the learning efficiency and performance of convolutional neural networks.

[0110] Convolutional neural networks should include hidden layers. The hidden layers of convolutional neural networks include three common structures: convolutional layers, pooling layers, and fully connected layers. In some more modern algorithms, there may be complex structures such as residual blocks. Among the common structures, convolutional layers and pooling layers are unique to convolutional neural networks. The convolution kernel in the convolutional layer contains weight coefficients, while the pooling layer does not contain weight coefficients. Taking LeNet-5 as an example, the order of the three common structures in the hidden layer is usually: input-convolutional layer-pooling layer-fully connected layer-output.

[0111] When building a convolutional neural network, training the neural network is essential. In one application scenario, it is trained through multiple image samples. Multiple image samples are actually pictures of various moving objects, such as kites, model airplanes, drones, or birds. Labels are set for multiple pictures to identify the category of the objects in the pictures and their locations in the pictures.

[0112] After multiple images are input into the neural network model in sequence, the neural network model will output the corresponding classification and the position of the image. According to the deviation between the label and the output of the neural network model, the weights of each transfer function in the neural network model are adjusted, that is, back propagation, which can improve the classification accuracy of the neural network model. When the classification accuracy reaches an acceptable level, our training is completed.

[0113] The second image is input into the trained recognition model, which will give a classification of a moving target, such as a drone, and the position of the drone in the second image, such as the upper right. Based on the position in the second image, the telephoto camera is adjusted so that the image of the drone is located in the center of the entire image. At the same time, based on the recognition of the moving target, focusing can be completed.

[0114] In step 104, a focus parameter is acquired, and a movement parameter of the moving target is obtained through the focus parameter.

[0115] In step 105, a warning message is issued according to the movement parameters.

[0116] For example, as mentioned above, for the telephoto camera, we adjust the rotation angle, pitch angle and focal length so that the moving target image is located in the center of the captured image. Based on the rotation angle and pitch angle, we can establish a vector, and based on the focal length, we can calculate the distance from the moving target to the telephoto camera. Based on the distance and the vector, we can determine the coordinates of the moving target. If the coordinates of the moving target are obtained twice in succession, based on the two coordinates, we can obtain the moving parameters of the moving target, such as moving speed, position, moving direction, etc.

[0117] Based on the movement parameters and the categories of the moving targets obtained above, we can send warning messages so that the on-duty personnel can be alert and take countermeasures as early as possible.

[0118] The implementation method of the image recognition and early warning method for the power line protection zone of the present invention first obtains multiple first images, identifies the moving target and the position of the moving target based on the multiple first images, and then accurately focuses on the moving target based on the position of the moving target, and then obtains the focus parameters, and obtains the moving parameters of the moving target through the focus parameters; finally, a warning message is issued according to the moving parameters. It can be seen that based on the analysis of the image, it is possible to quickly determine whether there is a moving target and the position of the moving target in the power line protection zone, and focus on the moving target through the position to obtain the moving parameters of the moving target. The image recognition method is relatively fast and can intelligently give the moving parameters of the moving target, which is convenient for taking targeted countermeasures according to the moving parameters of the moving target.

[0119] It should be understood that the size of the serial numbers of the steps in the above implementation does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the implementation method of the present invention.

[0120] The following is an implementation of the device of the present invention. For details not described in detail, reference may be made to the corresponding method implementation described above.

[0121] Figure 5 This is a functional block diagram of the power line protection zone image recognition and early warning device provided by the embodiment of the present invention, referring to Figure 5 The power line protection zone image recognition and early warning device 5 includes: an image acquisition module 501, a target recognition module 502, a focus module 503, a focus parameter acquisition module 504 and an early warning output module 505.

[0122] An image acquisition module 501 is used to acquire a plurality of first images, wherein the plurality of first images are images acquired at different times for a predetermined field of view within the protection area;

[0123] A target recognition module 502, configured to recognize a moving target and a position of the moving target based on the plurality of first images;

[0124] A focusing module 503, configured to accurately focus on the moving target based on the position of the moving target;

[0125] A focus parameter acquisition module 504 is used to acquire a focus parameter, and obtain a movement parameter of the moving target through the focus parameter; and

[0126] The warning output module 505 is used to issue a warning message according to the movement parameters.

[0127] Figure 6 is a functional block diagram of a terminal provided by an embodiment of the present invention. Figure 6 As shown, the terminal 6 of this embodiment includes: a processor 600 and a memory 601, and the memory 601 stores a computer program 602 that can be run on the processor 600. When the processor 600 executes the computer program 602, the steps in the above-mentioned power line protection area image recognition warning method and implementation method are implemented, for example Figure 1 Steps 101 to 105 are shown.

[0128] Exemplarily, the computer program 602 may be divided into one or more modules / units, and the one or more modules / units are stored in the memory 601 and executed by the processor 600 to implement the present invention.

[0129] The terminal 6 may be a computing device such as a desktop computer, a notebook, a PDA, or a cloud server. The terminal 6 may include, but is not limited to, a processor 600 and a memory 601. Those skilled in the art will appreciate that Figure 6 It is only an example of terminal 6 and does not constitute a limitation on terminal 6. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the terminal may also include input and output devices, network access devices, buses, etc.

[0130] The processor 600 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0131] The memory 601 may be an internal storage unit of the terminal 6, such as a hard disk or memory of the terminal 6. The memory 601 may also be an external storage device of the terminal 6, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the terminal 6. Further, the memory 601 may also include both an internal storage unit of the terminal 6 and an external storage device. The memory 601 is used to store the computer program and other programs and data required by the terminal. The memory 601 may also be used to temporarily store data that has been output or is to be output.

[0132] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In actual applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the implementation method can be integrated into a processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method implementation method, which will not be repeated here.

[0133] In the above-mentioned embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0134] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0135] In the embodiments provided by the present invention, it should be understood that the disclosed devices / terminals and methods can be implemented in other ways. For example, the device / terminal embodiments described above are only schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0136] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0137] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0138] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention implements all or part of the processes in the above-mentioned implementation method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, it can implement the steps of the above-mentioned power line protection area image recognition and early warning method and power line protection area image recognition and early warning device implementation. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device that can carry the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practices in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practices, computer-readable media does not include electrical carrier signals and telecommunication signals.

[0139] The above-described embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention, and should all be included in the protection scope of the present invention.

Claims

1. A method for early warning of power line protection zone image recognition, characterized in that: include: Acquire a plurality of first images, wherein the plurality of first images are images acquired at different times for a predetermined field of view in the protection area; Based on the plurality of first images, identifying a moving object and a position of the moving object; Based on the position of the moving target, accurately focusing the moving target; Acquiring a focus parameter, and obtaining a movement parameter of the moving target through the focus parameter; issuing a warning message according to the movement parameters; in, The step of identifying a moving target based on the plurality of first images includes: Acquire a picture taking frame and select two images from the plurality of first images, wherein the picture taking frame is a matrix having a plurality of rows and columns; Taking the image extraction frame as a mask, extracting a first selected area and a second selected area corresponding to the upper left corners of the two images respectively, wherein the first selected area and the second selected area correspond to the two images respectively; The difference matrix calculation step is as follows: calculating the difference between the first selected area and the second selected area as a first difference matrix, wherein the elements in the first difference matrix are the absolute values ​​of the differences between the pixel values ​​at the corresponding positions of the first selected area and the pixel values ​​at the corresponding positions of the second selected area; The step of calculating the selection kernel is as follows: calculating the sum of the elements in the first difference matrix as the first selection kernel; If the image taking frame position does not reach the lower right corner, the image taking frame is moved according to a predetermined moving distance, a first selection area and a second selection area are selected from the two images, and the process jumps to the difference matrix calculation step, wherein the first selection area and the second selection area correspond to the two images respectively; Selecting two largest first selection area cores from the multiple first selection area cores obtained in the selection area core calculation step; If the two largest first selection area cores both exceed the threshold, then there is a moving target in the predetermined field of view; in, The two images include a latest image, and the identifying the position of the moving target based on the plurality of first images includes: Obtain two positions of the image acquisition frame corresponding to the two largest first selection area cores in the latest image, respectively as a first position and a second position; Taking the image extraction frame as a mask, extracting a third selected area corresponding to the first position from the latest image, and extracting a fourth selected area corresponding to the second position from the other of the two images; Calculate the absolute value of the difference between the third selected area and the fourth selected area as a second difference matrix, wherein the elements in the second difference matrix are the absolute values ​​of the differences between the pixel values ​​at the corresponding positions of the third selected area and the pixel values ​​at the corresponding positions of the fourth selected area; Calculate the sum of each element in the second difference matrix as the second selection area kernel; If the second selection area kernel is smaller than the threshold, the position of the moving target is the position of the first position; Otherwise, the position of the moving target is the position of the second position.

2. The power line protection zone image recognition and early warning method according to claim 1 is characterized in that: The step of accurately focusing the moving target based on the position of the moving target includes: Based on the position of the moving target, acquiring a second image, wherein the second image includes an image of the moving target; identifying a main body of the moving target according to the second image; Based on the subject, accurate focusing on the moving target is achieved.

3. The power line protection zone image recognition and early warning method according to claim 2 is characterized in that: The step of identifying the main body of the moving target according to the second image includes: Acquire a recognition model, wherein the recognition model is an artificial neural network trained with moving target images; The second image is input into the recognition model to obtain the category of the moving target and the position of the image of the moving target in the second image.

4. The method for early warning of power line protection zone image recognition according to claim 3, characterized in that: The obtaining of the recognition model comprises: Acquire a plurality of image samples, wherein the plurality of image samples include a moving target image; Setting a label for identifying a moving target for each image in the plurality of image samples, the label comprising a moving target category label and a label for a position of the moving target in the image; Input the plurality of image samples into the recognition model, wherein the recognition model is a neural network having multiple convolutional layers, pooling layers, and fully connected layers, wherein the multiple convolutional layers are used to extract image features, the pooling layers are used to further extract image features to obtain main features, and the fully connected layers are used to classify images based on the main features; According to the label of each image in the multiple image samples and the output of the recognition model, the parameters of the recognition model are adjusted until the deviation between the output of the recognition model and the label of the image in the multiple image samples is less than a threshold.

5. The power line protection zone image recognition and early warning method according to claim 3 or 4, characterized in that: The completing accurate focusing on the moving target based on the subject includes: According to the position of the image of the moving target in the second image, shifting the field of view so that the image of the moving target is located at the center of the second image; The focus is adjusted based on the subject to obtain a clear image of the subject.

6. A power line protection zone image recognition warning device, characterized in that: Used to implement the power line protection area image recognition and early warning method according to any one of claims 1 to 5, the power line protection area image recognition and early warning device comprises: An image acquisition module, used to acquire a plurality of first images, wherein the plurality of first images are images acquired at different times for a predetermined field of view within the protection area; A target recognition module, configured to recognize a moving target and a position of the moving target based on the plurality of first images; A focusing module, used for accurately focusing on the moving target based on the position of the moving target; A focus parameter acquisition module, used to acquire focus parameters, and obtain movement parameters of the moving target through the focus parameters; and The warning output module is used to issue a warning message according to the movement parameters.

7. A terminal comprising a memory and a processor, wherein the memory stores a computer program that can be run on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 5 are implemented.

8. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

Images