Terminal detection and identification method and device, computer device, and storage medium

Abstract

The application provides a wiring terminal detection and recognition method and device, computer equipment and a computer readable storage medium. The method comprises: acquiring a wiring terminal image; presetting a feature pyramid network to perform feature detection on the wiring terminal image to obtain a plurality of feature maps of different dimensions; performing cascade processing on the feature maps to obtain an image feature; extracting a text region image from the wiring terminal image according to the image feature; and performing character recognition on the text region image through a preset convolution recurrent neural network. After the wiring terminal image is acquired, the feature maps of different dimensions are obtained, so that the wiring terminal is divided into a plurality of different parts. Detecting these parts can effectively avoid the problem of low text region detection accuracy caused by the bending and tilting of the wiring terminal, the dense arrangement, the different lengths and colors, thereby improving the accuracy of wiring terminal detection and recognition.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of text detection and recognition, and in particular relates to a terminal detection and recognition method and device, a computer device and a storage medium. BACKGROUND

[0002] In the process of smart grid construction, the informatization and digitization of substations is an essential part, but in the process of cable connection acceptance management of the secondary protection screen cabinet of the substation, the installation personnel need to check the secondary connection with the as-built drawing (including internal connection of the manufacturer), which is a huge workload.

[0003] The terminal is an insulating component in the secondary system of the substation, mainly used to connect the internal equipment of the screen cabinet and the external connection, and correct connection is particularly important for the safe and stable operation of the power system. However, when the cable connection of the secondary protection screen cabinet of the substation is accepted, the terminal number needs to be checked one by one with the as-built drawing, which is a huge workload. At present, the manual checking method not only has low efficiency, but also is prone to missed detection and misdiagnosis. With the development of deep learning, natural scene text recognition technology can be used for acceptance work, but the environment of the substation is complex, the cable arrangement is not standardized, there are problems such as bending and tilting of the terminal, dense arrangement, different lengths and different colors, and the existing detection algorithm cannot accurately locate the terminal number pipe position, which ultimately leads to poor terminal recognition effect. SUMMARY

[0004] The main purpose of the embodiment of the present application is to provide a terminal detection and recognition method, device, computer device and computer readable storage medium, to improve the accuracy of terminal detection and recognition in complex environments.

[0005] To achieve the above purpose, the first aspect of the embodiment of the present application provides a terminal detection and recognition method, which comprises:

[0006] obtaining a terminal image;

[0007] detecting the features of the terminal image by a preset feature pyramid network to obtain a plurality of feature maps with different dimensions, wherein the dimension represents the proportion of the size of the region in the terminal image, and a plurality of feature maps with the same dimension do not overlap and can constitute a complete terminal image after splicing;

[0008] performing cascade processing on the feature maps to obtain an image feature;

[0009] extracting a text region image from the terminal image according to the image feature;

[0010] The text area image is subjected to character recognition through a preset convolutional recurrent neural network.

[0011] In some embodiments, the preset feature pyramid network takes a Swin Transformer module as a backbone network, the Swin Transformer module is designed as an N-layer structure, and the feature detection is performed on the terminal image through the preset feature pyramid network to obtain a plurality of feature maps of different dimensions, wherein the feature maps of different dimensions correspond to regions of different size proportions in the terminal image, including:

[0012] The terminal image is segmented into a plurality of tiles;

[0013] The tiles are merged through the first layer of the Swin Transformer module to obtain a one-dimensional feature map;

[0014] The n-dimensional feature map generated by the nth layer of the Swin Transformer module is input to the n+1th layer, and the n-dimensional feature map is merged in the n+1th layer to obtain an n+1-dimensional feature map, so as to obtain the plurality of feature maps of different dimensions, wherein 1≤n≤N-1.

[0015] In some embodiments, an SE block module is arranged at the output channel of each layer of the Swin Transfoemer module, the SE block module includes a compression module and an excitation module, and before the cascading processing of the feature map to obtain the image feature, further including:

[0016] The feature map is subjected to global average pooling through the compression module to obtain a feature vector;

[0017] The feature vector is subjected to normalization processing through the excitation module to obtain a first weight;

[0018] The feature map is updated according to the first weight.

[0019] In some embodiments, before the feature map is subjected to global average pooling through the compression module to obtain a feature vector, further including:

[0020] The feature map is subjected to up-sampling processing to improve the resolution of the feature map.

[0021] In some embodiments, the text area is extracted from the terminal image according to the image feature, including:

[0022] A probability map and an adaptive threshold map are determined according to the image feature;

[0023] determine an approximate binary map according to the probability map and the adaptive threshold map;

[0024] determine the text region from the terminal image according to the approximate binary map.

[0025] In some embodiments, the preset convolution recurrent neural network comprises a convolution layer, a recurrent layer and a transcription layer, wherein the convolution layer comprises a ResNet module and an SE block module, and the text recognition on the text region image through the preset convolution recurrent neural network comprises:

[0026] extracting a text feature map from the text region image through the ResNet module;

[0027] determining a second weight of each text feature map through the SE block module;

[0028] determining a feature sequence according to the second weight and the text feature map;

[0029] determining a label distribution of the feature sequence through the recurrent layer;

[0030] obtaining a text recognition result according to the label distribution through the transcription layer.

[0031] In some embodiments, before the text recognition on the text region image through the preset convolution recurrent neural network, the method further comprises:

[0032] determining a Hough straight line in the text region image;

[0033] calculating the inclination of the Hough straight line;

[0034] correcting the inclination of the text region image according to the inclination of the Hough straight line;

[0035] removing the edge redundant information of the text region image through a projection segmentation algorithm.

[0036] A second aspect of the embodiment of the present application further proposes a terminal detection and recognition device, which comprises:

[0037] an acquisition unit configured to acquire a terminal image;

[0038] a feature detection unit configured to perform feature detection on the terminal image through a preset feature pyramid network to obtain a plurality of feature maps of different dimensions, wherein the dimension represents the proportion of the area size of the feature map in the terminal image, and a plurality of feature maps of the same dimension do not overlap and can constitute a complete terminal image after splicing.

[0039] An image feature determination unit is configured to perform cascade processing on the feature maps to obtain image features.

[0040] An extraction unit is configured to extract a text region image from the terminal image according to the image features.

[0041] A character recognition unit is configured to perform character recognition on the text region image by using a preset convolutional recurrent neural network.

[0042] A third aspect of the embodiment of the present application provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the terminal detection and recognition method according to any one of the first aspect when executing the computer program.

[0043] A fourth aspect of the embodiment of the present application provides a computer readable storage medium, which stores one or more programs executable by one or more processors to implement the terminal detection and recognition method according to any one of the first aspect.

[0044] In the embodiment of the present application, a terminal detection and recognition method, device, computer device, and computer readable storage medium are provided, and the method includes the following steps: obtaining a terminal image; performing feature detection on the terminal image by using a preset feature pyramid network to obtain a plurality of feature maps with different dimensions, wherein the dimension represents the proportion of the size of the region of the feature map in the terminal image, and the feature maps with the same dimension do not overlap and can constitute a complete terminal image after splicing; performing cascade processing on the feature maps to obtain image features; extracting a text region image from the terminal image according to the image features; and performing character recognition on the text region image by using a preset convolutional recurrent neural network. After obtaining the terminal image, the feature maps with different dimensions are obtained, and the different dimensions correspond to different sizes of the proportion of the region of the feature map in the terminal image. In this way, the terminal image is divided into a plurality of images with different sizes, so that the cables arranged in an irregular manner, the terminal with a curved and inclined shape, a dense arrangement, different lengths, and different colors are decomposed into a plurality of feature maps. After the terminal is divided into a plurality of different parts, the parts are detected, which can effectively avoid the problem of low accuracy of text region detection caused by the curved and inclined shape, dense arrangement, different lengths, and different colors of the terminal, thereby improving the accuracy of terminal detection and recognition. BRIEF DESCRIPTION OF DRAWINGS

[0045] Figure 1 is a flowchart of the terminal detection and recognition method according to the embodiment of the present application;

[0046] Figure 2is a feature pyramid network architecture schematic diagram of an embodiment of the present application;

[0047] Figure 3 is a Swin Transformer module architecture schematic diagram of an embodiment of the present application;

[0048] Figure 4 is a ResNet module principle diagram of an embodiment of the present application adding a SE block module;

[0049] Figure 5 is a subflowchart of an embodiment of the present application;

[0050] Figure 6 is a SE block module architecture schematic diagram of an embodiment of the present application;

[0051] Figure 7 is a subflowchart of an embodiment of the present application;

[0052] Figure 8 is a device structure schematic diagram of an embodiment of the present application;

[0053] Figure 9 is a computer device structure schematic diagram of an embodiment of the present application;

[0054] The accompanying drawings are used to provide a further understanding of the technical solutions of the present application, and constitute a part of the specification, and are used to explain the technical solutions of the present application together with the embodiments of the present application, and do not constitute a limitation on the technical solutions of the present application. DETAILED DESCRIPTION

[0055] In order to make the objects, technical solutions and advantages of the present application more clear, the present application is further described in detail below in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, and do not limit the present application.

[0056] It should be noted that although the functional modules are divided in the device schematic diagram, and the logical order is shown in the flowchart, in some cases, the steps shown or described in the module division in the device or the order of operation in the flowchart can be different. The terms "first", "second", etc. in the specification and claims and the above drawings are used to distinguish similar objects, and do not necessarily describe a specific order or sequence.

[0057] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present application belongs. The terms used herein are only for the purpose of describing the embodiments of the present application, and are not intended to limit the present application.

[0058] Moreover, the described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to give a thorough understanding of embodiments of the disclosure. One skilled in the relevant art will recognize, however, that the techniques described can be practiced without one or more of the specific details, or with other methods, components, materials, and so forth. In other instances, well-known structures, devices, implementations, or operations are not shown or described in detail to avoid obscuring aspects of the disclosure.

[0059] The block diagrams in the drawings show only the functionality of the features and can not imply that the described functionality must be implemented in a specific manner. For example, the functionality can be implemented in software, hardware, or a combination thereof. Furthermore, the described features can be combined in any suitable manner in one or more embodiments.

[0060] The flow diagrams depicted in the drawings show the functionality of the described embodiments and are not necessarily limited to the specific order of steps or sequence of steps described in the drawings. For example, the steps of the flow diagrams can be performed in an order other than that depicted in the drawings. Further, some steps can be combined, some steps can be performed in parallel, and some steps can be omitted.

[0061] Embodiments of the disclosure can be used in a variety of computing systems and environments. For example, embodiments of the disclosure can be used in personal computers, server computers, hand-held or laptop devices, tablet devices, multiprocessor systems, microprocessor-based systems, set top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments that include any of the above systems or devices, and the like. Embodiments of the disclosure can be described in the general context of computer-executable instructions, such as program modules, being executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, and the like, that perform particular tasks or implement particular abstract data types. Embodiments of the disclosure can also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules can be located in both local and remote computer storage media including memory storage devices.

[0062] With reference to Figure 1 The wiring terminal detection and identification method of the first aspect of the embodiments of the present application includes, but is not limited to, steps S100 to S500

[0063] In step S100, a wiring terminal image is acquired.

[0064] In step S100 of some embodiments, the terminal connection image can be obtained by shooting through the integrated device mounted at the transformer substation. In some embodiments, when the preset pyramid feature network is in the training stage, the terminal connection image can also be labeled after the terminal connection image is obtained, and the labeled terminal connection image is constructed as a terminal connection dataset to supervise the learning of the preset pyramid feature network.

[0065] In step S200, the terminal connection image is detected by the preset feature pyramid network to obtain a plurality of feature maps of different dimensions, wherein the dimension represents the proportion of the area size of the feature map in the terminal connection image, and the feature maps of the same dimension do not overlap and can constitute a complete terminal connection image after splicing.

[0066] In step S200 of some embodiments, referring to Figure 2 , the preset feature pyramid network takes the Swin Transformer module as the backbone network, and the Swin Transformer module is designed as an N-layer structure. Step S200 includes: dividing the terminal connection image into a plurality of patches; merging the patches by the first layer of the Swin Transformer module to obtain a one-dimensional feature map; inputting the n-dimensional feature map generated by the nth layer of the Swin Transformer module into the n+1th layer, and merging the n-dimensional feature map in the n+1th layer to obtain an n+1-dimensional feature map, so as to obtain a plurality of feature maps of different dimensions, wherein 1≤n≤N-1. Specifically, Figure 2 is the architecture of the Swin Transformer module proposed in the embodiments of the present application. Taking N=4 as an example, the input image is first divided into non-overlapping patches by a patch partition module, and then embedded into an embedding, wherein the patch size is set to 4x4, and the feature dimension of each patch is 4x4x3=48. The original value feature enters "Stage 1", first passes through a linear embedding layer, and sets the output dimension to 96, then is sent into 2 consecutive Swin Transformer blocks with improved self-attention mechanism, and the resolution is kept to be where H represents the height of the picture and W represents the width of the picture. To generate a hierarchical representation, as the network deepens, the number of patches is gradually reduced through a patch merging layer. The first patch merging layer splices the features of every group of 2x2 adjacent patches, and applies a linear layer on the 384-dimensional connection features, which reduces the number of patches by 2x2=4 times, and sets the output dimension to 192. Then, the Swin Transformer is applied for feature transformation, and the resolution is set to The first tile merging and feature conversion is called "Stage 2". The process is repeated twice, which are "Stage 3" and "Stage 4", and the output resolutions are and Based on this, the feature map of the corresponding dimension is generated at each stage.

[0067] At step S300, the feature map is cascaded to obtain the image feature;

[0068] Specifically, referring to Figure 2 , the feature map is cascaded by the concat function to obtain the image feature F.

[0069] At step S400, the text region image is extracted from the terminal image according to the image feature;

[0070] In step S400 of some embodiments, referring to the feature pyramid network architecture diagram shown in Figure 2 , the probability map and the adaptive threshold map are determined according to the image feature; the approximate binary map is determined according to the probability map and the adaptive threshold map; and the text region is determined from the terminal image according to the approximate binary map. Specifically, the image feature F is passed through the convolution layer, and the output channel is compressed to 1 / 4 of the input, the BN (Batch-normalization) layer and the relu function, to obtain the first shape of the image feature F. Then the image feature F is deconvolved to obtain the second shape of the image feature F. The output channel is set to 1, and the image feature F is deconvolved to obtain the third shape of the image feature F. Finally, the probability map is obtained by normalizing the image feature F through the sigmoid function.

[0071] After obtaining the probability map and the adaptive threshold map, the approximate binary map can be calculated according to the following formula,

[0072]

[0073] where B i,j is the approximate binary mapping, T is the adaptive threshold map, T i,j represents the element of the i-th row and j-th column in the adaptive threshold map, P is the probability map, and P i,jdenotes the element of the i-th row and j-th column in the probability map, k is a magnification factor, which is generally set to 50, and the approximate binary image has a shape similar to that of the standard binary function, but compared with the standard binary function, it can be differentiated and optimized in the training stage. Based on this, for terminal text detection, the differentiable binary with adaptive threshold can better distinguish the terminal text region from the complex background, so as to better detect the text region in the terminal image. In addition, based on the method of the differentiable binary with adaptive threshold, the closely connected terminal text regions can be better separated, and the accuracy of text region detection is improved.

[0074] In step S500, the text region image is recognized by the preset convolution recurrent neural network.

[0075] In step S500 of some embodiments, the preset convolution recurrent neural network includes a convolution layer, a recurrent layer and a transcription layer, wherein the convolution layer includes a ResNet module and an SE block module, the ResNet module is used to extract a text feature map from the text region image; the SE block module is used to determine a second weight of each text feature map; a feature sequence is determined according to the second weight and the text feature map; the recurrent layer is used to determine a label distribution of the feature sequence; and the transcription layer is used to obtain a text recognition result according to the label distribution.

[0076] Reference Figure 3 , the ResNet module added with the SE block module is as shown in Figure 3 In the embodiment of the application, after obtaining the input feature, i.e., the text region image, the text region image is globally averaged and pooled, and then reduced in dimension by a first fully connected layer, activated by a ReLu function, and then increased in dimension by a second fully connected layer, and then normalized by a Sigmod function, so as to determine the second weight of each channel of the text region image, and to weight the channels of the text region image according to the second weight, thereby completing the update of the text region image, highlighting the important feature part in the image and weakening the non-important feature part in the image. Based on this, the improved convolution layer has stronger feature extraction capability compared with the conventional CNN convolution layer, and can more accurately extract the text part in the text region image, so as to improve the accuracy of subsequent text recognition by the recurrent layer RNN and the transcription layer CTC.

[0077] In the embodiment of the present application, by improving the feature pyramid network, the Swin Transformer is introduced as the backbone network of the feature pyramid network, a plurality of feature maps of different dimensions are extracted from the terminal connection image, and the feature maps are cascaded to obtain the image feature, the text area image is extracted from the text area image according to the image feature, and the terminal connection sequence is obtained by recognizing the text in the text area image. Through the feature pyramid network, a plurality of feature maps of different dimensions are extracted from the image, and the features of different dimensions are cascaded to obtain the feature of the image. Based on this, the accurate detection of the text sequence in the terminal connection image is realized, and the sequence text of the terminal connection is obtained by recognizing the text in the text area image detected by the preset convolutional recurrent neural network (CRNN).

[0078] In the embodiment of the present application, the SE block module is further arranged at the output channel of the Swin Transformer module, so as to introduce the self-attention mechanism, calculate the first weight of each channel of each feature map output by the Swin Transformer module, and update the feature map according to the first weight, highlight the important feature part of the feature map, and weaken the non-important feature part, so as to improve the accuracy of the text area detection in the terminal connection image.

[0079] In the embodiment of the present application, the convolutional recurrent neural network is also improved, ResNet is used as the feature extraction method in the convolutional layer, and the SE block module is arranged, so as to introduce the self-attention mechanism, determine the second weight of each channel of the text area image, and update the text area image according to the second weight, highlight the important feature part therein, and weaken the non-important feature part therein, so as to improve the accuracy when recognizing the text by the recurrent layer RNN and the transcription layer CTC subsequently.

[0080] Reference Figure 5 In some embodiments, the SE block module is arranged at the output channel of each layer of the Swin Transfoemer module, the SE block module includes a compression module and an excitation module, and before step S300, the following steps S501 to S503 are further included, but are not limited to.

[0081] In step S501, the feature map is globally averaged and pooled by the compression module to obtain a feature vector;

[0082] In step S502, the feature vector is normalized by the excitation module to obtain a first weight;

[0083] Specifically, the excitation module includes a first fully connected layer and a Relu activation function, a second fully connected layer and a Sigmod function, and reference Figure 6 , Figure 6As a schematic diagram of the SE block module, the feature vector is reduced in dimension when passing through the first fully connected layer and is activated by the relu activation function, the specific form of the relu activation function is: f(x) = max(0, x), and then is increased in dimension to the original dimension when passing through the second fully connected layer, and then is normalized by the Sigmod activation function, so as to obtain the first weight of each channel of the feature map, which reflects the importance of each channel of the feature map, wherein the channel refers to the output channel of the image, specifically, it can be an RGB channel.

[0084] In step S503, the feature map is updated according to the first weight.

[0085] The embodiment of the application introduces the SE block module to calculate the weight of each image channel of each feature map, updates the output of each channel of the feature map based on the first weight, strengthens the important feature part in the feature map, and weakens the non-important feature part, thereby completing the update of the feature map, so that the feature map can better reflect the important feature part of the terminal image, and the ability of extracting image features is greatly improved.

[0086] In the embodiment of the application, by setting the SE block module at the output end of the Swin Transformer module, the first weight of each channel of the feature map is determined by introducing the self-attention mechanism, and the feature map is updated based on the first weight, the important feature part of the feature map is strengthened, and the non-important feature part of the feature map is weakened, thereby realizing the update of the feature map, greatly improving the ability of the feature pyramid network to extract the terminal image features, and improving the ability to detect the text region image in the terminal image.

[0087] It can be understood that, since in step S200, after the terminal image passes through the Swin Transformer, the resolution of the terminal image is reduced to expand the receptive field of the feature pyramid network, which will cause distortion of image details, loss of details and affect the first weight determined by the feature vector of the feature map, so that the first weight cannot reflect the importance of the feature map. Therefore, before step S201, the feature map is up-sampled to improve the resolution of the feature map. Specifically, the unknown pixel points can be predicted according to the known pixel points, and the pixel points can be inserted to realize up-sampling. In some embodiments, the up-sampling can also be realized by setting a convolution kernel to perform transpose convolution on the image matrix of the feature map. Specifically, after up-sampling the feature map, the compressed resolution feature map can be restored to the original resolution of the terminal image, so as to ensure that the image can preserve the details in the original terminal image, and prevent the problem of loss of image features due to loss of details.

[0088] In the embodiment of the present application, the SE block module is set by the output notification of the Swin Transformer module, so as to introduce the self-attention mechanism to determine the first weight of each channel of the feature map, and each channel of the feature map is weighted according to the first weight, so as to highlight the important feature part in the feature map and weaken the non-important feature part, thereby updating the feature map, so that the feature map can better reflect the text area of the terminal image, thereby improving the ability of detecting the text area of the terminal image. At the same time, since the image resolution is reduced when the feature map is extracted from the terminal image by the Swin Transformer module, the image details are lost, therefore, before calculating the first weight of the feature map, the feature map is up-sampled to improve the resolution of the feature map, so as to prevent the loss of image features.

[0089] With reference to Figure 7 In some embodiments, before step S500, the following steps S701 to S704 are further included, but are not limited to.

[0090] Step S701, determining a Hough straight line in the text area image;

[0091] Step S702, calculating the inclination of the Hough straight line;

[0092] Step S703, correcting the inclination of the text area image according to the inclination of the Hough straight line;

[0093] Step S704, removing the edge redundant information of the text area image by a projection segmentation algorithm.

[0094] It can be understood that due to the complex environment of the transformer substation site, there are situations such as non-standard cable arrangement, bent and inclined terminal, etc., and the extracted text area image also has the same inclination, which leads to poor text recognition effect. Based on this, in the embodiment of the present application, the Hough transform is performed on the text area image to extract the Hough straight line in the text area image, and the specific inclination of the text in the text area image is calculated according to the inclination of the Hough straight line, and the text area image is corrected according to the inclination, so that the subsequent text recognition effect is better.

[0095] It can be understood that, in the process of text detection, due to the bending of the terminal and the like, when the text area is extracted according to the approximate binary graph, a large amount of invalid redundant information may be contained in the detected text area. When the text image is sent to the CRNN network for character recognition, the image is generally compressed at a constant ratio. If there is too much edge redundant information, the originally small text area will become more blurred, the font will be deformed, and even compressed into a character group, thereby causing poor character recognition effect. Therefore, before character recognition, the edge redundant information of the text area image is cut off by the projection segmentation algorithm.

[0096] In the embodiment of the application, after detecting the text area image from the terminal image, the inclination of the text area image is corrected by the Hough straight line detection before the text area image is recognized. In addition, the edge redundant information of the text area image is cut off by the projection segmentation algorithm. Therefore, the accuracy of character recognition can be effectively improved.

[0097] The embodiment of the present disclosure also provides a terminal detection and recognition device, as shown in Figure 8 The terminal detection and recognition device can realize the terminal detection and recognition method described above. The terminal detection and recognition device comprises: an acquisition unit 810 configured to acquire a terminal image; a feature detection unit 820 configured to perform feature detection on the terminal image by using a preset feature pyramid network to obtain a plurality of feature maps with different dimensions, wherein the dimension represents the proportion of the size of the region of the feature map in the terminal image, and a plurality of feature maps with the same dimension do not overlap and can constitute a complete terminal image after splicing; an image feature determination unit 830 configured to perform cascaded processing on the feature maps to obtain an image feature; an extraction unit 840 configured to extract a text area image from the terminal image according to the image feature; and a character recognition unit 850 configured to perform character recognition on the text area image by using a preset convolutional recurrent neural network. The terminal detection and recognition device of the embodiment of the present disclosure is used to execute the terminal detection and recognition method in the above embodiment, and the specific processing process is the same as that of the terminal detection and recognition method in the above embodiment, which will not be described here.

[0098] With reference to Figure 9 , the embodiment of the present application also provides an electronic device 900, comprising:

[0099] at least one processor, and

[0100] a memory in communication connection with the at least one processor; wherein

[0101] The memory stores instructions, and the instructions are executed by the at least one processor to enable the at least one processor to implement the method of any one of the fourth aspect embodiments of the present application.

[0102] The following describes the hardware structure of the electronic device 900 in detail. The computer device includes a processor 910, a memory 920, an input / output interface 930, a communication interface 940, and a bus 950. Figure 9 The hardware structure of the electronic device 900 is described in detail. The computer device includes a processor 910, a memory 920, an input / output interface 930, a communication interface 940, and a bus 950.

[0103] The processor 910 can be implemented in the form of a general-purpose central processing unit (CPU), a microprocessor, an application specific integrated circuit (ASIC), or one or more integrated circuits, and is configured to execute related programs to implement the technical solutions provided by the embodiments of the present disclosure.

[0104] The memory 920 can be implemented in the form of a read only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 920 can store an operating system and other application programs. When the technical solutions provided by the embodiments of the present disclosure are implemented by software or firmware, the related program codes are stored in the memory 920 and are called and executed by the processor 910 to implement the terminal detection and identification method of the embodiments of the present disclosure.

[0105] The input / output interface 930 is configured to realize information input and output.

[0106] The communication interface 940 is configured to realize the communication interaction between the device and other devices. The communication can be realized by a wired manner (for example, a USB, a network cable, etc.) or a wireless manner (for example, a mobile network, WIFI, Bluetooth, etc.).

[0107] The bus 950 is configured to transmit information between various components (for example, the processor 910, the memory 920, the input / output interface 930, and the communication interface 940) of the device.

[0108] The processor 910, the memory 920, the input / output interface 930, and the communication interface 940 are connected to each other through the bus 950 to realize the communication connection between them in the device.

[0109] The flowchart shown in the accompanying drawings is only an exemplary description and does not necessarily include all the contents and operations / steps, nor does it necessarily run in the order described. For example, some operations / steps can be further divided, and some operations / steps can be combined or partially combined, so that the actual running order can be changed according to the actual situation.

[0110] Those of ordinary skill in the art can understand that all or some of the steps in the method disclosed above, the functions of the modules / units in the system, the device can be implemented as software, firmware, hardware and appropriate combinations thereof.

[0111] The terms "comprising" and "having" and any variations thereof in the specification and in the claims of the present application are intended to cover a non-exclusive inclusion, for example, a process, method, system, product or apparatus that includes a list of steps or units not necessarily limited to those explicitly listed, but can include other steps or units not expressly listed or inherent to such processes, methods, products or apparatus.

[0112] It should be understood that in the present application, "at least one" means one or more, and "multiple" means two or more. "And / or" is used to describe the relationship between the associated objects, which means that there can be three relationships, for example, "A and / or B" can represent three cases: only A, only B, and A and B at the same time, where A and B can be singular or plural. The character " / " generally represents an "or" relationship between the associated objects. "At least one of the following" or similar expressions means any combination of these items, including any combination of single or multiple items. For example, at least one of a, b or c can mean a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0113] The units described as separate components can or can not be physically separated, and the components shown as units can or can not be physical units, that is, they can be located in one place, or they can be distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the present embodiment.

[0114] The preferred embodiments of the embodiments of the present application are described above with reference to the accompanying drawings, but this does not limit the scope of the embodiments of the present application. Any modifications, equivalent replacements and improvements made by those skilled in the art without departing from the scope and essence of the embodiments of the present application shall be within the scope of the embodiments of the present application.

Claims

1. A method for detecting and identifying terminal blocks, the method comprising: Obtain terminal block images; The terminal block image is subjected to feature detection by a preset feature pyramid network to obtain multiple feature maps of different dimensions. The dimension represents the proportion of the region size of the feature map in the terminal block image. Multiple feature maps of the same dimension do not overlap and can be stitched together to form a complete terminal block image. The preset feature pyramid network uses a Swing Transformer module as the backbone network. The Swing Transformer module is designed as an N-layer structure. Each output channel of the Swing Transformer module is equipped with an SE block module. The SE block module includes a compression module and an excitation module. The feature map is subjected to global average pooling by the compression module to obtain the feature vector; The feature vector is normalized by the activation module to obtain the first weight. Update the feature map according to the first weight; The feature maps are cascaded to obtain image features; Extract the text region image from the terminal image based on the image features; The text region image is used to perform text recognition by a preset convolutional recurrent neural network.

2. The method according to claim 1, characterized in that, The method involves performing feature detection on the terminal block image using a preset feature pyramid network to obtain multiple feature maps of different dimensions. These feature maps correspond to regions of different sizes and proportions within the terminal block image, including: The terminal block image is divided into multiple blocks; The tiles are merged through the first layer of the Swin Transformer module to obtain a one-dimensional feature map; The n-dimensional feature map generated by the nth layer of the Swin Transformer module is input to the (n+1)th layer, and the n-dimensional feature maps are merged in the (n+1)th layer to obtain the (n+1)-dimensional feature map, so as to obtain the multiple feature maps of different dimensions, where 1≤n≤N-1.

3. The method according to claim 1, characterized in that, Before performing global average pooling on the feature map through the compression module to obtain the feature vector, the method further includes: The feature map is upsampled to improve its resolution.

4. The method according to claim 1, characterized in that, Extracting the text region from the terminal block image based on the image features includes: Determine the probability map and adaptive threshold map based on the image features; An approximate binary map is determined based on the probability map and the adaptive threshold map; The text region is determined from the terminal block image based on the approximate binary image.

5. The method according to claim 1, characterized in that, The preset convolutional recurrent neural network includes convolutional layers, recurrent layers, and transcription layers. The convolutional layers include a ResNet module and an SE block module. The text recognition of the text region image using the preset convolutional recurrent neural network includes: The ResNet module extracts text feature maps from the text region image; The second weight of each text feature map is determined by the SE block module; A feature sequence is determined based on the second weight and the text feature map; The label distribution of the feature sequence is determined through the recurrent layer; The text recognition results are obtained through the transcription layer based on the label distribution.

6. The method according to claim 1, characterized in that, Before performing text recognition on the text region image using a preset convolutional recurrent neural network, the method further includes: Determine the Hough line in the text region image; Calculate the inclination of the Hough line; The tilt of the text region image is corrected based on the tilt of the Hough line; Redundant edge information in the text region image is removed using a projection segmentation algorithm.

7. A terminal block detection and identification device, characterized in that, The device includes: Acquisition unit, used to acquire terminal block images; The feature detection unit is used to perform feature detection on the terminal block image through a preset feature pyramid network to obtain multiple feature maps of different dimensions. The dimension represents the proportion of the region size of the feature map in the terminal block image. Multiple feature maps of the same dimension do not overlap and can be stitched together to form a complete terminal block image. The preset feature pyramid network uses a Swing Transformer module as the backbone network. The Swing Transformer module is designed as an N-layer structure. Each output channel of the Swing Transformer module is equipped with an SE block module. The SE block module includes a compression module and an excitation module. The device is further configured to perform global average pooling on the feature map through the compression module to obtain a feature vector; normalize the feature vector through the activation module to obtain a first weight; and update the feature map according to the first weight. An image feature determination unit is used to obtain image features by performing cascaded processing on the feature maps; The extraction unit is used to extract a text region image from the terminal block image based on the image features; The text recognition unit is used to perform text recognition on the text region image through a preset convolutional recurrent neural network.

8. A computer device, comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, when the processor executes the computer program, it implements the terminal block detection and identification method as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores one or more programs, which can be executed by one or more processors to implement the terminal block detection and identification method as described in any one of claims 1 to 6.

Images