Nameplate text detection method and system based on dynamic adaptive threshold
The nameplate text detection method improves accuracy and robustness by employing dynamic adaptive thresholds, salient position selection, and CAFM feature fusion to address the limitations of traditional algorithms in complex environments.
Patent Information
- Application Number
- CN202411106071.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-13
- Publication Date
- 2024-12-31
AI Technical Summary
Traditional nameplate text detection algorithms suffer from low recognition accuracy, sensitivity to lighting and noise, and limited performance in complex backgrounds, requiring large amounts of training data and complex algorithm design.
A nameplate text detection method utilizing a dynamic adaptive threshold that includes downsampling and upsampling feature maps, salient position selection, CAFM feature fusion, and adaptive threshold determination to improve accuracy and robustness.
Enhances text detection and recognition capabilities by adaptively selecting significant positions, comprehensively utilizing features, and dynamically adjusting thresholds to handle varying text styles and lighting conditions.
Smart Images
Figure CN119229425A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of image processing technology, and in particular to a nameplate text detection method and system based on dynamic adaptive thresholds. Background Art
[0002] Nameplate text detection is a key application in image processing and computer vision. Traditional nameplate text detection algorithms typically use image processing and pattern recognition techniques to identify textual information in images, thereby automatically recognizing and extracting the textual content on nameplates. This technology has wide applications in industrial production, security monitoring, intelligent transportation, and other fields. However, prior methods based on image segmentation and feature extraction typically require large amounts of training data and complex algorithm design, and suffer from problems such as low recognition accuracy and sensitivity to lighting and noise. Furthermore, this method has limited accuracy in text recognition under complex backgrounds, and may result in misidentification or missed recognition. Summary of the Invention
[0003] In order to solve the above technical problems, the purpose of the present invention is to provide a nameplate text detection method and system based on a dynamic adaptive threshold with high recognition accuracy.
[0004] To achieve the above objectives, one aspect of an embodiment of the present application provides a nameplate text detection method based on a dynamic adaptive threshold, comprising the following steps:
[0005] Obtain the image of the nameplate to be detected;
[0006] Downsampling the nameplate image to be detected according to different downsampling multiples to obtain multiple downsampling feature maps, and then upsampling each of the downsampling feature maps according to a preset upsampling multiple to obtain multiple downsampling sub-feature maps;
[0007] Inputting the downsampled feature map and the downsampled sub-feature map into a preset salient position selection module for feature selection to obtain a plurality of initial feature maps;
[0008] Inputting the downsampled feature map and the initial feature map into a preset CAFM feature fusion module for feature fusion to obtain a composite feature map;
[0009] Inputting the composite feature map into a preset prediction head module to obtain a probability map and a threshold map for predicting text;
[0010] An adaptive threshold corresponding to each first pixel point in the threshold map is determined according to the probability map, and then the probability map is binarized according to the adaptive threshold to obtain a nameplate text detection result.
[0011] In some embodiments, downsampling the nameplate image to be detected according to different downsampling multiples to obtain multiple downsampling feature maps, and then upsampling each of the downsampling feature maps according to a preset upsampling multiple to obtain multiple downsampling sub-feature maps, specifically includes:
[0012] Downsampling the nameplate image to be detected according to different downsampling multiples to obtain a first downsampling feature map, a second downsampling feature map, a third downsampling feature map, and a fourth downsampling feature map;
[0013] Upsampling the second downsampled feature map according to the upsampling multiple to obtain a second downsampled subfeature map;
[0014] Upsampling the third downsampled feature map according to the upsampling multiple to obtain a third downsampled subfeature map;
[0015] The fourth down-sampled feature map is up-sampled according to the up-sampling multiple to obtain a fourth down-sampled sub-feature map.
[0016] In some embodiments, inputting the downsampled feature map and the downsampled sub-feature map into a preset salient position selection module for feature selection to obtain a plurality of initial feature maps specifically includes:
[0017] Inputting the first down-sampled feature map and the second down-sampled sub-feature map into the salient position selection module for feature selection, and then performing element-wise addition on the first down-sampled feature map and the second down-sampled sub-feature map after feature selection to obtain a first initial feature map;
[0018] Inputting the second down-sampled feature map and the third down-sampled sub-feature map into the salient position selection module for feature selection, and then performing element-wise addition on the second down-sampled feature map and the third down-sampled sub-feature map after feature selection to obtain a second initial feature map;
[0019] The third down-sampled feature map and the fourth down-sampled sub-feature map are input into the salient position selection module for feature selection, and then the third down-sampled feature map and the fourth down-sampled sub-feature map after feature selection are element-wise added to obtain a third initial feature map.
[0020] In some embodiments, the downsampled feature map and the initial feature map are input into a preset CAFM feature fusion module for feature fusion to obtain a composite feature map, specifically:
[0021] The first initial feature map, the second initial feature map, the third initial feature map, and the fourth down-sampled feature map are up-sampled, and then the up-sampled first initial feature map, the second initial feature map, the third initial feature map, and the fourth down-sampled feature map are input into the CAFM feature fusion module for feature fusion to obtain the composite feature map.
[0022] In some embodiments, determining the adaptive threshold corresponding to each first pixel in the threshold map according to the probability map specifically includes:
[0023] Obtaining a momentum attenuation coefficient, and obtaining a predicted probability of the region corresponding to the nameplate image to be detected according to the probability map;
[0024] Calculating a weighted average of the regional prediction probabilities, and then obtaining a global threshold according to the momentum decay coefficient and the weighted average;
[0025] Calculate the predicted expected values of text area categories and non-text area categories to obtain the local threshold;
[0026] Performing maximum normalization processing on the global threshold and the local threshold to obtain the adaptive threshold.
[0027] In some embodiments, the adaptive threshold is determined by the following formula:
[0028]
[0029] Among them, τ t (c) represents the adaptive threshold corresponding to the c-th category in the t-th time step, τ t represents the global threshold corresponding to the t-th time step, represents the local threshold corresponding to the c-th category in the t-th time step, MaxNorm(·) represents the maximum normalization operation, C represents the total number of categories, and t and c are positive integers.
[0030] In some embodiments, the probability map is binarized according to the adaptive threshold to obtain the nameplate text detection result, specifically:
[0031] The adaptive threshold is used as a threshold for binarization processing, and the probability map is binarized according to the threshold to obtain the nameplate text detection result.
[0032] To achieve the above objectives, another aspect of the present application provides a nameplate text detection system based on a dynamic adaptive threshold, comprising:
[0033] An image acquisition module, used to acquire an image of the nameplate to be detected;
[0034] An image sampling module is used to downsample the nameplate image to be detected according to different downsampling multiples to obtain multiple downsampled feature maps, and then upsample each of the downsampled feature maps according to a preset upsampling multiple to obtain multiple downsampled sub-feature maps;
[0035] a feature selection module, configured to input the downsampled feature map and the downsampled sub-feature map into a preset salient position selection module for feature selection to obtain a plurality of initial feature maps;
[0036] A feature fusion module is used to input the downsampled feature map and the initial feature map into a preset CAFM feature fusion module for feature fusion to obtain a composite feature map;
[0037] A text prediction module, configured to input the composite feature map into a preset prediction head module to obtain a probability map and a threshold map for predicting text;
[0038] The binarization processing module is used to determine the adaptive threshold corresponding to each pixel point in the threshold map according to the probability map, and then perform binarization processing on the probability map according to the adaptive threshold to obtain the nameplate text detection result.
[0039] To achieve the above-mentioned purpose, another aspect of an embodiment of the present application proposes an electronic device, which includes a memory, a processor, a program stored on the memory and executable on the processor, and a data bus for realizing connection and communication between the processor and the memory. When the program is executed by the processor, the nameplate text detection method based on dynamic adaptive threshold as described above is realized.
[0040] To achieve the above-mentioned purpose, another aspect of an embodiment of the present application proposes a storage medium, which is a computer-readable storage medium used for computer-readable storage. The storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the nameplate text detection method based on dynamic adaptive threshold as described above.
[0041] The beneficial effects of the present invention are as follows: the nameplate text detection method and system based on dynamic adaptive threshold of the present invention, on the one hand, before fusing the downsampled sub-feature map and the downsampled feature map, introduces a significant position selection module to perform feature selection to generate an initial feature map, which can adaptively select a significant position in the nameplate image to be detected, thereby more accurately locating and extracting the nameplate text; on the other hand, by introducing a CAFM feature fusion module to perform feature fusion to generate a composite feature map, it can comprehensively utilize multiple features to improve the detection and recognition capabilities of the nameplate text area; in addition, before binarizing the probability map, by dynamically determining the adaptive threshold in the threshold map, it can better cope with the changes of the nameplate text under different styles, sizes and lighting conditions, thereby improving the robustness and generalization capabilities of the nameplate text detection. BRIEF DESCRIPTION OF THE DRAWINGS
[0042] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following introduction is made to the drawings required for use in the embodiments of the present invention. It should be understood that the drawings introduced below are only for the convenience of clearly describing some embodiments of the technical solutions of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without any creative work.
[0043] Figure 1 A flowchart of a method for nameplate text detection based on a dynamic adaptive threshold provided by an embodiment of the present invention;
[0044] Figure 2 A structural diagram of a nameplate text detection system based on a dynamic adaptive threshold provided by an embodiment of the present invention;
[0045] Figure 3 A schematic diagram of the hardware structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION
[0046] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application is further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the embodiments of the present application. They are merely examples of devices and methods consistent with some aspects of the embodiments of the present application as detailed in the appended claims.
[0047] It will be understood that the terms "first", "second", etc. used in this application may be used herein to describe various concepts, but unless otherwise specified, these concepts are not limited by these terms. These terms are only used to distinguish one concept from another. For example, without departing from the scope of the embodiments of the present application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the words "if" and "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".
[0048] The terms "at least one", "plurality", "each", "any", etc. used in this application include "at least one", "two" or more, "plurality" or "each", "any" or "any one", "each" or "any one" in the context of the present invention, and "at least one" or "at least one" includes one, two or more, "plurality" or "any one" includes two or more, "each" or "each one" in the context of the present invention, and "any" or "any one
[0049] Nameplate text detection is a key application in image processing and computer vision. Traditional nameplate text detection algorithms typically use image processing and pattern recognition techniques to identify text in images, automatically recognizing and extracting the text on the nameplate. This technology has widespread applications in industrial production, security monitoring, intelligent transportation, and other fields. Currently, a variety of nameplate text detection algorithms have been proposed and widely used. One common approach is based on image segmentation and feature extraction. This method first preprocesses the image and then uses techniques such as edge detection and color segmentation to separate the text area from the background. Next, feature extraction and text recognition techniques are used to identify the nameplate text content. However, prior methods based on image segmentation and feature extraction typically require large amounts of training data and complex algorithm design, resulting in low recognition accuracy and sensitivity to lighting and noise. Furthermore, these methods have limited accuracy in text recognition against complex backgrounds, potentially leading to misidentification or omission.
[0050] To this end, an embodiment of the present invention proposes a nameplate text detection method based on dynamic adaptive thresholding. On the one hand, before fusing the downsampled sub-feature map and the downsampled feature map, a salient position selection module is introduced to perform feature selection and generate an initial feature map. This allows for adaptive selection of salient locations in the nameplate image to be detected, thereby more accurately locating and extracting the nameplate text. On the other hand, a CAFM feature fusion module is introduced to perform feature fusion and generate a composite feature map. This allows for comprehensive utilization of multiple features to improve the detection and recognition capabilities of nameplate text areas. Furthermore, before binarizing the probability map, the adaptive threshold in the threshold map is dynamically determined. This allows for better response to changes in nameplate text styles, sizes, and lighting conditions, thereby improving the robustness and generalization capabilities of nameplate text detection. This nameplate text detection method can be applied to scenarios such as license plate recognition, product label recognition, and packaging label recognition, but is not limited thereto.
[0051] Reference Figure 1 , Figure 1 This is a flowchart of a method for detecting nameplate text based on a dynamic adaptive threshold provided by an embodiment of the present invention. The embodiment of the present invention provides a method for detecting nameplate text based on a dynamic adaptive threshold, which includes steps S101 to S106:
[0052] S101, obtaining an image of a nameplate to be detected;
[0053] S102, downsampling the nameplate image to be inspected according to different downsampling multiples to obtain multiple downsampled feature maps, and then upsampling each downsampled feature map according to a preset upsampling multiple to obtain multiple downsampled sub-feature maps;
[0054] As an optional implementation, the nameplate image to be inspected is downsampled according to different downsampling multiples to obtain multiple downsampled feature maps, and then each downsampled feature map is upsampled according to a preset upsampling multiple to obtain multiple downsampled sub-feature maps. This step can be specifically divided into the following steps S1021 to S1024:
[0055] S1021, downsampling the nameplate image to be inspected according to different downsampling multiples to obtain a first downsampling feature map, a second downsampling feature map, a third downsampling feature map, and a fourth downsampling feature map;
[0056] S1022. Upsample the second downsampled feature map according to the upsampling multiple to obtain a second downsampled sub-feature map;
[0057] S1023. Upsample the third downsampled feature map according to the upsampling multiple to obtain a third downsampled sub-feature map;
[0058] S1024. Upsample the fourth downsampled feature map according to the upsampling multiple to obtain a fourth downsampled sub-feature map.
[0059] Furthermore, the nameplate image to be inspected is first downsampled, and the first downsampled feature map is obtained by downsampling by 2 times, the second downsampled feature map is obtained by downsampling by 4 times, the third downsampled feature map is obtained by downsampling by 16 times, and the fourth downsampled feature map is obtained by downsampling by 32 times. Then, the second downsampled feature map downsampled by 8 times is upsampled by 2 times (i.e., the upsampling multiple) to obtain the second downsampled sub-feature map downsampled by 4 times; the third downsampled feature map downsampled by 16 times is upsampled by 2 times (i.e., the upsampling multiple) to obtain the third downsampled sub-feature map downsampled by 8 times; the fourth downsampled feature map downsampled by 32 times is upsampled by 2 times (i.e., the upsampling multiple) to obtain the fourth downsampled sub-feature map downsampled by 16 times.
[0060] S103, inputting the downsampled feature map and the downsampled sub-feature map into a preset salient position selection module for feature selection to obtain multiple initial feature maps;
[0061] As an optional implementation, the downsampled feature map and the downsampled sub-feature map are input into a preset salient position selection module for feature selection to obtain multiple initial feature maps. This step can be specifically divided into the following steps S1031 to S1033:
[0062] S1031, inputting the first down-sampled feature map and the second down-sampled sub-feature map into a salient position selection module for feature selection, and then performing element-wise addition on the first down-sampled feature map and the second down-sampled sub-feature map after feature selection to obtain a first initial feature map;
[0063] S1032: Input the second down-sampled feature map and the third down-sampled sub-feature map into a salient position selection module for feature selection, and then perform element-wise addition on the second down-sampled feature map and the third down-sampled sub-feature map after feature selection to obtain a second initial feature map;
[0064] S1033. Input the third down-sampled feature map and the fourth down-sampled sub-feature map into a salient position selection module for feature selection, and then perform element-wise addition on the third down-sampled feature map and the fourth down-sampled sub-feature map after feature selection to obtain a third initial feature map.
[0065] Furthermore, the second downsampled sub-feature map that is downsampled 4 times is combined with the first downsampled feature map that is downsampled 4 times in the previous layer. Figure 1 The first initial feature map is obtained by adding the two parts after feature selection element-wise. The third down-sampled sub-feature map that is down-sampled 8 times is added to the second down-sampled sub-feature map that is down-sampled 8 times of the previous layer. Figure 1 The first sub-feature map is input into the salient position selection module, and the two parts after feature selection are element-wise added to obtain the second initial feature map; the fourth sub-feature map of the downsampled 16 times is added to the third sub-feature map of the previous layer of downsampled 16 times. Figure 1 The two parts after feature selection are added element-wise to obtain the third initial feature map.
[0066] Among them, the salient position selection module uses the custom neural network layer of SPABlock (Salient Positions-based Attention), which selects significant positions or features from the input data by using the SPS (Salient Positions Selection) algorithm to improve the performance and efficiency of the network.
[0067] It should be noted that before adding and fusing the downsampled sub-feature map and the downsampled feature map, the embodiment of the present invention first inputs these two parts into the significant position selection module for feature selection to generate an initial feature map, which can adaptively select the significant positions in the nameplate image to be detected, thereby more accurately locating and extracting the nameplate text.
[0068] S104: Input the downsampled feature map and the initial feature map into a preset CAFM feature fusion module for feature fusion to obtain a composite feature map;
[0069] As an optional implementation, the downsampled feature map and the initial feature map are input into a preset CAFM feature fusion module for feature fusion to obtain a composite feature map. This step can be specifically divided into the following steps S1041:
[0070] S1041. Upsample the first initial feature map, the second initial feature map, the third initial feature map, and the fourth downsampled feature map, and then input the upsampled first initial feature map, the second initial feature map, the third initial feature map, and the fourth downsampled feature map into a CAFM feature fusion module for feature fusion to obtain a composite feature map.
[0071] Furthermore, after upsampling the first initial feature map, the second initial feature map, the third initial feature map and the fourth downsampled feature map to obtain feature maps of the same size, the feature maps are input into the CAFM feature fusion module for feature fusion to obtain a composite feature map.
[0072] The CAFM feature fusion module consists of a channel-wise attention module and a spatial attention fusion module. The CAFM feature fusion module first fuses the first, second, third, and fourth downsampled input feature maps through convolution and pooling operations. This combines feature information at different levels, thereby improving text detection accuracy. Next, a cross-attention mechanism is used to focus the model on key text-related regions in the nameplate image to be detected. By calculating the correlations between the input feature maps, the model learns which regions are more important for text detection. Convolution is then used to extract spatial information from the image, and average and max pooling are used to extract global and local features in the nameplate image to better understand the image content. The weights derived from the attention mechanism are multiplied with the original input feature map to enhance text-related features while suppressing irrelevant ones. Finally, the module fuses the processed feature maps to produce the final composite feature map.
[0073] It should be noted that the embodiment of the present invention introduces a CAFM feature fusion module to perform feature fusion to generate a composite feature map, which can comprehensively utilize multiple features to improve the detection and recognition capabilities of the nameplate text area.
[0074] S105: Input the composite feature map into a preset prediction head module to obtain a probability map and a threshold map for predicting text;
[0075] Specifically, the generated composite feature map is passed through the two prediction heads in the prediction head module to generate a text probability map and a threshold map. The probability map indicates the confidence that each pixel belongs to a text region, while the threshold map indicates which locations are considered text regions and which are non-text regions under the predicted threshold. The global threshold is dynamically adjusted based on the characteristics of text and non-text regions in the current threshold map.
[0076] S106 , determining an adaptive threshold corresponding to each first pixel point in the threshold map according to the probability map, and then performing binarization processing on the probability map according to the adaptive threshold value to obtain a nameplate text detection result.
[0077] Specifically, in the prior art, a binarization operation is directly performed after the probability map and threshold map are obtained through feature map prediction. In this process, the obtained threshold may not accurately segment the text area and the non-text area. Therefore, after obtaining the probability map and threshold map, the embodiment of the present invention further obtains a dynamic adaptive threshold and further processes the threshold based on these two maps.
[0078] As an optional implementation, the step of determining the adaptive threshold corresponding to each first pixel in the threshold map according to the probability map can be specifically divided into the following steps S1061 and S1064:
[0079] S1061. Obtain a momentum attenuation coefficient, and obtain a predicted probability of the region corresponding to the nameplate image to be detected based on the probability map;
[0080] S1062. Calculate the weighted average of the regional prediction probabilities, and then obtain a global threshold based on the momentum attenuation coefficient and the weighted average.
[0081] Furthermore, in text detection, regions with high confidence are more likely to be real text, so the global threshold is set according to the system's average confidence in unlabeled data. The global threshold is set to the system's average confidence in unlabeled data. However, due to the large amount of unlabeled data, it will be very time-consuming to calculate the confidence of all unlabeled data at each time step or even each training period. Therefore, the embodiment of the present invention estimates the global confidence as the exponential moving average (EMA) of the confidence (i.e., regional prediction probability) for each training time step. The global threshold τ is initialized to Where C represents the number of categories, and the global threshold τ is determined by the following formula:
[0082]
[0083] Among them, τ t represents the global threshold corresponding to the t-th time step, τ t-1 represents the global threshold corresponding to the t-1th time step, λ represents the momentum decay coefficient of the exponential moving average and λ∈(0,1), q b It represents the probability that the system predicts the correct text region and non-text region in the bth batch (i.e., the region prediction probability), B represents the batch size (i.e., the number of data processed simultaneously by the system in one iteration), μ represents the ratio of the unlabeled data batch size to the labeled data batch size, b is a positive integer and b∈(1,B).
[0084] The design of a global threshold that steadily increases during training helps ensure that noise and false labels are discarded in the later stages of training. As training progresses, the model's confidence in the nameplate images to be detected increases. Therefore, by gradually increasing the global threshold, low-confidence detection results can be effectively filtered out, improving the accuracy and reliability of detection.
[0085] S1063, calculating the predicted expected values of the text region category and the non-text region category to obtain a local threshold;
[0086] Furthermore, the predicted expected value of the model for each category c (i.e., text region category and non-text region category) is calculated to estimate the learning state of a specific category and obtain the local threshold. Determined by the following formula:
[0087]
[0088] in, represents the local threshold corresponding to the cth category in the tth time step, represents the local threshold corresponding to the cth category in the t-1th time step, and is a list containing all categories c, i.e. q b (c) represents the predicted probability of the cth category in the bth batch.
[0089] S1064: Perform maximum normalization processing on the global threshold and the local threshold to obtain an adaptive threshold.
[0090] As a further optional implementation, the adaptive threshold is determined by the following formula:
[0091]
[0092] Among them, τ t (c) represents the adaptive threshold corresponding to the c-th category in the t-th time step, τ t represents the global threshold corresponding to the t-th time step, represents the local threshold corresponding to the cth category in the tth time step, MaxNorm(·) represents the maximum normalization operation, C represents the total number of categories, and t and c are positive integers.
[0093] As an optional implementation, the step of performing binarization processing on the probability map according to the adaptive threshold to obtain the nameplate text detection result can be specifically divided into the following steps S1065:
[0094] S1065 , using the adaptive threshold as a threshold for binarization processing, performing binarization processing on the probability map according to the threshold, and obtaining a nameplate text detection result.
[0095] Specifically, the adaptive threshold obtained by the adaptive threshold method is used as the binarization threshold, and the system performs a differentiable binarization process to convert the probability map into a binary map, that is, the text area and non-text area are binarized into 1 and 0 respectively, to achieve accurate segmentation of the text area in the nameplate image to be detected.
[0096] Furthermore, the binarization process is performed by a step function approximated by the following formula:
[0097]
[0098] in, represents an approximate binary mapping, T i,j represents the adaptive threshold corresponding to the (i, j)th pixel in the threshold map, P i,jrepresents the predicted probability corresponding to the (i, j)th pixel in the probability map, and k represents the magnification factor, which is empirically set to 50. This approximate binarization function is differentiable and can therefore be optimized together with the segmentation network during training.
[0099] The above describes the nameplate text detection method based on dynamic adaptive threshold according to an embodiment of the present invention. It can be appreciated that, compared to the prior art, the embodiment of the present invention, on the one hand, introduces a significant position selection module to perform feature selection to generate an initial feature map before adding and fusing the downsampled sub-feature map and the downsampled feature map, and can adaptively select significant positions in the nameplate image to be detected, thereby more accurately locating and extracting the nameplate text; on the other hand, by introducing a CAFM feature fusion module to perform feature fusion to generate a composite feature map, it can comprehensively utilize multiple features to improve the detection and recognition capabilities of the nameplate text area; in addition, before binarizing the probability map, by dynamically determining the adaptive threshold in the threshold map, it can better cope with changes in nameplate text under different styles, sizes, and lighting conditions, thereby improving the robustness and generalization capabilities of nameplate text detection.
[0100] Reference Figure 2 , an embodiment of the present invention further provides a nameplate text detection system based on a dynamic adaptive threshold, comprising:
[0101] An image acquisition module, used to acquire an image of the nameplate to be detected;
[0102] An image sampling module is used to downsample the nameplate image to be inspected according to different downsampling multiples to obtain multiple downsampled feature maps, and then upsample each downsampled feature map according to a preset upsampling multiple to obtain multiple downsampled sub-feature maps;
[0103] A feature selection module is used to input the downsampled feature map and the downsampled sub-feature map into a preset salient position selection module for feature selection to obtain multiple initial feature maps;
[0104] The feature fusion module is used to input the downsampled feature map and the initial feature map into the preset CAFM feature fusion module for feature fusion to obtain a composite feature map;
[0105] A text prediction module is used to input the composite feature map into a preset prediction head module to obtain a probability map and a threshold map for predicting text;
[0106] The binarization processing module is used to determine the adaptive threshold corresponding to each pixel point in the threshold map according to the probability map, and then perform binarization processing on the probability map according to the adaptive threshold to obtain the nameplate text detection result.
[0107] The contents of the above-mentioned nameplate text detection method embodiment based on dynamic adaptive threshold are all applicable to the nameplate text detection system embodiment based on dynamic adaptive threshold. The functions specifically implemented by the nameplate text detection system embodiment based on dynamic adaptive threshold are the same as those of the above-mentioned nameplate text detection method embodiment based on dynamic adaptive threshold, and the beneficial effects achieved are also the same as those achieved by the above-mentioned nameplate text detection method embodiment based on dynamic adaptive threshold.
[0108] An embodiment of the present invention further provides an electronic device comprising: a memory, a processor, a program stored in the memory and executable on the processor, and a data bus for enabling communication between the processor and the memory. When the program is executed by the processor, the aforementioned method for nameplate text detection based on a dynamic adaptive threshold is implemented. The electronic device can be any intelligent terminal, including a tablet computer and an in-vehicle computer.
[0109] like Figure 3 FIG2 is a schematic diagram of the hardware structure of the electronic device provided by the embodiment of the present invention, referring to FIG2 Figure 3 , an embodiment of the present invention provides an electronic device, including:
[0110] The processor 1001 may be implemented as a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is configured to execute relevant programs to implement the technical solutions provided by the embodiments of the present invention.
[0111] The memory 1002 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 1002 can store an operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 1002 and is called by the processor 1001 to execute the nameplate text detection method based on dynamic adaptive threshold according to the embodiment of the present invention.
[0112] Input / output interface 1003, used to implement information input and output;
[0113] Communication interface 1004, used to implement communication interaction between this device and other devices, which can be achieved through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WiFi, Bluetooth, etc.);
[0114] Bus 1005 , which transmits information between various components of the device (e.g., processor 1001 , memory 1002 , input / output interface 1003 , and communication interface 1004 );
[0115] The processor 1001 , the memory 1002 , the input / output interface 1003 and the communication interface 1004 are connected to each other in communication within the device via the bus 1005 .
[0116] An embodiment of the present invention also provides a storage medium, which is a computer-readable storage medium used for computer-readable storage. The storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the above-mentioned nameplate text detection method based on dynamic adaptive threshold.
[0117] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely arranged relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.
[0118] The embodiment of the present invention also discloses a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device can read the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs Figure 1 The method shown.
[0119] In some optional embodiments, the function / operation mentioned in the block diagram may not occur in the order mentioned in the operation diagram. For example, depending on the function / operation involved, the two boxes shown in succession can actually be executed substantially simultaneously or the above-mentioned boxes can sometimes be executed in reverse order. In addition, the embodiment presented and described in the flow chart of the present invention is provided in an exemplary manner for the purpose of providing a more comprehensive understanding of the technology. The disclosed method is not limited to the operation and logic flow presented herein. Optional embodiments are contemplated in which the order of the various operations is changed and the sub-operations described as a part of a larger operation are performed independently.
[0120] In addition, although the present invention is described in the context of functional modules, it should be understood that, unless otherwise stated, one or more of the above-mentioned functions and / or features can be integrated into a single physical device and / or software module, or one or more functions and / or features can be implemented in separate physical devices or software modules. It is also understood that a detailed discussion of the actual implementation of each module is not necessary for understanding the present invention. More specifically, given the properties, functions, and internal relationships of the various functional modules in the devices disclosed herein, the actual implementation of the module will be understood within the routine skills of an engineer. Therefore, a person skilled in the art can implement the present invention set forth in the claims using ordinary skills without undue experimentation. It is also understood that the specific concepts disclosed are merely illustrative and are not intended to limit the scope of the present invention, which is determined by the full scope of the appended claims and their equivalents.
[0121] If the above functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the above methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, and other media that can store program code.
[0122] The logic and / or steps represented in the flowcharts or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing the logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (e.g., a computer-based system, a system including a processor, or other system that can fetch and execute instructions from an instruction execution system, apparatus, or device). For purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport a program for use by, or in conjunction with, an instruction execution system, apparatus, or device.
[0123] It should be understood that various parts of the present invention can be implemented using hardware, software, firmware, or a combination thereof. In the above-described embodiments, multiple steps or methods can be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented using hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof can be used: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application-specific integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.
[0124] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to the embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the claims and their equivalents.
[0125] The above is a specific description of the preferred implementation of the present invention, but the present invention is not limited to the embodiments. Those skilled in the art can make various equivalent modifications or substitutions without violating the spirit of the present invention. These equivalent modifications or substitutions are all included in the scope defined by the claims of this application.
Claims
1. A nameplate text detection method based on dynamic adaptive threshold, characterized in that: The following steps are involved: Obtain the image of the nameplate to be detected; Downsampling the nameplate image to be detected according to different downsampling multiples to obtain a plurality of downsampling feature maps, and then upsampling each of the downsampling feature maps according to a preset upsampling multiple to obtain a plurality of downsampling sub-feature maps; Inputting the downsampled feature map and the downsampled sub-feature map into a preset salient position selection module for feature selection to obtain a plurality of initial feature maps; Inputting the downsampled feature map and the initial feature map into a preset CAFM feature fusion module for feature fusion to obtain a composite feature map; Inputting the composite feature map into a preset prediction head module to obtain a probability map and a threshold map for predicting text; The adaptive threshold corresponding to each first pixel in the threshold map is determined according to the probability map, and then the probability map is binarized according to the adaptive threshold to obtain a nameplate text detection result.
2. The nameplate text detection method based on dynamic adaptive threshold according to claim 1, characterized in that: The method of downsampling the nameplate image to be detected according to different downsampling multiples to obtain a plurality of downsampling feature maps, and then upsampling each of the downsampling feature maps according to a preset upsampling multiple to obtain a plurality of downsampling sub-feature maps, specifically includes: Downsampling the nameplate image to be detected according to different downsampling multiples to obtain a first downsampling feature map, a second downsampling feature map, a third downsampling feature map, and a fourth downsampling feature map; Upsampling the second downsampled feature map according to the upsampling multiple to obtain a second downsampled sub-feature map; Upsampling the third downsampled feature map according to the upsampling multiple to obtain a third downsampled sub-feature map; The fourth down-sampled feature map is up-sampled according to the up-sampling multiple to obtain a fourth down-sampled sub-feature map.
3. The nameplate text detection method based on dynamic adaptive threshold according to claim 2, characterized in that: The step of inputting the downsampled feature map and the downsampled sub-feature map into a preset salient position selection module for feature selection to obtain a plurality of initial feature maps specifically includes: Inputting the first down-sampled feature map and the second down-sampled sub-feature map into the salient position selection module for feature selection, and then performing element-wise addition of the first down-sampled feature map and the second down-sampled sub-feature map after feature selection to obtain a first initial feature map; Inputting the second down-sampled feature map and the third down-sampled sub-feature map into the salient position selection module for feature selection, and then performing element-wise addition of the second down-sampled feature map and the third down-sampled sub-feature map after feature selection to obtain a second initial feature map; The third down-sampled feature map and the fourth down-sampled sub-feature map are input into the salient position selection module for feature selection, and then the third down-sampled feature map and the fourth down-sampled sub-feature map after feature selection are element-wise added to obtain a third initial feature map.
4. The nameplate text detection method based on dynamic adaptive threshold according to claim 3 is characterized in that: The down-sampled feature map and the initial feature map are input into a preset CAFM feature fusion module for feature fusion to obtain a composite feature map, specifically: The first initial feature map, the second initial feature map, the third initial feature map and the fourth down-sampled feature map are up-sampled, and then the up-sampled first initial feature map, the second initial feature map, the third initial feature map and the fourth down-sampled feature map are input into the CAFM feature fusion module for feature fusion to obtain the composite feature map.
5. The nameplate text detection method based on dynamic adaptive threshold according to claim 1, characterized in that: Determining the adaptive threshold corresponding to each first pixel point in the threshold map according to the probability map specifically includes: Obtaining a momentum attenuation coefficient, and obtaining a predicted probability of a region corresponding to the nameplate image to be detected according to the probability map; Calculating a weighted average of the regional prediction probabilities, and then obtaining a global threshold according to the momentum decay coefficient and the weighted average; Calculate the predicted expected values of the text area category and the non-text area category to obtain the local threshold; The global threshold and the local threshold are subjected to maximum normalization processing to obtain the adaptive threshold.
6. The nameplate text detection method based on dynamic adaptive threshold according to claim 5, characterized in that: The adaptive threshold is determined by the following formula: Among them, τ t (c) represents the adaptive threshold corresponding to the c-th category in the t-th time step, τ t represents the global threshold corresponding to the t-th time step, represents the local threshold corresponding to the cth category in the tth time step, MaxNorm(·) represents the maximum normalization operation, C represents the total number of categories, and t and c are positive integers.
7. A nameplate text detection method based on dynamic adaptive threshold according to any one of claims 1 to 6, characterized in that: The probability map is binarized according to the adaptive threshold to obtain the nameplate text detection result, specifically: The adaptive threshold is used as a threshold for binarization processing, and the probability map is binarized according to the threshold to obtain the nameplate text detection result.
8. A nameplate text detection system based on dynamic adaptive threshold, characterized in that: include: An image acquisition module, used to acquire the image of the nameplate to be detected; An image sampling module, used to downsample the nameplate image to be detected according to different downsampling multiples to obtain a plurality of downsampled feature maps, and then upsample each of the downsampled feature maps according to a preset upsampling multiple to obtain a plurality of downsampled sub-feature maps; A feature selection module, used for inputting the downsampled feature map and the downsampled sub-feature map into a preset salient position selection module for feature selection to obtain a plurality of initial feature maps; A feature fusion module, used for inputting the downsampled feature map and the initial feature map into a preset CAFM feature fusion module for feature fusion to obtain a composite feature map; A text prediction module, used for inputting the composite feature map into a preset prediction head module to obtain a probability map and a threshold map for predicting text; The binarization processing module is used to determine the adaptive threshold corresponding to each pixel point in the threshold map according to the probability map, and then perform binarization processing on the probability map according to the adaptive threshold to obtain the nameplate text detection result.
9. An electronic device, characterized in that: The electronic device includes a memory, a processor, a program stored in the memory and executable on the processor, and a data bus for realizing connection and communication between the processor and the memory. When the program is executed by the processor, the steps of the nameplate text detection method based on dynamic adaptive threshold as described in any one of claims 1 to 7 are realized.
10. A storage medium, the storage medium being a computer-readable storage medium, used for computer-readable storage, characterized in that: The storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps of the nameplate text detection method based on dynamic adaptive threshold as described in any one of claims 1 to 7.