A method and device for identifying the content of a nixie tube, electronic equipment and storage medium

Abstract

The present application relates to the technical field of digital recognition, and particularly provides a nixie tube content recognition method and device, electronic equipment and storage medium, wherein the method comprises: acquiring a continuous video image sequence of a nixie tube, acquiring position information of a nixie tube character display segment according to a preset frame number by using a template matching algorithm; recognizing bright and dark states of the nixie tube character display segment by using a support vector machine (SVM); acquiring a brightness sequence unit of the nixie tube character display segment under the preset frame number according to a preset nixie tube arrangement mode; classifying the brightness sequence unit by using a deep learning method and outputting a recognition result; and the device comprises: a nixie tube and a PC terminal connected with the nixie tube, the nixie tube and the PC terminal are configured in an asynchronous state, and the PC terminal comprises: an acquisition module, a position recognition unit, a brightness recognition unit and a training module. The purpose is to solve the misrecognition problem caused by uneven brightness and defects in the nixie tube content recognition process.

Description

Technical Field

[0001] This invention relates to the field of digital recognition technology, and more specifically, to a method, apparatus, electronic device, and storage medium for recognizing the content of a digital tube. Background Technology

[0002] Digital tube displays are a type of LED display screen. Due to their low price and ease of use, they are widely used in substations for instruments such as voltmeters, ammeters, and thermometers, displaying numerical information. However, existing digital tube LED displays suffer from asynchronous display units due to their driving mechanism. For example, substation auxiliary monitoring systems require 24 / 7 inspection of equipment and instruments to accurately perceive their status and parameters, reduce manual inspection workload, and ultimately achieve unmanned inspection. Current auxiliary monitoring systems typically use commercial cameras with limited control parameters and no physical link between them and the LED display screen, or the cameras and LED display screens are independent units. This asynchronous operation often results in incomplete or fragmented images captured by the cameras, or differences in brightness, making the images unrecognizable and significantly reducing the accuracy of LED display content recognition.

[0003] The invention patent CN108182400A, "Digital Tube Dynamic Display Recognition Method and System," obtains the dynamic display cycle of the digital tube. This method sets the dynamic scanning cycle of the digital tube to the camera exposure time via a PC to acquire the complete content of the digital tube and thus achieve recognition. However, this method has high requirements for the camera and system, requiring the camera to be controllable and the LED display screen to be synchronized with the system. Although the solution can solve the problem of incomplete image acquisition, there are certain difficulties in system deployment.

[0004] To this end, we propose a method, device, electronic device, and storage medium for recognizing digital tube content. Summary of the Invention

[0005] The purpose of this invention is to provide a method, device, electronic device and storage medium for recognizing digital tube content, so as to solve the problem of misrecognition caused by uneven brightness and defects in the digital tube content recognition process.

[0006] The first aspect of the present invention provides a method for recognizing the content of a digital tube, comprising:

[0007] Obtain a continuous video image sequence of the digital tube, and use a template matching algorithm to obtain the position information of the character display segment of the digital tube according to a preset frame number;

[0008] The brightness and darkness status of the character display segments on a digital tube are identified using a support vector machine (SVM).

[0009] Obtain the brightness sequence units of the digital tube character display segment at a preset frame number according to the preset digital tube arrangement;

[0010] The brightness sequence units are classified using deep learning methods, and the recognition results are output.

[0011] Furthermore, the template matching algorithm specifically includes: using corner matching methods to match and identify the positional information of the displayed segment, specifically:

[0012] The original image information of the digital tube and the target image information in the video image sequence are obtained, and key feature points are extracted using a feature point extraction algorithm.

[0013] The key feature points are located, feature vectors are extracted and compared, and matching key feature points are obtained.

[0014] The RANSAC algorithm is used to eliminate noise for mutually matching key feature points in order to obtain the specific location information of the display segment.

[0015] Furthermore, key feature points include at least corner points, edge points, two points in dark areas, and dark points in bright areas.

[0016] Furthermore, the preset digital tube arrangement specifically includes: using the order in which the digital tube refresh display segments are arranged as the brightness sequence unit arrangement rule.

[0017] Furthermore, the specific steps of using deep learning methods to separately process the luminance sequence units and output the recognition results include:

[0018] Manually annotate the brightness sequence units and obtain the annotated dataset;

[0019] A classification model is obtained by training on a labeled dataset;

[0020] A classification model is used to identify brightness sequence units and output the identification results.

[0021] Furthermore, the classification model obtained after training on the labeled dataset specifically includes:

[0022] A classification model is obtained by training a labeled dataset using a ResNet-50 convolutional neural network model.

[0023] The second aspect of the present invention provides a digital tube content recognition device, including a digital tube and a PC terminal connected to the digital tube by signals, wherein the digital tube and the PC terminal are configured to be in a asynchronous state, and the PC terminal includes:

[0024] The acquisition module is configured to acquire a continuous video image sequence of the digital tube.

[0025] The position recognition unit is configured to use a template matching algorithm to obtain the position information of the digital tube character display segment according to a preset number of frames;

[0026] The brightness recognition unit is configured to obtain the brightness sequence of the digital tube character display segment at a preset frame number according to the preset digital tube arrangement.

[0027] The training module is configured to use deep learning methods to classify luminance sequence units and output recognition results.

[0028] The third aspect of the present invention provides an electronic device, the electronic device comprising: a processor and a memory communicatively connected to the processor; wherein the memory stores instructions executable by the processor, the instructions being executed by the processor to enable the processor to perform the steps of the digital tube content recognition method as described in any one of the first aspects of the present invention.

[0029] The fourth aspect of the present invention provides a computer-readable storage medium storing a program for implementing a digital tube content recognition method, wherein the program for implementing the digital tube content recognition method is executed by a processor to implement the steps of the content recognition method described in any one of the first aspects of the present invention.

[0030] The beneficial effects of this invention include:

[0031] 1. The digital tube content recognition method provided by this invention uses Support Vector Machine (SVM) to recognize the brightness and darkness of the digital tube character display segments across multiple frames, solving the problem of misrecognition caused by uneven brightness and incompleteness of the digital tube. By acquiring brightness sequence units, using deep learning methods to classify the brightness sequence units and output recognition results, it realizes content recognition of the digital tube and the image acquisition device in a asynchronous state, reducing the performance requirements of the image acquisition device, simplifying deployment, and effectively improving the accuracy of digital tube content recognition. Attached Figure Description

[0032] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments of the present invention will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0033] Figure 1 A flowchart of a digital tube content recognition method provided in an embodiment of the present invention;

[0034] Figure 2 A flowchart of the template matching algorithm provided in an embodiment of the present invention;

[0035] Figure 3 This is a schematic diagram of the structure of the digital tube content recognition device provided in an embodiment of the present invention;

[0036] Figure 4 This is a schematic diagram of the bottleneck residual block provided in an embodiment of the present invention;

[0037] Figure 5 A schematic diagram of the network structure of the ResNet-50 convolutional neural network model provided in an embodiment of the present invention. Detailed Implementation

[0038] The technical solutions of the present invention will now be described with reference to the accompanying drawings in the embodiments of the present invention.

[0039] It should be noted that similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. Furthermore, in the description of this invention, terms such as "first," "second," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0040] Please see Figure 1 As shown, the first aspect of the present invention provides a method for recognizing the content of a digital tube, comprising:

[0041] Step S1: Obtain a continuous video image sequence of the digital tube, and use a template matching algorithm to obtain the position information of the character display segment of the digital tube according to the preset frame number;

[0042] In step S1, to achieve stable recognition of the digital tube, under the premise that the parameters of the image acquisition device are unknown or uncontrollable, continuous video images of the digital tube within 1 second are acquired and the video is converted into a video image sequence; the preset number of frames can be a single frame or multiple frames.

[0043] Preferably, the length, width, and height information of the digital tube can be obtained, the rect of the digital tube can be marked manually, and then the position information of the corresponding display segment of the digital tube in the video image sequence can be matched by the mapping method.

[0044] Step S2: Use Support Vector Machine (SVM) to identify the brightness state of the character display segments on the digital tube;

[0045] Among them, the support vector machine (SVM) recognition is used to avoid misrecognition caused by uneven brightness of the digital tube character display segments by recognizing multiple frames of video image sequences. Specifically, the ResNet-50 neural network algorithm can be used to mark the brightness state of each character display segment and represent it with an output of 0 or 1, where 0 represents dark and 1 represents bright.

[0046] Step S3: Obtain the brightness sequence unit of the digital tube character display segment at a preset frame number according to the preset digital tube arrangement;

[0047] In step S3, based on the brightness and darkness status of the character display segments of the digital tube in step S2, a brightness sequence unit is formed according to the preset digital tube arrangement. For example, "01000001" is a complete brightness sequence unit. The first seven digits of the brightness sequence unit represent the brightness and darkness status of the seven character display segments of the digital tube. The eighth digit of the brightness sequence unit represents the brightness and darkness status of the decimal point of the digital tube.

[0048] Step S4: Use deep learning methods to classify the brightness sequence units and output the recognition results;

[0049] The output recognition result is the actual number displayed on a certain digital tube. Step S4 requires labeling the brightness sequence unit based on the semantic information in the continuous video image sequence, such as "1." The semantic information includes the scene to be recognized, numbers, letters, etc.

[0050] The digital tube content recognition method provided in this embodiment solves the misrecognition problem caused by uneven brightness and incompleteness of the digital tube by using support vector machine (SVM) to recognize the brightness and darkness of the character display segments of the digital tube through multi-frame recognition. By acquiring brightness sequence units, deep learning methods are used to classify the brightness sequence units and output the recognition results, realizing content recognition of the digital tube and the image acquisition device in a asynchronous state. In practical applications, it reduces the requirements for image acquisition devices such as cameras. Overall, it improves the content recognition of digital tubes from the algorithm level, effectively improving the accuracy of digital tube content recognition through multi-frame recognition and deep learning methods, while also facilitating the deployment of the recognition system.

[0051] Please see Figure 2 As shown, preferably, the template matching algorithm specifically includes: using a corner matching method to match and identify the position information of the display segment, specifically:

[0052] The original image information of the digital tube and the target image information in the video image sequence are obtained, and key feature points are extracted using feature point extraction algorithms such as SIFT and HariiH corner point extraction.

[0053] Specifically, the SIFT algorithm is used to identify image locations in all scale spaces of the original image information through the Gaussian differential function, thereby identifying potential interest points that are scale- and rotation-invariant.

[0054] The key feature points are located, feature vectors are extracted and compared, and matching key feature points are obtained.

[0055] Among them, the location of key feature points is determined by fitting a model to determine the location of each preset key feature point.

[0056] The RANSAC algorithm is used to eliminate noise for mutually matching key feature points in order to obtain the specific location information of the display segment.

[0057] In this embodiment, since the relative positions of the acquisition device and the digital tube are not fixed in actual work, the images acquired from the digital tube have size differences. Therefore, a template matching algorithm with scale invariance is selected.

[0058] Preferably, the key feature points include at least corner points, edge points, two points in dark areas, and dark points in bright areas; in this embodiment, the key feature points are points that will not disappear due to factors such as lighting, size, or rotation.

[0059] Preferably, the preset digital tube arrangement method specifically includes: using the order of the digital tube refresh display segments as the arrangement rule for brightness sequence units.

[0060] Preferably, the method of using deep learning to separately process the luminance sequence units and output the recognition results specifically includes:

[0061] Manually annotate the brightness sequence units and obtain the annotated dataset;

[0062] A classification model is obtained by training on a labeled dataset;

[0063] A classification model is used to identify brightness sequence units and output the identification results.

[0064] Furthermore, the classification model obtained after training on the labeled dataset specifically includes:

[0065] A classification model is obtained by training a labeled dataset using a ResNet-50 convolutional neural network model.

[0066] The ResNet-50 convolutional neural network model contains 49 convolutional layers and 1 fully connected layer; for example... Figure 4 As shown, Figure 4 The bottleneck residual block is a fundamental module component of ResNet-50. Through dimensionality increase and decrease operations, the number of parameters in the model can be greatly reduced; for example... Figure 5 As shown, Figure 5 The diagram shows the network structure of ResNet-50. The network input is 224×224×3. After the first five convolutional calculations, the output is 7×7×2048. The pooling layer will transform it into a feature vector. Finally, the classifier will calculate the class probability from this feature vector.

[0067] Please see Figure 3 As shown, the second aspect of the present invention provides a digital tube content recognition device, including a digital tube and a PC terminal connected to the digital tube via signal, wherein the digital tube and the PC terminal are configured to be in a asynchronous state, and the PC terminal includes:

[0068] The acquisition module is configured to acquire a continuous video image sequence of the digital tube.

[0069] The position recognition unit is configured to use a template matching algorithm to obtain the position information of the digital tube character display segment according to a preset number of frames;

[0070] The brightness recognition unit is configured to obtain the brightness sequence of the digital tube character display segment at a preset frame number according to the preset digital tube arrangement.

[0071] The training module is configured to use deep learning methods to classify luminance sequence units and output recognition results.

[0072] The third aspect of the present invention provides an electronic device, the electronic device comprising: a processor and a memory communicatively connected to the processor; wherein the memory stores instructions executable by the processor, the instructions being executed by the processor to enable the processor to perform the steps of the digital tube content recognition method as described in any one of the first aspects of the present invention.

[0073] The fourth aspect of the present invention provides a computer-readable storage medium storing a program for implementing a digital tube content recognition method, wherein the program for implementing the digital tube content recognition method is executed by a processor to implement the steps of the content recognition method described in any one of the first aspects of the present invention.

[0074] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for recognizing the content of a digital tube display, characterized in that, include: Obtain a continuous video image sequence of the digital tube, and use a template matching algorithm to obtain the position information of the character display segment of the digital tube according to a preset frame number. This includes: using a corner matching method to match and identify the position information of the display segment, specifically: The original image information of the digital tube and the target image information in the video image sequence are obtained. Key feature points are extracted using a feature point extraction algorithm. Key feature points include at least corner points, edge points, bright spots in dark areas and dark spots in bright areas. The key feature points are located, feature vectors are extracted and compared, and matching key feature points are obtained. The RANSAC algorithm is used to eliminate noise for mutually matching key feature points in order to obtain the specific location information of the display segment. The bright and dark states of the character display segments of a digital tube are identified using Support Vector Machine (SVM), which involves multi-frame recognition of video image sequences using SVM. The brightness sequence units of the digital tube character display segments at a preset frame number are obtained according to a preset digital tube arrangement; the preset digital tube arrangement specifically includes: using the order in which the digital tube refreshes the display segments as the arrangement rule for the brightness sequence units; The luminance sequence units are classified using deep learning methods, and the recognition results are output. Specifically, this includes: Manually annotate the brightness sequence units and obtain the annotated dataset; A classification model is obtained by training on a labeled dataset; A classification model is used to identify brightness sequence units and output the identification results.

2. The identification method according to claim 1, characterized in that, The classification model obtained after training on the labeled dataset specifically includes: A classification model is obtained by training a labeled dataset using a ResNet-50 convolutional neural network model.

3. A digital tube content recognition device, characterized in that, The digital tube content recognition method according to any one of claims 1 to 2 includes a digital tube and a PC terminal connected to the digital tube by a signal, wherein the digital tube and the PC terminal are configured to be in a asynchronous state, and the PC terminal includes: The acquisition module is configured to acquire a continuous video image sequence of the digital tube. The position recognition unit is configured to use a template matching algorithm to obtain the position information of the digital tube character display segment according to a preset number of frames; The brightness recognition unit is configured to obtain the brightness sequence of the digital tube character display segment at a preset frame number according to the preset digital tube arrangement. The training module is configured to use deep learning methods to classify luminance sequence units and output recognition results.

4. An electronic device, characterized in that, The electronic device includes: a processor and a memory communicatively connected to the processor; wherein the memory stores instructions executable by the processor, the instructions being executed by the processor to enable the processor to perform the steps of the digital tube content recognition method according to any one of claims 1 to 2.

5. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a program for implementing the digital tube content recognition method, which is executed by a processor to implement the steps of the content recognition method according to any one of claims 1 to 2.

Images