Deep learning-based digital display signal recognition method, system, device and medium

Abstract

The present disclosure relates to a deep learning-based digital display signal identification method, system, device and medium, which comprises the following steps: acquiring an image containing multiple non-dispersive infrared methane sensor digital display signals; using the flood fill method to segment the image and mark the marked area containing the digital display signal; establishing the color distribution range of the digital display signal in the marked area based on the H channel of the HSV model, and obtaining a numerical area image; inputting the numerical area image into a prediction model for matching to obtain a numerical value, thereby realizing accurate identification of multiple non-dispersive infrared methane sensor digital display signals, ensuring control over sensor value changes without human intervention, and achieving full-process monitoring. It is highly adaptable and stable.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the field of image signal recognition, in particular to a deep learning-based digital display signal recognition method, system, device and medium. BACKGROUND

[0002] In the coal mine environment, gas explosion has always been the biggest threat to the safety of workers underground. A large part of the gas explosion is caused by the disaster of high gas concentration in the mine, so controlling the gas concentration is an effective measure to avoid gas explosion accidents. In order to accurately detect the methane concentration underground, the sensor for detecting methane gas concentration not only needs to have high enough accuracy, but also needs to be calibrated by instrument equipment to ensure the reliability of the measurement results. Instrument equipment calibration is the main means to ensure the accuracy of measurement results, so the calibration of these sensors has become a basic and essential work.

[0003] In the traditional image recognition of methane sensor numerical signal, only single methane sensor numerical signal can be recognized, and the unknown number and position of methane sensor numerical signal cannot be effectively distinguished. In the traditional methane sensor numerical extraction, based on RGB color space, the image is binarized by threshold value, which has the problems of large noise, low robustness, inaccurate recognition, easy to be disturbed and the like. In the traditional methane sensor numerical matching, the digital tube is first grayed and binarized, the number is changed to 255, and the background is changed to 0, then the threading method is used to thread the abcdefg seven regions in turn, and it is judged whether there is a value of 255. If yes, it indicates that the region is highlighted, and finally the highlight information of the seven regions is combined to comprehensively judge the numerical value. However, for some digital tube with low overall brightness, the digital information will be lost after gray scale.

[0004] At present, the display mode of digital display instrument mainly has digital tube and liquid crystal. The commonly used methane sensor is mainly a non-dispersive infrared methane sensor, which mainly adopts digital tube display. When the methane sensor automatic calibration system collects the numerical value image of the sensor, due to the light reflection of the material of the sensor itself, and the attachment on the display panel, the quality of the collected numerical value image is poor, which causes difficulty in character recognition and reduces the recognition accuracy. The image recognition algorithm based on deep learning can well solve the problem of sensitivity to image noise and poor robustness of traditional image processing method. SUMMARY

[0005] The present disclosure provides a deep learning-based digital display signal recognition method, system, device and medium, which can solve at least one problem mentioned in the background. To solve the above technical problems, the present disclosure provides the following technical solutions:

[0006] S1. Obtain an image containing digital display signals of multiple non-dispersive infrared methane sensors;

[0007] S2. Realize the segmentation of the image by using the flood fill method and mark the marked area containing the digital signal;

[0008] Realizing the segmentation of the image by using the flood fill method further includes the following steps:

[0009] Carrying out the binaryzation processing on the image to obtain a gray scale image and carrying out the Gaussian filtering on the image to obtain a blurred image;

[0010] Obtaining the change of adjacent pixel points of the gray scale image and the change of adjacent pixel points of the blurred image;

[0011] According to the change of adjacent pixel points of the gray scale image and the change of adjacent pixel points of the blurred image, the pixel values of the gray scale image and the blurred image are normalized, if the high frequency component in the blurred image has almost no change compared with the high frequency component of the gray scale image, it is determined as a completely blurred image and is not processed, if the high frequency component changes obviously, it is determined as a partially blurred image, if the high frequency component changes greatly, it is determined as a clear image. After comparison and analysis, normalization processing is carried out; the partially blurred image, the clear image and the completely blurred image are obtained;

[0012] The clear image will be directly processed in the next link of segmentation, the partially blurred image will be re-judged whether it is a clear image after deblurring processing, if yes, the next link of segmentation processing is carried out, and the completely blurred image is not processed.

[0013] The deblurring processing includes: carrying out pixel gray scale statistics on the partially blurred image to obtain the pixel probability distribution of the gray scale image; obtaining the cumulative distribution function of the image to obtain the deblurred image after transformation; for the partially blurred image, first carry out pixel gray scale statistics to calculate the pixel probability distribution of the original image, obtain the cumulative distribution function of the image from the pixel probability distribution, and obtain the deblurred image after transformation according to the mapping function, the mapping function is as follows:

[0014] k = 0, 1, 2, 3, ··· L-1

[0015] Wherein, n is the total sum of pixels in the image, is the number of pixels of the current gray scale, and L is the total number of possible gray scales in the image.

[0016] The next link of segmentation processing includes:

[0017] Marking the flood fill area by using a mask matrix;

[0018] A single-channel image with a size of 2 pixels larger than the width and height of the input image is taken as a mask matrix, and the flood fill region is marked by filling the pixel value of the pixel point in the mask matrix

[0019] Obtain upper and lower boundary values of the seed point region condition, wherein the upper and lower boundary values include an upper boundary value and a lower boundary value.

[0020] Determine the coordinate value of the seed point.

[0021] With the seed point as the center, when the difference between the pixel value of a pixel point in the neighborhood and the pixel value of the seed point is greater than the lower boundary value, the pixel point is added to the region where the seed point is located; when the difference between the pixel value of the seed point and the pixel value of a pixel point in the neighborhood is less than the upper boundary value, the pixel point is added to the region where the seed point is located.

[0022] If all pixel points are not added, mark the position; take the newly added pixel point as a new seed point, and return to the previous step: with the seed point as the center, when the difference between the pixel value of a pixel point in the neighborhood and the pixel value of the seed point is greater than the lower boundary value, the pixel point is added to the region where the seed point is located; when the difference between the pixel value of the seed point and the pixel value of a pixel point in the neighborhood is less than the upper boundary value, the pixel point is added to the region where the seed point is located; until all pixel points are added.

[0023] If all pixel points are added; separate and mark the output of each region.

[0024] S3. Establish the color distribution range of the digital signal in the marked region based on the H channel of the HSV model, and obtain a numerical region image.

[0025] In HSV, the V channel is most affected by light, and the H channel is basically not affected by shadows or high brightness. The H channel will be the main reliance in the color extraction of the system, and will further establish the color distribution range of the numerical signal, generate a mask, and finally accurately extract the numbers in the marked region.

[0026] S4. Input the numerical region image into a prediction model to obtain a value, wherein the prediction model is obtained by training a neural network based on a training data set, and the training data set includes non-dispersive infrared methane sensor digital signal images under different environments, positions, or illuminations.

[0027] Before training the neural network based on the training data set to obtain the prediction model, the following steps are further included: collecting non-dispersive infrared methane sensor digital signal images under different environments, positions, or illuminations as a test data set; inputting the test data set into the prediction model for testing; and stopping training the prediction model when the test result meets the requirements of the preset recognition accuracy.

[0028] The step of training the neural network by the training data set further comprises the following steps:

[0029] Collecting sample library of non-dispersive infrared methane sensor digital display signal images under different environments, positions or illuminations;

[0030] Constructing training data set and test data set of neural network according to the sample library;

[0031] Training the neural network according to the constructed training data set to obtain the prediction model of the digital display signal.

[0032] The step of inputting the numerical area image into the prediction model for matching to obtain the numerical value further comprises the following steps:

[0033] Obtaining the numerical area image and performing size scaling to obtain a numerical area vector;

[0034] Substituting the numerical area vector into the prediction model for matching to obtain the numerical value;

[0035] According to the pixel coordinates, the numbers are sorted from left to right to be connected as a string for output.

[0036] The sample library is established by collecting methane sensor digital display images under different environments, positions and illuminations, and is used to construct training data set and test data set of neural network. According to the constructed training data set, deep neural network is adopted, the number of hidden layers is increased to increase the connection weight, threshold and other parameters of neurons, which is used to increase the number of neurons of activation function and the number of nested layers. Through the method of pre-training and fine-tuning, the parameters are grouped, and the local optimal setting is found for each group. Based on the local optimal result, global optimization is carried out. The freedom provided by the large number of parameters of the model is utilized while the training cost is effectively saved. The loss is calculated by continuously iterating the training model, and the model parameters are updated. Finally, the methane sensor digital display image to be recognized is input into the model for matching to obtain the specific numerical value of the digital display image.

[0037] Through multiple collection of digital display images under the field environment, test data set and training data set are constructed, the network is trained by the constructed data set, the required model for recognition is completed, the numerical value region extracted above is scaled to the size of the model and substituted into the model for numerical value matching and output. The recognition of the digital display signals of multiple methane sensors is completed.

[0038] As another aspect of the embodiments of the present disclosure, a digital display signal recognition system based on deep learning is provided, comprising:

[0039] An image acquisition module acquires images containing digital display signals of multiple non-dispersive infrared methane sensors;

[0040] The image segmentation module realizes segmentation of the image and marks a marked area containing the digital signal by using a flood fill method.

[0041] The image segmentation module further comprises an image preprocessing module: performing binaryzation on the image to obtain a gray image, and performing Gaussian filtering on the image to obtain a blurred image.

[0042] The change of adjacent pixel points of the gray image and the change of adjacent pixel points of the blurred image are obtained.

[0043] According to the change of adjacent pixel points of the gray image and the change of adjacent pixel points of the blurred image, the pixel values of the gray image and the blurred image are normalized, if the high-frequency component in the blurred image has almost no change compared with the high-frequency component of the gray image, it is determined as a completely blurred image and is not processed, if the high-frequency component changes more obviously, it is determined as a partially blurred image, and if the high-frequency component changes greatly, it is determined as a clear image. After comparison and analysis, normalization processing is performed; a partially blurred image, a clear image and a completely blurred image are obtained.

[0044] The clear image will be directly subjected to the next link of segmentation processing, the partially blurred image will be subjected to deblurring processing and then re-determined whether the image is a clear image, if yes, the next link of segmentation processing is performed, and the completely blurred image is not processed.

[0045] The image segmentation module further comprises a deblurring processing module: performing pixel gray scale statistics on the partially blurred image to obtain the pixel probability distribution of the gray image; obtaining the cumulative distribution function of the image and the deblurred image after transformation; for the partially blurred image, first perform pixel gray scale statistics to calculate the pixel probability distribution of the original image, obtain the cumulative distribution function of the image from the pixel probability distribution, and obtain the deblurred image after transformation according to a mapping function, the mapping function is as follows:

[0046] k = 0, 1, 2, 3, ··· L-1

[0047] wherein n is the total sum of pixels in the image, is the number of pixels of the current gray level, and L is the total number of possible gray levels in the image.

[0048] The image segmentation module further comprises a next link of segmentation processing module, which is configured to:

[0049] The flood fill area is marked by using a mask matrix.

[0050] A single-channel image with a size of 2 pixels larger than the width and height of the input image is taken as a mask matrix, and a flood fill region is marked by filling the pixel values of the pixels in the mask matrix

[0051] Obtain upper and lower boundary values of the seed point region condition, wherein the upper and lower boundary values include an upper boundary value and a lower boundary value.

[0052] Determine the coordinate value of the seed point.

[0053] Take the seed point as the center, and when the difference between the pixel value of a pixel in the neighborhood and the pixel value of the seed point is greater than the lower boundary value, the pixel is added to the region where the seed point is located; when the difference between the pixel value of the seed point and the pixel value of a pixel in the neighborhood is less than the upper boundary value, the pixel is added to the region where the seed point is located.

[0054] If all the pixels are not added, mark the position; take the newly added pixel as a new seed point, and return to the previous step: take the seed point as the center, and when the difference between the pixel value of a pixel in the neighborhood and the pixel value of the seed point is greater than the lower boundary value, the pixel is added to the region where the seed point is located; when the difference between the pixel value of the seed point and the pixel value of a pixel in the neighborhood is less than the upper boundary value, the pixel is added to the region where the seed point is located; until all the pixels are added.

[0055] If all the pixels are added, separate and mark the output of each region.

[0056] The extraction module establishes the color distribution range of the digital signal in the marked region based on the H channel of the HSV model, and obtains a numerical region image.

[0057] In the HSV model, the V channel is most affected by light, and the H channel is basically not affected by shadows or high brightness. The H channel will be the main basis for color extraction in the system, and will establish the color distribution range of the numerical signal, generate a mask, and finally accurately extract the numbers in the marked region.

[0058] For the extraction of the numerical value in the to-be-recognized region, based on the HSV model, the distribution histogram of the H component is analyzed to find the distribution range of the numerical color. The extraction of the region numerical value is completed based on the H channel.

[0059] The matching module inputs the numerical region image into a prediction model to obtain the numerical value, wherein the prediction model is obtained by training a neural network based on a training data set, and the training data set includes non-dispersive infrared methane sensor digital signal images under different environments, positions, or illuminations.

[0060] The matching module further comprises a training module: collecting non-dispersive infrared methane sensor digital display signal images under different environments, positions, and illuminations as a test data set; inputting the test data set into the prediction model for testing; and stopping training the prediction model after the test result meets the preset recognition accuracy requirement.

[0061] The matching module further comprises a model prediction module:

[0062] Collecting non-dispersive infrared methane sensor digital display signal images under different environments, positions, or illuminations to obtain a sample library;

[0063] According to the sample library, a training data set and a test data set of the neural network are constructed;

[0064] According to the constructed training data set, the neural network is trained to obtain a prediction model of the digital display signal.

[0065] The matching module further comprises a result output module:

[0066] Obtaining a numerical region image and performing size scaling to obtain a numerical region vector;

[0067] Substituting the numerical region vector into the prediction model to obtain a numerical value;

[0068] According to the pixel coordinates, the numbers are sorted from left to right to obtain a string output.

[0069] Collecting methane sensor digital display images under different environments, positions, and illuminations to establish a sample library, which is used to construct a training data set and a test data set of the neural network. According to the constructed training data set, a deep neural network is used to increase the number of neuron connections, threshold values, and other parameters by increasing the number of hidden layers, so as to increase the number of neurons and the number of nested layers of the activation function. Through the pre-training and fine-tuning method, the parameters are grouped, and the local optimal setting is found for each group. Based on the local optimal result, the global optimization is performed. The freedom provided by the large number of parameters of the model is utilized while the training cost is effectively saved. The loss is calculated through continuous iteration of the training model, and the model parameters are updated. Finally, the methane sensor digital display image to be recognized is input into the model for matching to obtain the specific numerical value of the digital display image.

[0070] The present disclosure provides a method for segmenting, collecting, and accurately recognizing multi-channel methane sensor digital display signal images, which aims to flexibly process the recognition of multiple methane sensor numerical signal values and realize the task of monitoring the numerical changes of methane sensors. Through a multi-channel methane sensor recognition system based on deep learning, the accurate recognition of multiple non-dispersive infrared methane sensor digital display signals is realized, the control of the sensor value changes is ensured without human intervention, and the whole process is monitored. It has strong adaptability and high stability. BRIEF DESCRIPTION OF DRAWINGS

[0071] Figure 1 Flow chart for the deep learning based digital display signal recognition method in embodiment 1 of the present disclosure;

[0072] Figure 2 Flow chart for the image segmentation step in embodiment 1 of the present disclosure;

[0073] Figure 3 System diagram for the deep learning based digital display signal recognition system in embodiment 2 of the present disclosure.

[0074] Figure 4 Block diagram for the deep learning based digital display signal recognition system in embodiment 2 of the present disclosure. Embodiments

[0075] Various exemplary embodiments, features, and aspects of the present disclosure will be explained in detail below with reference to the accompanying drawings. Like reference numerals in the drawings denote like or similar elements. Although various aspects of embodiments are illustrated in the drawings, the drawings are not necessarily drawn to scale unless specifically noted.

[0076] The term "exemplary" is used herein to mean "serving as an example, instance, or illustration." Any implementation described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other implementations.

[0077] The term "and / or" used herein only means an association relationship of the associated objects, and means that there can be three relationships, for example, A and / or B can mean that A exists alone, A and B exist together, and B exists alone. In addition, the term "at least one" herein means any one of a plurality or any combination of at least two of a plurality, for example, including at least one of A, B, and C can mean including any one or more elements selected from the set consisting of A, B, and C.

[0078] In addition, in order to better illustrate the present disclosure, numerous specific details are given in the specific embodiments below. Those skilled in the art should understand that the present disclosure can also be implemented without certain specific details. In some examples, methods, means, elements and circuits that are well known to those skilled in the art are not described in detail in order to highlight the main idea of the present disclosure.

[0079] It can be understood that the above-mentioned various method embodiments mentioned in the present disclosure can be combined with each other to form combined embodiments without violating the principle logic. Limited by the length, the present disclosure will not be described again.

[0080] In addition, the disclosure also provides a deep learning-based digital display signal recognition method system, an electronic device, a computer readable storage medium, and a program, which can be used to implement any of the deep learning-based digital display signal recognition methods provided by the disclosure. The corresponding technical solutions and descriptions are described in the method section and are not repeated here.

[0081] The execution subject of the deep learning-based digital display signal recognition method can be a computer or other device capable of implementing the method. For example, the method can be executed by a terminal device or a server or other processing device. The terminal device can be a user equipment (UE), a mobile device, a user terminal, a terminal, a cellular phone, a cordless phone, a personal digital assistant (PDA), a handheld device, a computing device, a vehicle-mounted device, a wearable device, etc. In some possible implementation manners, the method can be implemented by a processor invoking computer-readable instructions stored in a memory. Embodiments

[0082] As an aspect of the embodiments of the disclosure, a deep learning-based digital display signal recognition method is provided, which includes the following steps as shown in the method. Figure 1 The method includes the following steps as shown in the method.

[0083] S1. Obtain an image containing digital display signals of a multi-channel non-dispersive infrared methane sensor;

[0084] S2. Segment the image using a flood fill method and mark a marked area containing the digital display signals;

[0085] Segmenting the image using the flood fill method further includes the following steps.

[0086] Perform binaryzation processing on the image to obtain a grayscale image and perform Gaussian filtering on the image to obtain a blurred image;

[0087] Obtain changes in adjacent pixel points of the grayscale image and changes in adjacent pixel points of the blurred image;

[0088] According to the changes in adjacent pixel points of the grayscale image and the changes in adjacent pixel points of the blurred image, normalize the pixel values of the grayscale image and the blurred image. If the high-frequency components in the blurred image have almost no changes compared with the high-frequency components in the grayscale image, it is determined as a completely blurred image and is not processed. If the high-frequency components change significantly, it is determined as a partially blurred image. If the high-frequency components change greatly, it is determined as a clear image.

[0089] For clear image will directly next link segmentation processing, for part of the fuzzy image will be through the defuzzification processing after re-judging image is clear image, if is then next link segmentation processing, for completely fuzzy image, not to deal with.

[0090] Defuzzification processing includes: for part of the fuzzy image pixel gray statistics, get its gray image pixel probability distribution; get the image cumulative distribution function, get the transformed defuzzification image; for part of the fuzzy image first pixel gray statistics, calculate the pixel probability distribution of the original image, get the cumulative distribution function of the image from the pixel probability distribution, get the transformed defuzzification image according to the mapping function, the mapping function is as follows:

[0091] K=0,1,2,3,···L-1

[0092] Wherein, n is the total sum of pixels in the image, The number of pixels of the current gray level, L is the total number of possible gray levels in the image.

[0093] As Figure 2 Shown in Fig. 1 is the flow chart of the image segmentation step in the embodiment, the next link segmentation processing includes:

[0094] Using mask matrix to mark the flood fill area; a single channel image with a size of 2 pixels larger than the width and height of the input image is used as a mask matrix, and the pixel value of the pixel point in the mask matrix is filled to mark the flood fill area (the flood fill area is the display area of the multi-channel methane sensor digital signal on the image)

[0095] Obtain the upper and lower boundary values of the seed point area condition, wherein the upper and lower boundary values include: upper boundary value and lower boundary value;

[0096] Determine the coordinate value of the seed point;

[0097] Taking the seed point as the center, when the difference between the pixel value of a pixel point in the neighborhood and the pixel value of the seed point is greater than the lower boundary value, the pixel point is added to the region where the seed point is located; when the difference between the pixel value of the seed point and the pixel value of a pixel point in the neighborhood is less than the upper boundary value, the pixel point is added to the region where the seed point is located;

[0098] If all pixel points are not added, mark the position; take the newly added pixel point as a new seed point, return to the previous step: take the seed point as the center, when the difference between the pixel value of a pixel point in the neighborhood and the pixel value of the seed point is greater than the lower bound, the pixel point is added to the region where the seed point is located; when the difference between the pixel value of a seed point and the pixel value of a pixel point in the neighborhood is less than the upper bound, the pixel point is added to the region where the seed point is located; until all pixel points are added; for example, take the seed point as the center, take the seed point as the center, judge the difference between the pixel value in the 4-neighborhood and the pixel value of the seed point, the difference is less than the upper bound and greater than the lower bound, add the pixel point to the region where the seed point is located; and take the newly added pixel point as a new seed point.

[0099] If all pixel points are added, separate and mark the output of each region;

[0100] The image is segmented by using the flood fill method, and the region block where the numerical value is located is segmented from the image frame containing the digital display signal of the multi-channel methane sensor. The same region is found according to the difference between the pixel gray value, the gray value of the pixel point is understood as the height of the pixel point, and a frame of image is regarded as a rugged mountainous area. A certain amount of water is injected into a low-lying area on the ground, and the water surface will cover the area below a certain height. Based on this principle, a water injection pixel is selected in the image, which is a seed point. The seed point will continuously diffuse outward according to certain rules to form an independent region with similar characteristics, thereby realizing image segmentation and framing and marking the region to be identified.

[0101] The host computer sends a specified instruction to the processor, and the camera continuously and intermittently captures multiple methane sensor digital display images. The first clarity recognition processing is performed on the original image, the next step of segmentation processing is directly performed on the clear image, the image is re-judged after deblurring processing for part of the blurred image, and there is no processing significance for the completely blurred image, which is not processed. After completing the selection of the image, the image is segmented based on the distribution of the numerical value, and the region to be identified is found and marked.

[0102] S3. Establish the color distribution range of the digital display signal in the marked region based on the H channel of the HSV model, and obtain the numerical value region image;

[0103] In some embodiments, in the HSV color space, the value range of hue H is [0, 360]. Each pixel in an 8-bit image can represent 256 gray levels, so when representing the HSV image in the 8-bit image, the angle of the hue needs to be mapped to the range of [0, 255]. After determining the value range, the corresponding value in the H channel of the image can be directly found to find the specific color. In the saturation S, the R, G, and B components of the gray color are equivalent, which corresponds to a very unsaturated color. Therefore, the saturation of the gray color is 0. When displayed as a gray image, the color corresponding to the brighter area has a higher saturation. If the saturation of the color is very low, the calculated hue is unreliable. The brightness V range is consistent with the saturation range. The larger the brightness value, the brighter the image; the lower the brightness value, the darker the image.

[0104] In some embodiments, in the HSV, the V channel is most affected by light, and the H channel is basically not affected by shadows or high brightness. The H channel will be the main basis for color extraction in the system, and the color distribution range of the numerical signal will be determined to generate a mask and finally accurately extract the digital value of the marked area.

[0105] In some embodiments, for the extraction of numerical values in the to-be-identified area, based on the HSV model, the distribution range of the numerical color is found by analyzing the distribution histogram of the H component. The extraction of the numerical value in the area is completed based on the H channel.

[0106] S4. Inputting the numerical value area image into the prediction model to obtain the numerical value, wherein the prediction model is obtained by training a neural network based on a training data set, and the training data set includes non-dispersive infrared methane sensor digital display signal images under different environments, positions, or illuminations.

[0107] Before training the neural network based on the training data set to obtain the prediction model, the following steps are further included: collecting non-dispersive infrared methane sensor digital display signal images under different environments, positions, or illuminations as a test data set; inputting the test data set into the prediction model for testing; and stopping training the prediction model when the test result meets the requirements of the preset recognition accuracy.

[0108] The prediction model is obtained by training a neural network based on a training data set, which further includes the following steps:

[0109] Collecting non-dispersive infrared methane sensor digital display signal images under different environments, positions, or illuminations to obtain a sample library;

[0110] Constructing a training data set and a test data set of the neural network according to the sample library;

[0111] Training the neural network based on the constructed training data set to obtain a prediction model of the digital display signal.

[0112] The numerical area image is input into the prediction model for matching to obtain a numerical value, and further comprising the following steps:

[0113] The numerical area image is input into the prediction model for matching to obtain a numerical value, and further comprising the following steps:

[0114] The numerical area image is input into the prediction model for matching to obtain a numerical value, and further comprising the following steps:

[0115] According to the pixel coordinates, the numbers are sorted from left to right to obtain a string output.

[0116] In some embodiments, different environments, positions, and illuminations are collected to establish a sample library for constructing a training data set and a test data set of a neural network. According to the constructed training data set, a deep neural network is used to increase the number of neurons, the connection weight, and the threshold value by increasing the number of hidden layers, so as to increase the number of neurons and the number of nested layers of the activation function. Through the pre-training and fine-tuning method, the parameters are grouped, and the local optimal setting is found for each group. Based on the local optimal result, the global optimization is performed. The freedom provided by the large number of parameters of the model is utilized while the training cost is effectively saved. The loss is calculated by continuously iterating the training model, and the model parameters are updated. Finally, the digital display image of the methane sensor to be recognized is input into the model for matching to obtain the specific numerical value of the digital display image.

[0117] The numerical matching process includes:

[0118] S301. Collecting digital display signal images of methane sensors in different environments

[0119] S302. Constructing a training and test data set of a neural network according to a sample library

[0120] S303. Training the neural network according to the constructed training set to obtain a methane sensor digital display signal prediction model

[0121] S304. Inputting the image to be recognized into the prediction model

[0122] S305. Obtaining a numerical area and performing size scaling processing to obtain a vector form

[0123] S306. Inputting the numerical area vector into the model for matching to obtain a numerical value

[0124] S307. According to the pixel coordinates, the numbers are sorted from left to right to obtain a string output.

[0125] Through multiple acquisition of the digital display image under the field environment, a test data set and a training data set are constructed, the data set is used for training the network, a required model is completed, the numerical value area extracted above is scaled to the model size and is substituted into the model for numerical value matching and output. The identification of the digital display signals of the multiple methane sensors is completed.

[0126] The method is tested in a laboratory of a metrological verification test center, the environmental temperature is kept between 15 and 35 DEG C, the relative humidity is not greater than 85%, and the pressure is standard atmospheric pressure. During the test process, the sensor to be tested is hung in a verification cabinet designed by the system, four rows of hooks are fixed in the cabinet to hang the sensors to be tested, so that twelve sensors can be simultaneously verified. The wide-angle camera used for detection is placed in front of the twelve sensors and is connected to the processor through a signal line. Embodiment

[0127] As another aspect of the embodiments of the present disclosure, a deep learning-based multi-channel methane sensor digital display signal identification system is provided, as shown in the accompanying drawings, comprising: Figure 3 An image acquisition module acquires an image containing digital display signals of multiple non-dispersive infrared methane sensors;

[0128] An image segmentation module realizes segmentation of the image and labels a marked area containing the digital display signals by using a flood fill method.

[0129] For an image frame containing digital display signals of multiple methane sensors, a region block where the numerical values are located is segmented. According to the difference between the pixel gray values, the same region is found, the pixel gray value is understood as the height of the pixel point, and a frame of image is regarded as rugged mountainous land. A certain amount of water is injected into a certain low-lying section of the ground, and the water surface will cover the area below a certain height. Based on this principle, a water injection pixel is selected in the image, which is a seed point. The seed point will continuously diffuse outward according to certain rules to form an independent region with similar characteristics, thereby realizing image segmentation and framing and labeling the region to be identified.

[0130] The host computer sends a specified instruction to the processor, the camera continuously and intermittently takes multiple methane sensor digital display images, the original image is subjected to a first definition identification process, the clear image is directly subjected to a segmentation process of the next link, the part of the fuzzy image is subjected to a de-fuzzing process and then re-judged whether the image is qualified, and the completely fuzzy image has no processing significance and is not processed. After the image screening is completed, the image is segmented based on the distribution of the numerical values, the region to be identified is found and labeled.

[0131]

[0132] ​The image segmentation module further comprises an image preprocessing module: the image is binarized to obtain a gray image, and the image is subjected to Gaussian filtering to obtain a blurred image;

[0133] Changes of adjacent pixel points of the gray image and changes of adjacent pixel points of the blurred image are obtained;

[0134] According to the changes of adjacent pixel points of the gray image and the changes of adjacent pixel points of the blurred image, pixel values of the gray image and the blurred image are normalized, if high-frequency components in the blurred image have almost no change compared with high-frequency components of the gray image, the blurred image is determined to be a completely blurred image and is not processed, if the high-frequency components change obviously, the blurred image is determined to be a partially blurred image, and if the high-frequency components change greatly, the blurred image is determined to be a clear image;

[0135] The clear image is directly subjected to segmentation processing of the next link, the partially blurred image is subjected to deblurring processing and then it is determined whether the image is a clear image, if yes, the image is subjected to segmentation processing of the next link, and the completely blurred image is not processed.

[0136] The image segmentation module further comprises a deblurring processing module, which comprises: pixel gray statistics of the partially blurred image are obtained to obtain pixel probability distribution of the gray image; an image cumulative distribution function is obtained to obtain a transformed image.

[0137] The image segmentation module further comprises a segmentation processing module of the next link, which is used for:

[0138] A mask matrix is used to mark a flood fill region;

[0139] A single-channel image with a size of 2 pixels larger than the width and height of an input image is used as the mask matrix, and the flood fill region (the flood fill region is a display region of a multi-channel methane sensor digital display signal on an image) is marked by filling pixel values of pixel points in the mask matrix.

[0140] Upper and lower boundary values of a seed point region condition are obtained, wherein the upper and lower boundary values comprise an upper boundary value and a lower boundary value;

[0141] A coordinate value of the seed point is determined;

[0142] When a difference between a pixel value of a pixel point in a neighborhood and a pixel value of the seed point is greater than the lower boundary value, the pixel point is added to the region where the seed point is located; when the difference between the pixel value of the seed point and the pixel value of the pixel point in the neighborhood is less than the upper boundary value, the pixel point is added to the region where the seed point is located;

[0143] If all pixel points are not added, mark the position; take the newly added pixel point as a new seed point, return to the previous step: take the seed point as the center, when the difference between the pixel value of a pixel point in the neighborhood and the pixel value of the seed point is greater than the lower bound, the pixel point is added to the region where the seed point is located; when the difference between the pixel value of a seed point and the pixel value of a pixel point in the neighborhood is less than the upper bound, the pixel point is added to the region where the seed point is located; until all pixel points are added; for example, take the seed point as the center, take the seed point as the center, judge the difference between the pixel value in the 4-neighborhood and the pixel value of the seed point, the difference is less than the upper bound and greater than the lower bound, add the pixel point to the region where the seed point is located; and take the newly added pixel point as a new seed point.

[0144] If all pixel points are added, separate and mark the output of each region;

[0145] The extraction module establishes the color distribution range of the digital signal in the marked region based on the H channel of the HSV model, and obtains a digital region image;

[0146] In HSV, the V channel is most affected by light, and the H channel is basically not affected by shadows or high brightness. The H channel will be the main reliance in the color extraction of the system, and will further establish the color distribution range of the digital signal, generate a mask, and finally accurately extract the numbers in the marked region.

[0147] For the extraction of the numbers in the to-be-recognized region, based on the HSV model, the distribution histogram of the H component is analyzed to find the distribution range of the number color. The extraction of the region numbers is completed based on the H channel.

[0148] In some embodiments, in the HSV color space, the value range of hue H is [0, 360]. Each pixel point in an 8-bit image can represent 256 gray levels, so when an HSV image is represented in an 8-bit image, the angle of hue needs to be mapped to the range [0, 255]. After determining the value range, the corresponding value in the H channel of the image can be directly found, thereby finding a specific color. In saturation S, the R, G, and B components contained in a gray color are quite comparable, which is equivalent to a very unsaturated color. Therefore, the saturation of a gray color is 0. When displayed as a gray image, the color corresponding to a brighter area has a higher saturation. If the saturation of a color is very low, the calculated hue is unreliable. The brightness V range is consistent with the saturation range. The larger the brightness value, the brighter the image; the lower the brightness value, the darker the image.

[0149] The matching module inputs the digital region image into a prediction model to obtain the number, wherein the prediction model is obtained by training a neural network through a training data set, and the training data set includes non-dispersive infrared methane sensor digital signal images under different environments, positions, or illuminations.

[0150] The matching module further comprises a training module: collecting non-dispersive infrared methane sensor digital display signal images under different environments, positions, and illuminations as a test data set; inputting the test data set into the prediction model for testing; and stopping training the prediction model after the test result meets the requirements of a preset recognition accuracy.

[0151] The matching module further comprises a model prediction module:

[0152] Collecting non-dispersive infrared methane sensor digital display signal images under different environments, positions, or illuminations to obtain a sample library;

[0153] According to the sample library, a training data set and a test data set of a neural network are constructed;

[0154] According to the constructed training data set, the neural network is trained to obtain a prediction model of the digital display signal.

[0155] The matching module further comprises a result output module:

[0156] Obtaining a numerical region image and performing size scaling to obtain a numerical region vector;

[0157] Substituting the numerical region vector into the prediction model to obtain a numerical value;

[0158] According to the pixel coordinates, the numbers are sorted from left to right to obtain a string output.

[0159] In some embodiments, different environments, positions, and illuminations are collected to obtain methane sensor digital display images, a sample library is established, and a training data set and a test data set of a neural network are constructed. According to the constructed training data set, a deep neural network is used to increase the number of neurons, the connection weight, the threshold value, and other parameters by increasing the number of hidden layers, so as to increase the number of neurons and the number of nested layers of the activation function. Through the pre-training and fine-tuning method, the parameters are grouped, and the local optimal setting is found for each group. Based on the local optimal result, global optimization is performed. The freedom provided by the large number of parameters of the model is utilized while the training cost is effectively saved. The loss is calculated through continuous iteration of the training model, and the model parameters are updated. Finally, the methane sensor digital display image to be recognized is input into the model for matching to obtain the specific numerical value of the digital display image.

[0160] Through multiple collections of digital display images under field environments, test data sets and training data sets are constructed, the data sets are trained, the required model for recognition is completed, the numerical region extracted above is scaled to the size of the model and substituted into the model for numerical matching and output. The recognition of the digital display signals of multiple methane sensors is completed. Figure 4 The above is the structure diagram of the system. Embodiments

[0161] An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, the processor implementing a deep learning based digital display signal recognition method of embodiment 1 when executing the computer program.

[0162] Embodiment 3 of the present disclosure is merely an example and should not bring any limitation to the function and use range of embodiments of the present disclosure.

[0163] The electronic device can be in the form of a general computing device, for example, it can be a server device. The components of the electronic device can include, but are not limited to, at least one processor, at least one memory, a bus connecting different system components including the memory and the processor.

[0164] The bus includes a data bus, an address bus, and a control bus.

[0165] The memory can include volatile memory, such as random access memory (RAM) and / or cache memory, and can further include non-volatile memory, such as read-only memory (ROM).

[0166] The memory can also include program tools having a set of (at least one) program modules, such as each of or a combination of the following examples: an operating system, one or more application programs, other program modules, and program data, which can include the implementation of a network environment.

[0167] The processor performs various function applications and data processing by running the computer program stored in the memory.

[0168] The electronic device can also communicate with one or more external devices (such as a keyboard, a pointing device, etc.) through an input / output (I / O) interface. And the electronic device can also communicate with one or more networks (such as a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) through a network adapter. The network adapter communicates with other modules of the electronic device through the bus. It should be understood that although not shown in the figure, other hardware and / or software modules can be used in conjunction with the electronic device, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID (Redundant Array of Independent Disks) systems, tape drives, and data backup storage systems, etc.

[0169] It should be noted that although several units / modules or sub-units / modules of the electronic device are mentioned in the foregoing detailed description, such a division is merely exemplary and not mandatory. Indeed, according to an embodiment of the application, the features and functions of two or more units / modules described above can be embodied in one unit / module. Conversely, the features and functions of one unit / module described above can be further divided into units / modules embodied by several units / modules. Embodiments

[0170] A computer readable storage medium, the readable storage medium storing a computer program, the program being executed by a processor to implement the steps of the deep learning based digital display signal recognition method in embodiment 1.

[0171] More specifically, the readable storage medium can include, but is not limited to, a portable disc, a hard disk, a random access memory, a read-only memory, an erasable programmable read-only memory, an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0172] In possible embodiments, the present disclosure can also be implemented in the form of a program product, which includes program codes for causing the terminal device to perform the steps of the deep learning based digital display signal recognition method described in embodiment 1 when the program product is run on the terminal device.

[0173] The program codes for implementing the present disclosure can be written in any combination of one or more programming languages, and can be executed entirely on the user device, partially on the user device, as a standalone software package, partially on the user device and partially on a remote device, or entirely on a remote device.

Claims

1. A digital display signal recognition method based on deep learning, characterized in that, Includes the following steps: Acquire an image containing digital display signals from multiple nondispersive infrared methane sensors; The image is segmented and the regions containing digital display signals are marked using a flood filling method. The flood filling method further includes the following steps: binarizing the image to obtain a grayscale image; performing Gaussian filtering on the image to obtain a blurred image; acquiring the changes in adjacent pixels of the grayscale image and the blurred image; normalizing the pixel values ​​of both the grayscale and blurred images based on these changes; and determining whether the blurred image is completely blurred, partially blurred, or clear based on a comparison of high-frequency components in the blurred image and the grayscale image. Clear images will proceed directly to the next segmentation stage; partially blurred images will be deblurred and then re-evaluated for clarity; if clear, the next segmentation stage will proceed; completely blurred images will not be processed. The H channel based on the HSV model establishes the color distribution range of the digital display signal in the marked area and acquires the image of the numerical region; The numerical region image is input into the prediction model for matching to obtain the numerical value. The prediction model is obtained by training the neural network with a training dataset, which includes non-dispersive infrared methane sensor digital display signal images under different environments, locations, or lighting conditions.

2. The deep learning-based digital display signal recognition method as described in claim 1, characterized in that, Before training the neural network to obtain the prediction model using the training dataset, the following steps are also included: acquiring digital display signal images of nondispersive infrared methane sensors under different environments, locations, and lighting conditions as a test dataset; and inputting the test dataset into the prediction model for testing. Training of the prediction model will stop once the test results meet the preset recognition accuracy requirements.

3. The deep learning-based digital display signal recognition method as described in claim 1, characterized in that, The deblurring process includes the following steps: performing pixel grayscale statistics on a partially blurred image to obtain the pixel probability distribution of the grayscale image; obtaining the cumulative distribution function of the image to acquire the transformed deblurred image; for the partially blurred image, first performing pixel grayscale statistics to calculate the pixel probability distribution of the original image, obtaining the cumulative distribution function of the image from the pixel probability distribution, and obtaining the transformed deblurred image according to the mapping function, wherein the mapping function is as follows: ,k= 0,1,2,3,···L-1 Where n is the sum of the pixels in the image. L is the number of pixels at the current gray level, and L is the total number of possible gray levels in the image.

4. The deep learning-based digital display signal recognition method as described in claim 1, characterized in that, The next step of segmentation includes: A single-channel image, which is 2 pixels larger than the width and height of the input image, is used as a mask matrix. The flood-filled area is marked by filling the pixel values ​​of the pixels in the mask matrix. Obtain the upper and lower bounds of the seed point region conditions, where the upper and lower bounds include an upper bound and a lower bound. Determine the coordinates of the seed point; Centered on the seed point, when the difference between the pixel value of a neighboring pixel and the pixel value of the seed point is greater than the lower bound, the pixel is added to the region where the seed point is located; when the difference between the pixel value of the seed point and the pixel value of a neighboring pixel is less than the upper bound, the pixel is added to the region where the seed point is located. If not all pixels have been added, mark the position; use the newly added pixel as a new seed point, and return to the previous step: with the seed point as the center, when the difference between the pixel value of a neighboring pixel and the pixel value of the seed point is greater than the lower bound, the pixel is added to the region where the seed point is located; when the difference between the pixel value of the seed point and the pixel value of a neighboring pixel is less than the upper bound, the pixel is added to the region where the seed point is located; until all pixels have been added; Once all pixels have been added, each region is separated, marked, and output.

5. The deep learning-based digital display signal recognition method as described in claim 1, characterized in that, The prediction model, obtained by training the neural network using a training dataset, also includes the following steps: A sample library was obtained by collecting digital display signal images of nondispersive infrared methane sensors under different environments, locations, or lighting conditions. The training and test datasets for the neural network are constructed based on the sample library. The neural network is trained based on the constructed training dataset to obtain a prediction model for digital display signals.

6. The deep learning-based digital display signal recognition method as described in claim 1 or 5, characterized in that, The process of inputting the numerical region image into a prediction model for matching to obtain numerical values ​​also includes the following steps: Obtain the numerical region image and scale it to obtain the numerical region vector; The numerical region vector is substituted into the prediction model for matching to obtain the numerical value. Sort the numbers from left to right according to their pixel coordinates, concatenate them into a string, and output the string.

7. A digital display signal recognition system based on deep learning, characterized in that, include: The image acquisition module acquires images containing digital display signals from multiple non-dispersive infrared methane sensors; The image segmentation module uses a flood fill method to segment the image and mark the regions containing digital display signals. The flood fill method further includes the following steps: binarizing the image to obtain a grayscale image; performing Gaussian filtering on the image to obtain a blurred image; acquiring the changes in adjacent pixels of the grayscale image and the blurred image; normalizing the pixel values ​​of both the grayscale and blurred images based on these changes; and determining whether the blurred image is completely blurred, partially blurred, or clear based on a comparison of high-frequency components in the blurred image and the grayscale image. Clear images will proceed directly to the next segmentation stage; partially blurred images will be deblurred and then re-evaluated for clarity; if clear, the next segmentation stage will proceed; completely blurred images will not be processed. The extraction module establishes the color distribution range of the digital display signal in the marked area based on the H channel of the HSV model and acquires the image of the numerical region. The matching module inputs the numerical region image into the prediction model for matching to obtain the numerical value. The prediction model is obtained by training the neural network with a training dataset, which includes non-dispersive infrared methane sensor digital display signal images under different environments, positions, or lighting conditions.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the deep learning-based digital display signal recognition method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the deep learning-based digital display signal recognition method according to any one of claims 1 to 6.

Images