Device panel signal detection method based on image recognition algorithm

Abstract

The present application discloses a device panel signal detection method based on an image recognition algorithm, which obtains an image to be detected of a device panel, and performs image segmentation based on at least one signal detection area corresponding to the device panel to obtain a signal detection image corresponding to each signal detection area; the signal detection image includes at least one indicator and the coordinates corresponding to each indicator; color detection is performed on each signal detection image to determine the status color corresponding to the signal detection image; based on the abnormal strategy corresponding to the signal detection image and the status color, it is determined whether the signal corresponding to the signal detection image is abnormal, so that device panel signal detection can be completed efficiently, accurately and at low cost.

Description

Technical Field

[0001] The present disclosure generally relates to the field of signal detection technology, and in particular to a device panel signal detection method based on an image recognition algorithm. Background Art

[0002] With the development of the times, the abnormal positioning technology of equipment panel signals based on image recognition algorithms has received extensive attention and application in the field of industrial control and automation. The existing equipment panel signal detection technology based on image recognition algorithms mainly includes traditional manual detection methods and training recognition models.

[0003] Traditional manual detection methods rely on operators to visually check the signal indicators on the equipment panel, such as indicator lights. Operators need to check the signal indicators one by one, judge the operating status of the equipment according to the status of the indicators, and take corresponding measures when abnormalities are found. There are problems of low efficiency, poor accuracy and high labor costs, and it is easy to miss or misdetect due to fatigue and lack of concentration. Although the training recognition model has high automation and high recognition accuracy, it requires a large amount of training data and computing resources, the model training and maintenance costs are high, the requirements for technical personnel are high, and the implementation is complex. Summary of the invention

[0004] In view of the above-mentioned defects or deficiencies in the prior art, it is desirable to provide a device panel signal detection method based on an image recognition algorithm, which can complete device panel signal detection efficiently, accurately and at low cost.

[0005] In a first aspect, the present application proposes a device panel signal detection method based on an image recognition algorithm, comprising:

[0006] Acquire an image to be detected of a device panel, and perform image segmentation based on at least one signal detection area corresponding to the device panel to obtain a signal detection image corresponding to each signal detection area; the signal detection image includes at least one indicator and coordinates corresponding to each indicator;

[0007] Performing color detection on each of the signal detection images to determine a state color corresponding to the signal detection image;

[0008] Based on the abnormality strategy corresponding to the signal detection image and the status color, it is determined whether the signal corresponding to the signal detection image is abnormal.

[0009] In some embodiments, performing image segmentation based on at least one signal detection area corresponding to the device panel includes:

[0010] Converting the image to be detected into a grayscale image;

[0011] Marking the grayscale image pixel by pixel to determine at least one connected region in the grayscale image,

[0012] Calculating the area of ​​each of the connected regions, and taking at least one of the connected regions that meets a preset area condition as the at least one signal detection region;

[0013] The image to be detected is segmented based on the at least one signal detection area.

[0014] In some embodiments, the performing image segmentation on the image to be detected based on the at least one signal detection area includes:

[0015] Performing mask processing on the image to be detected based on each of the signal detection areas to obtain an edge contour corresponding to each of the signal detection areas;

[0016] The image to be detected is segmented based on the edge contour, and the coordinates corresponding to each of the signal detection images are obtained.

[0017] In some embodiments, performing image segmentation based on at least one signal detection area corresponding to the device panel includes:

[0018] Inputting the image to be detected into an image semantic segmentation model for processing;

[0019] The image to be detected is segmented by the image semantic segmentation model, and the coordinates corresponding to each of the signal detection images are obtained.

[0020] In some embodiments, performing color detection on each of the signal detection images to determine the state color corresponding to the signal detection image includes:

[0021] Obtaining the color ratio corresponding to each color component in each of the signal detection images;

[0022] The color component with the largest color ratio is used as the state color corresponding to the signal detection image.

[0023] In some embodiments, performing color detection on each of the signal detection images to determine the state color corresponding to the signal detection image includes:

[0024] Performing image segmentation on the grayscale image based on the at least one signal detection area to obtain a grayscale detection image corresponding to the signal detection image;

[0025] For each of the grayscale detection images, calculate the color values ​​corresponding to the three color channels based on the grayscale values;

[0026] The channel color with the largest color value ratio is used as the status color corresponding to the signal detection image.

[0027] In some embodiments, before performing image segmentation on the grayscale image based on the at least one signal detection area, the method further includes:

[0028] Performing image enhancement on the grayscale image to obtain an enhanced image;

[0029] Obtaining a contrast evaluation index corresponding to at least one indicator color in the enhanced image;

[0030] When the contrast evaluation index corresponding to any indicator color is greater than a preset evaluation index, the grayscale image is corrected to obtain the corrected grayscale image.

[0031] In some embodiments, the contrast rating index is calculated using the following formula:

[0032]

[0033] Wherein, P is the contrast evaluation index corresponding to any indicator color in the enhanced image, S n is the grayscale average value of any indicator color in the enhanced image, T m is the grayscale average of other indicator colors in the enhanced image, S s is the standard grayscale value of the indicator color, x is the number of pixels corresponding to the indicator color in the enhanced image, y is the number of pixels corresponding to other indicator colors, K is S n and S s Same number of pixels.

[0034] In some embodiments, determining whether the signal corresponding to the signal detection area is abnormal based on the abnormal strategy corresponding to the signal detection area and the status color includes:

[0035] Using text recognition technology to obtain identification information corresponding to each of the signal detection areas;

[0036] Obtaining an exception policy corresponding to the identification information;

[0037] When the status color satisfies the abnormal condition in the abnormal strategy, it is determined that the signal corresponding to the signal detection area is abnormal.

[0038] In some embodiments, the obtaining identification information corresponding to each of the signal detection areas by using a text recognition technology further includes:

[0039] Based on the coordinates corresponding to the indicator, acquiring the identification area of ​​the identification information corresponding to the indicator;

[0040] Perform text recognition on the identification area to obtain identification information corresponding to the indicator.

[0041] The device panel signal detection method based on the image recognition algorithm provided in the embodiment of the present application can perform signal detection on the image to be detected corresponding to the device panel through the image algorithm. By using the improved image algorithm, the influence of halo, scattering, etc. between adjacent indicator lights is effectively overcome, and the accuracy of image-based color detection is greatly improved, and the reliability of device detection is improved. Moreover, no human participation is required, which effectively avoids the generation of human recognition errors, and no large model is used. On the basis of ensuring data accuracy and reliability, the training pressure and cost are reduced.

[0042] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Other features, objects and advantages of the present application will become more apparent by reading the detailed description of non-limiting embodiments made with reference to the following drawings:

[0044] Figure 1 The following is a diagram showing an implementation environment architecture of a device panel signal detection method based on an image recognition algorithm provided in an embodiment of the present application;

[0045] Figure 2 A schematic diagram of a process of a device panel signal detection method based on an image recognition algorithm provided in an embodiment of the present application is shown;

[0046] Figure 3 A schematic diagram of a process of a device panel signal detection method based on an image recognition algorithm provided by another embodiment of the present application is shown;

[0047] Figure 4 A schematic diagram of a device panel provided by an embodiment of the present application is shown;

[0048] Figure 5 A schematic diagram of a color histogram provided in an embodiment of the present application is shown;

[0049] Figure 6 A schematic diagram of a process of a device panel signal detection method based on an image recognition algorithm provided by another embodiment of the present application is shown;

[0050] Figure 7 A schematic diagram of the structure of a device panel signal detection device based on an image recognition algorithm provided by an embodiment of the present application is shown;

[0051] Figure 8A schematic diagram of the structure of a computer system of an electronic device or server suitable for implementing an embodiment of the present application is shown. DETAILED DESCRIPTION

[0052] The present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the relevant invention, rather than to limit the invention. It is also necessary to explain that, for ease of description, only the parts related to the invention are shown in the accompanying drawings.

[0053] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0054] For the specific implementation environment of the device panel signal detection method based on image recognition algorithm proposed in this application, see Figure 1 . Figure 1 The following is a diagram showing the implementation environment architecture of the device panel signal detection method based on the image recognition algorithm provided in an embodiment of the present application.

[0055] like Figure 1 As shown, the implementation environment architecture includes: a device to be detected 101, an image acquisition device 102 and a server 103.

[0056] The device to be detected 101 is a device provided with an indicator 1011, and the indicator 1011 is used to indicate the status information of the device to be detected 101 under the corresponding item, such as normal, abnormal, etc. In the embodiment of the present application, the device to be detected 101 can be an electrical device, and the indicator 1011 is an indicator light. That is, the indicator 1011 indicates the status of the device to be detected 101 in the corresponding item of the indicator 1011 by turning the indicator light on, off, or changing the color.

[0057] The image acquisition device 102 is used to acquire the panel image of the device to be detected 101, so that the server 103 uses the panel image as the image to be detected for signal detection. The image acquisition device 102 can be a device capable of acquiring images at a certain preset frequency, such as a camera, a still camera, etc.

[0058] The server 103 is connected to the image acquisition device 102 and is used to receive the panel image acquired by the image acquisition device 102 and use it as the image to be detected to perform signal detection and determine whether the signal on the device to be detected 101 is abnormal.

[0059] Server 103 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers. It can also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, as well as big data and artificial intelligence platforms.

[0060] The image acquisition device 102 is directly or indirectly connected to the server 102 via wired or wireless communication. Optionally, the wireless network or wired network uses standard communication technology and / or protocols. The network is usually the Internet, and can also be any network, including but not limited to a local area network (LAN), a metropolitan area network (MAN), a wide area network (WAN), a mobile, wired or wireless network, a private network or any combination of a virtual private network.

[0061] The device panel signal detection method based on image recognition algorithm proposed in the present application can be implemented by a device panel signal detection apparatus based on image recognition algorithm, and the device panel signal detection apparatus based on image recognition algorithm can be installed on a terminal device or a server.

[0062] In order to further illustrate the technical solution provided by the embodiment of the present application, this is described in detail below in conjunction with the accompanying drawings and specific implementation methods. Although the embodiment of the present application provides the method operation instruction steps shown in the following embodiments or drawings, more or less operation instruction steps may be included in the method based on routine or no creative labor. In the steps where there is no necessary causal relationship logically, the execution order of these steps is not limited to the execution order provided by the embodiment of the present application. The method may be executed in the order of the method shown in the embodiment or drawings or in parallel during the actual processing process or when the device is executed.

[0063] Please refer to Figure 2 , Figure 2 FIG. 1 is a flow chart of a device panel signal detection method based on an image recognition algorithm provided by an embodiment of the present application. Figure 2 As shown, the method includes:

[0064] Step 201 : acquiring an image to be detected of a device panel, and performing image segmentation based on at least one signal detection area corresponding to the device panel to obtain a signal detection image corresponding to each signal detection area.

[0065] The signal detection image includes at least one indicator and corresponding coordinates of each indicator.

[0066] It should be noted that the display panel of large electrical equipment has the characteristics of a large range and scattered indicators. Based on this, in order to reduce the amount of calculation of the overall image, this application first performs image segmentation on the image to be detected on the equipment panel before performing signal detection and analysis, and obtains the signal detection image of the area where the indicator (light) is located, and further detects and analyzes the signal detection image to improve the accuracy of the detection and analysis.

[0067] In a possible embodiment, if Figure 3 As shown, image segmentation is performed based on at least one signal detection area corresponding to the device panel, including:

[0068] Step 301: convert the image to be detected into a grayscale image.

[0069] Specifically, the image to be detected is subjected to linear weighted summation calculation based on the principle of color space conversion to obtain a grayscale image. The grayscale image obtained by this method can retain the brightness information of the image to be detected, simplify the complexity of image processing, reduce the interference of color information, and improve the accuracy and efficiency of subsequent region segmentation.

[0070] Exemplarily, the grayscale image can be calculated using the following formula:

[0071] gray(i,j)=0.30×R(i,j)+0.59×G(i,j)+0.11×B(i,j)

[0072] Among them, gray(i,j) is the gray value at the pixel point (i,j), R(i,j), G(i,j) and B(i,j) are the red, blue and yellow channel values ​​of the pixel point in the image to be detected.

[0073] Step 302: label the grayscale image pixel by pixel to determine at least one connected region in the grayscale image.

[0074] Specifically, the grayscale image is marked pixel by pixel, and the 8-neighbor method is used to determine whether two pixels are adjacent. If two pixels are adjacent and have the same value, they are considered to be in a connected area, and all pixels in the same connected area are marked with a value.

[0075] Step 303: Calculate the area of ​​each connected region, and use at least one connected region that meets a preset area condition as at least one signal detection region.

[0076] That is, after obtaining at least one connected region by pixel-by-pixel labeling, the area size of each connected region is calculated based on the number of pixels in the connected region, such as area0, area1, ..., arean.

[0077] Then, it is determined whether the area of ​​each connected region is greater than the first preset area. If the area of ​​the connected region is greater than the first preset area, the connected region is retained and determined as the signal detection region. In this way, noise and small objects with too small an area can be removed, reducing the impact of noise.

[0078] Preferably, in Figure 4 In the case of the device panel shown, it is further determined whether the area of ​​the connected area is less than the second preset area, wherein the second preset area is greater than the first preset area. If the area of ​​the connected area is less than the second preset area, it means that the connected area is the area corresponding to the indicator (light), and the connected area is used as the signal detection area. It should be understood that when the signal detection area is determined by this embodiment, the image recognition of the signal detection area focuses on the color state recognition of the signal light, and the comprehensive analysis of the abnormal situation indicated by the indicator (light) is implemented by other steps.

[0079] It should also be understood that one or more of the first preset area and the second preset area can be set according to the actual distribution of indicators (lights) on the equipment panel, so as to determine connected areas that meet different conditions through different combinations of the first preset area and the second preset area, so that when performing abnormality analysis later, the indication items corresponding to each connected area can be determined according to the area of ​​the connected area, thereby improving the efficiency of abnormality identification.

[0080] Step 304: perform image segmentation on the image to be detected based on at least one signal detection area.

[0081] Specifically, the image to be detected can be masked based on each signal detection area to obtain the edge contour corresponding to each signal detection area, and the image to be detected can be segmented based on the edge contour to obtain the coordinates corresponding to each signal detection image.

[0082] The mask is a binary image composed of 0 and 1. When an image is processed by masking, the 1-value area is processed, and the masked 0-value area is not processed. Therefore, when performing image segmentation, the signal detection area is retained as the masked 0-value area, and the intersection of the 0-value area and the 1-value area is used as the edge contour corresponding to the signal detection area. The signal detection image is extracted according to the edge contour, thereby extracting each signal detection image after segmentation.

[0083] It should be understood that when the signal detection image is extracted, the coordinate position of the signal detection image is further obtained. Optionally, it can be the actual coordinate position of the signal detection image in the physical world, or it can be the pixel coordinate position of the signal detection image in the image to be detected, which is not specifically limited in this application. The coordinate position is used to associate with the identification information later.

[0084] In another feasible embodiment, image segmentation is performed based on at least one signal detection area corresponding to the device panel, including: inputting the image to be detected into an image semantic segmentation model for processing, and performing image segmentation through the image semantic segmentation model.

[0085] It should be noted that image semantic segmentation is to segment the pixels in the image according to different expression semantics. Image semantic segmentation is pixel-level classification, which realizes semantic annotation of the entire image by classifying each pixel in the image. Specifically, image semantic segmentation classifies each pixel in the image and determines the semantic category to which each pixel belongs, thereby obtaining multiple semantic pixel regions. The pixels in the same semantic pixel region belong to the same semantic category. Semantic category is a language expression divided according to different expression semantics. For example, in the image to be detected of the device panel, the identification area and the indicator (light) belong to different expression semantics. Therefore, the pixels corresponding to the identification area and the pixels corresponding to the indicator (light) belong to different semantic categories.

[0086] In an embodiment of the present application, the image semantic segmentation model may include a convolutional neural network structure, which can perform operations such as convolution calculation, normalization processing, and activation function calculation.

[0087] In a specific embodiment, the training method of the image semantic segmentation model may be: obtaining a training sample image and a training label corresponding to the training sample image, wherein the training label is used to mark the semantic category to which each pixel in the corresponding training sample image belongs. Next, the parameters of the image semantic segmentation model are initialized, and the training sample image is input into the image semantic segmentation model to obtain a predicted semantic segmentation image, in which the pixels in the predicted semantic segmentation image have pixel values ​​representing the semantic category to which they belong. Next, based on the difference between the predicted semantic segmentation image and the training label, a loss function is used to optimize the parameters of the image semantic segmentation model based on a gradient descent algorithm. The image semantic segmentation model is iteratively trained according to the above method until a training stop condition is met. The training stop condition may be that the number of iterations reaches a specified number, or the change in the loss function is less than a specified threshold.

[0088] Therefore, after the image to be detected is input into the image semantic segmentation model for processing, the image is segmented by the image semantic segmentation model, and the signal detection image corresponding to the signal detection area can be directly obtained.

[0089] In a preferred embodiment, the signal detection area corresponds to the indicator one-to-one, that is, each signal detection area contains an indicator. Taking the use of grayscale images for image segmentation as an example, the area of ​​at least one pixel occupied by each indicator is used as a connected area, so that each signal detection area contains only one indicator (light). By performing color recognition on the signal detection image corresponding to the signal detection area one-to-one, the color recognition of each indicator (light) can be achieved, thereby realizing device panel signal detection based on image recognition.

[0090] Therefore, the present application provides two methods for performing image segmentation on the image to be detected. The arithmetic method uses grayscale conversion to effectively reduce the influence of the indicator (light) on the image information, improves the accuracy of signal detection image determination and segmentation, reduces the computational complexity, and improves the efficiency of image segmentation; the model method can obtain a model that can directly perform semantic segmentation through training, and use the model to perform image segmentation, which effectively reduces the computational complexity in the segmentation process, improves the degree of automation and recognition accuracy of image segmentation, can automatically adapt to complex backgrounds and lighting changes, has high adaptability, and improves the generalization ability of image segmentation.

[0091] Step 202: Perform color detection on each signal detection image to determine the state color corresponding to the signal detection image.

[0092] It should be noted that in the embodiment of the present application, the device to be detected displays the status result of the internal self-test of the device to be detected through the indicator light on the device panel. The indicator light display status result includes but is not limited to the indicator light being on, off, flashing, red, yellow, green, etc. The specific display method of the indicator light is not specifically limited in the present application.

[0093] It should be understood that, preferably, each signal detection image includes an indicator (light), and by performing color detection on each signal detection image, that is, performing color detection on the indicator (light) in the signal detection image, the color of the indicator light used to indicate the internal status result of the device is determined.

[0094] In a feasible embodiment, color detection is performed on each signal detection image to determine the status color corresponding to the signal detection image, including: obtaining the color ratio corresponding to each color component in each signal detection image, and taking the color component with the largest color ratio as the status color corresponding to the signal detection image.

[0095] Specifically, for each signal detection image, the three color components R, G, and B of the signal detection image can be extracted using RGB color to obtain the amount of the three color components R, G, and B in the signal detection image. For example, Figure 5As shown, a color histogram can be constructed based on the signal detection image, where the horizontal axis is the three color components R, G, and B, and the vertical axis is the number of pixels occupied by each color component in the signal detection image.

[0096] Optionally, the horizontal axis can also use other color expressions, such as all methods of computer color expression, etc., which can be specifically determined based on the color of the indicator light on the actual device to be tested. In the embodiment of the present application, the preferred indicator light colors of the electrical equipment include but are not limited to red, yellow and blue. Therefore, the three color components R, G, and B can better express the color distribution.

[0097] Furthermore, after determining the number of pixels occupied by each color component in the detection image, the color component with the largest color ratio is used as the state color corresponding to the signal detection image.

[0098] It should be noted that Figure 4 It can be seen that a plurality of indicator lights are arranged in a row on the device panel of the device to be detected, and light waves have the ability to diffuse. Therefore, the area where any indicator light is located may be affected by the light of the adjacent indicator lights. However, since the indicator light itself in the signal detection image is luminous and the segmented signal detection image contains fewer pixels outside the area itself, the indicator light's own luminous color accounts for the color component with the largest color ratio in the signal detection image. The color ratio can then be used to determine the status color corresponding to the indicator light contained in the signal detection image, and at the same time effectively reduce the influence of adjacent indicator lights on the indicator light color in the signal detection image, thereby improving the accuracy of the status color and providing reliable color data for subsequent abnormal judgment.

[0099] In another feasible embodiment, color detection may be performed based on the grayscale image used for image segmentation to determine the state color corresponding to the signal detection image. Specifically, Figure 6 As shown, color detection is performed on each signal detection image to determine the state color corresponding to the signal detection image, including:

[0100] Step 601 : performing image segmentation on a grayscale image based on at least one signal detection region to obtain a grayscale detection image corresponding to the signal detection image.

[0101] It should be noted that since a plurality of continuously arranged indicator lights are provided on the device panel of the device to be detected, light waves have the ability to diffuse. Therefore, the area where any indicator light is located may be affected by the light of the adjacent indicator lights. Therefore, the embodiment of the present application performs image enhancement based on the grayscale image to reduce the influence of the adjacent indicator lights on the signal detection image.

[0102] In a feasible embodiment, before performing image segmentation on the grayscale image based on at least one signal detection area, it also includes: performing image enhancement on the grayscale image to obtain an enhanced image; obtaining a contrast evaluation index corresponding to at least one indicator color in the enhanced image; when the contrast evaluation index corresponding to any indicator color is greater than a preset evaluation index, completing the correction of the grayscale image to obtain a corrected grayscale image.

[0103] Optionally, in an embodiment of the present application, a logarithmic transformation is used to perform image enhancement on the grayscale image to increase the contrast in the grayscale image, thereby reducing the effects caused by indicator light halo, scattering, etc.

[0104] Exemplarily, the logarithmic transformation may adopt the following formula:

[0105] gray ‘ (i,j)=a×log(1+gray(i,j))

[0106] Among them, gray ‘ (i, j) is the gray value of pixel (i, j) after transformation, that is, the gray value of pixel (i, j) in the enhanced image, a is the scale comparison constant, and gray(i, j) is the gray value of pixel (i, j) before transformation.

[0107] And, the contrast evaluation index can be expressed as follows:

[0108]

[0109] Among them, P is the contrast evaluation index corresponding to any indicator color in the enhanced image, S n To enhance the grayscale average value of any indicator color in the image, T m To enhance the grayscale average of other indicator colors in the image, S s is the standard grayscale value of the indicator color, x is the number of pixels corresponding to the indicator color in the enhanced image, y is the number of pixels corresponding to other indicator colors, K is S n and S s Same number of pixels.

[0110] It should be understood that in a color image, different colors will form a fixed gray value after grayscale conversion. Even after image enhancement, the standard gray value corresponding to the color can be determined. The contrast evaluation index is mainly used to evaluate the color of the main color body in the signal detection image after enhancement, and the color tends to the standard gray value.

[0111] After the contrast evaluation index is calculated, by judging that the contrast evaluation index is greater than a preset evaluation index, the obtained contrast evaluation index, that is, the color of the indicator light in the signal detection image, can be maximized.

[0112] Step 602: for each grayscale detection image, calculate the three-channel color values ​​based on the grayscale values.

[0113] That is to say, the (corrected) grayscale image is converted into a color image using the color values ​​of the three color channels.

[0114] For example, the following formula can be used for conversion:

[0115]

[0116] Among them, R(i,j), G(i,j) and B(i,j) are the three-channel color values ​​corresponding to the grayscale value conversion.

[0117] Step 603: Use the channel color with the largest color value as the status color corresponding to the signal detection image.

[0118] Similar to the color analysis based on the image to be detected, it is also necessary to determine the status color based on the proportion of colors.

[0119] Therefore, the embodiment of the present application can effectively reduce the influence of adjacent indicator lights on the color of the indicator lights in the signal detection image by correcting the grayscale image and then performing segmentation and color detection, thereby improving the accuracy of color detection.

[0120] Step 203: Determine whether the signal corresponding to the signal detection area is abnormal based on the abnormal strategy and status color corresponding to the signal detection area.

[0121] It should be noted that after accurately obtaining the status color of the indicator light, it can be determined whether the signal corresponding to the signal detection area (indicator light) is abnormal based on the color indicated by the indicator light and the abnormal strategy corresponding to the indicated item.

[0122] For example, Figure 4 For example, STAT is the hardware status indicator of the board. The green light indicates that the board is working normally, and the red light indicates that the board hardware is faulty or the board hardware does not match. ACT is the service activation status or the active / standby status indicator of the board. The green light indicates that the equipment service is in the active state, the board is working, and the board is in the active state. The light is off, indicating that the equipment service is in the inactive state, and the board is in the standby state.

[0123] Based on this, it is necessary to determine the abnormal strategy corresponding to each indicator light and then judge the signal abnormality according to the status color.

[0124] In a feasible embodiment, text recognition technology is used to obtain identification information corresponding to each signal detection area, and an abnormal strategy corresponding to the identification information is obtained. When the status color meets the abnormal condition in the abnormal strategy, the signal abnormality corresponding to the signal detection area is determined.

[0125] Specifically, based on the coordinates corresponding to the indicator contained in the information detection image, the identification area of ​​the identification information corresponding to the indicator can be obtained, and then text recognition can be performed on the identification area to obtain the identification information corresponding to the indicator. Figure 4 As shown, the indicator light and its corresponding identification area are on the same horizontal line and adjacent to each other, that is, the vertical coordinate range of the pixel points is the same and the horizontal coordinates are adjacent, thereby determining the identification area corresponding to the indicator light, and then using text recognition technology, such as OCR (Optical Character Recognition), to identify the identification information.

[0126] After identifying the identification information corresponding to the indicator light, the identification information can be used to obtain the exception strategy corresponding to the identification information from the database, such as the exception strategy corresponding to the aforementioned STAT and ACT indicator lights. When the color status of the indicator light meets the exception conditions in the exception strategy, for example, the STAT light is always red, then the signal corresponding to the signal detection area is confirmed to be abnormal.

[0127] Therefore, the device panel signal detection method based on the image recognition algorithm provided by the embodiment of the present application can perform signal detection on the image to be detected corresponding to the device panel through the image algorithm, and effectively overcome the influence of halo, scattering, etc. between adjacent indicator lights by using the improved image algorithm, greatly improving the accuracy of image-based color detection and the reliability of device detection. Moreover, no human participation is required, effectively avoiding the generation of human recognition errors, and no large model is used, which reduces the training pressure and cost on the basis of ensuring data accuracy and reliability.

[0128] It should be noted that although the operations of the method of the present invention are described in a particular order in the drawings, this does not require or imply that the operations must be performed in this particular order or that all illustrated operations must be performed to achieve desired results.

[0129] Figure 7 A schematic diagram of the structure of a device panel signal detection apparatus based on an image recognition algorithm provided in one embodiment of the present application is shown.

[0130] like Figure 7 As shown, the device panel signal detection device 10 based on the image recognition algorithm includes:

[0131] The acquisition module 11 is used to acquire the image to be detected of the device panel, and perform image segmentation based on at least one signal detection area corresponding to the device panel to obtain a signal detection image corresponding to each signal detection area; the signal detection image includes at least one indicator and the coordinates corresponding to each indicator;

[0132] A detection module 12, configured to perform color detection on each of the signal detection images to determine a state color corresponding to the signal detection image;

[0133] The determination module 13 is used to determine whether the signal corresponding to the signal detection image is abnormal based on the abnormal strategy corresponding to the signal detection image and the state color.

[0134] In some embodiments, the acquisition module 11 is specifically used to:

[0135] Converting the image to be detected into a grayscale image;

[0136] Marking the grayscale image pixel by pixel to determine at least one connected region in the grayscale image,

[0137] Calculating the area of ​​each of the connected regions, and taking at least one of the connected regions that meets a preset area condition as the at least one signal detection region;

[0138] The image to be detected is segmented based on the at least one signal detection area.

[0139] In some embodiments, the acquisition module 11 is specifically used to:

[0140] Performing mask processing on the image to be detected based on each of the signal detection areas to obtain an edge contour corresponding to each of the signal detection areas;

[0141] The image to be detected is segmented based on the edge contour, and the coordinates corresponding to each of the signal detection images are obtained.

[0142] In some embodiments, the acquisition module 11 is specifically used to:

[0143] Inputting the image to be detected into an image semantic segmentation model for processing;

[0144] The image to be detected is segmented by the image semantic segmentation model, and the coordinates corresponding to each of the signal detection images are obtained.

[0145] In some embodiments, the detection module 12 is specifically configured to:

[0146] Obtaining the color ratio corresponding to each color component in each of the signal detection images;

[0147] The color component with the largest color ratio is used as the state color corresponding to the signal detection image.

[0148] In some embodiments, the detection module 12 is specifically configured to:

[0149] Performing image segmentation on the grayscale image based on the at least one signal detection area to obtain a grayscale detection image corresponding to the signal detection image;

[0150] For each of the grayscale detection images, calculate the color values ​​corresponding to the three color channels based on the grayscale values;

[0151] The channel color with the largest color value ratio is used as the status color corresponding to the signal detection image.

[0152] In some embodiments, the detection module 12 is specifically configured to:

[0153] Performing image enhancement on the grayscale image to obtain an enhanced image;

[0154] Obtaining a contrast evaluation index corresponding to at least one indicator color in the enhanced image;

[0155] When the contrast evaluation index corresponding to any indicator color is greater than a preset evaluation index, the grayscale image is corrected to obtain the corrected grayscale image.

[0156] In some embodiments, the detection module 12 is specifically configured to:

[0157] The contrast rating index is calculated using the following formula:

[0158]

[0159] Wherein, P is the contrast evaluation index corresponding to any indicator color in the enhanced image, S n is the grayscale average value of any indicator color in the enhanced image, T m is the grayscale average of other indicator colors in the enhanced image, S s is the standard grayscale value of the indicator color, x is the number of pixels corresponding to the indicator color in the enhanced image, y is the number of pixels corresponding to other indicator colors, K is S n and S s Same number of pixels.

[0160] In some embodiments, the determination module 13 is specifically configured to:

[0161] Using text recognition technology to obtain identification information corresponding to each of the signal detection areas;

[0162] Obtaining an exception policy corresponding to the identification information;

[0163] When the status color satisfies the abnormal condition in the abnormal strategy, it is determined that the signal corresponding to the signal detection area is abnormal.

[0164] Based on the coordinates corresponding to the indicator, acquiring the identification area of ​​the identification information corresponding to the indicator;

[0165] Perform text recognition on the identification area to obtain identification information corresponding to the indicator.

[0166] It should be understood that the units or modules described in the device panel signal detection device 10 based on the image recognition algorithm are similar to those described in the reference Figure 2 The various steps in the described method correspond to each other. Therefore, the operations and features described above for the method are also applicable to the device panel signal detection device 10 based on the image recognition algorithm and the units contained therein, and will not be repeated here. The device panel signal detection device 10 based on the image recognition algorithm can be pre-implemented in the browser or other security applications of the electronic device, and can also be loaded into the browser or its security application of the electronic device by downloading or the like. The corresponding units in the device panel signal detection device 10 based on the image recognition algorithm can cooperate with the units in the electronic device to implement the solution of the embodiment of the present application.

[0167] For the several modules or units mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into multiple modules or units to be embodied.

[0168] Reference below Figure 8 , Figure 8 A schematic diagram of the structure of a computer system of an electronic device or server suitable for implementing an embodiment of the present application is shown.

[0169] like Figure 8 As shown, the computer system includes a central processing unit (CPU) 801, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 802 or the program loaded from the storage part 808 into the random access memory (RAM) 803. In the RAM 803, various programs and data required for the operation instructions of the system are also stored. The CPU 801, the ROM 802 and the RAM 803 are connected to each other through a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0170] The following components are connected to the I / O interface 805: an input section 806 including a keyboard, a mouse, etc.; an output section 807 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 808 including a hard disk, etc.; and a communication section 809 including a network interface card such as a LAN card, a modem, etc. The communication section 809 performs communication processing via a network such as the Internet. A drive 810 is also connected to the I / O interface 805 as needed. A removable medium 811, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 810 as needed, so that a computer program read therefrom is installed into the storage section 808 as needed.

[0171] In particular, according to an embodiment of the present application, the above reference flow chart Figure 2 The described process can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a computer readable medium, and the computer program includes a program code for executing the method shown in the flow chart. In such an embodiment, the computer program includes a program code for executing the method shown in the flow chart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 809, and / or installed from the removable medium 811. When the computer program is executed by the central processing unit (CPU) 801, the above-mentioned functions defined in the system of the present application are executed.

[0172] It should be noted that the computer-readable medium shown in the present application may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium such as a computer-readable storage medium that can send, propagate or transmit a program for use by or in conjunction with an instruction execution system, device or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0173] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, functions and operating instructions of the system, method and computer program product according to various embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the aforementioned module, program segment or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the box can also occur in a sequence different from that marked in the accompanying drawings. For example, the boxes represented by two connections can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operating instruction, or can be implemented with a combination of dedicated hardware and computer instructions.

[0174] As another aspect, the present application further provides a computer-readable storage medium, which may be included in the electronic device described in the above embodiment, or may exist independently without being assembled into the electronic device. The above computer-readable storage medium stores one or more programs, and when the above programs are used by one or more processors to execute the device panel signal detection method based on the image recognition algorithm described in the present application.

[0175] The above description is only a preferred embodiment of the present application and an explanation of the technical principles used. Those skilled in the art should understand that the scope of disclosure involved in the present application is not limited to the technical solution formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the aforementioned disclosed concept. For example, the above features are replaced with the technical features with similar functions disclosed in this application (but not limited to) by each other to form a technical solution.

Claims

1. A device panel signal detection method based on image recognition algorithm, characterized in that: include: Acquire an image to be detected of a device panel, and perform image segmentation based on at least one signal detection area corresponding to the device panel to obtain a signal detection image corresponding to each signal detection area; The signal detection image includes at least one indicator and coordinates corresponding to each of the indicators; Performing color detection on each of the signal detection images to determine a state color corresponding to the signal detection image; Determining whether a signal corresponding to the signal detection image is abnormal based on an abnormality strategy corresponding to the signal detection image and the status color; The performing of image segmentation based on at least one signal detection area corresponding to the device panel includes: Converting the image to be detected into a grayscale image; Marking the grayscale image pixel by pixel to determine at least one connected region in the grayscale image, Calculating the area of ​​each of the connected regions, and taking at least one of the connected regions that meets a preset area condition as the at least one signal detection region; Performing image segmentation on the image to be detected based on the at least one signal detection area; Wherein, before performing image segmentation on the grayscale image based on the at least one signal detection area, the method further includes: Performing image enhancement on the grayscale image to obtain an enhanced image; Obtaining a contrast evaluation index corresponding to at least one indicator color in the enhanced image; When the contrast evaluation index corresponding to any indicator color is greater than a preset evaluation index, the grayscale image is corrected to obtain the corrected grayscale image.

2. The device panel signal detection method based on image recognition algorithm according to claim 1 is characterized in that: The performing image segmentation on the image to be detected based on the at least one signal detection area comprises: Performing mask processing on the image to be detected based on each of the signal detection areas to obtain an edge contour corresponding to each of the signal detection areas; The image to be detected is segmented based on the edge contour, and the coordinates corresponding to each of the signal detection images are obtained.

3. The device panel signal detection method based on image recognition algorithm according to claim 1, characterized in that: The performing image segmentation based on at least one signal detection area corresponding to the device panel includes: Inputting the image to be detected into an image semantic segmentation model for processing; The image to be detected is segmented by the image semantic segmentation model, and the coordinates corresponding to each of the signal detection images are obtained.

4. The device panel signal detection method based on image recognition algorithm according to claim 1, characterized in that: The performing color detection on each of the signal detection images to determine the state color corresponding to the signal detection image includes: Obtaining the color ratio corresponding to each color component in each of the signal detection images; The color component with the largest color ratio is used as the state color corresponding to the signal detection image.

5. The device panel signal detection method based on image recognition algorithm according to claim 1, characterized in that: The performing color detection on each of the signal detection images to determine the state color corresponding to the signal detection image includes: Performing image segmentation on the grayscale image based on the at least one signal detection area to obtain a grayscale detection image corresponding to the signal detection image; For each of the grayscale detection images, calculate the color values ​​corresponding to the three color channels based on the grayscale values; The channel color with the largest color value ratio is used as the status color corresponding to the signal detection image.

6. The device panel signal detection method based on image recognition algorithm according to claim 1, characterized in that: The contrast rating index is calculated using the following formula: Wherein, P is the contrast evaluation index corresponding to any indicator color in the enhanced image, S n is the grayscale average value of any indicator color in the enhanced image, T m is the grayscale average of other indicator colors in the enhanced image, S s is the standard grayscale value of the indicator color, x is the number of pixels corresponding to the indicator color in the enhanced image, y is the number of pixels corresponding to other indicator colors, K is S n and S s Same number of pixels.

7. The device panel signal detection method based on image recognition algorithm according to claim 1, characterized in that: The determining whether the signal corresponding to the signal detection area is abnormal based on the abnormal strategy corresponding to the signal detection area and the status color includes: Using text recognition technology to obtain identification information corresponding to each of the signal detection areas; Obtaining an exception policy corresponding to the identification information; When the status color satisfies the abnormal condition in the abnormal strategy, it is determined that the signal corresponding to the signal detection area is abnormal.

8. The device panel signal detection method based on image recognition algorithm according to claim 7 is characterized in that: The obtaining of identification information corresponding to each of the signal detection areas by using a text recognition technology also includes: Based on the coordinates corresponding to the indicator, acquiring the identification area of ​​the identification information corresponding to the indicator; Perform text recognition on the identification area to obtain identification information corresponding to the indicator.

Images