Integrated monitoring system for electronic control based on machine vision

Abstract

The present application relates to the field of image data processing technology, and in particular to an integrated monitoring system for electric control based on machine vision. It includes: an acquisition module, configured to acquire an initial image of an indication panel of an indication device; a first determination module, configured to determine the edge similarity between a first area and a second area in a search window of the initial image; a second determination module, configured to acquire an edge complexity value of the initial image to determine the regional similarity between the first area and the second area; a filtering module, configured to use the regional similarity to determine the weight value of the center of the second area relative to the target pixel point to filter the target pixel point to obtain a target image; a monitoring module, configured to monitor the operating status information according to the target image. Through the above technical solution, centralized monitoring of the operating status information of multiple electrical equipment to be monitored can be achieved more accurately.

Description

Technical Field

[0001] The present application relates to the technical field of image data processing, and in particular to an integrated monitoring system for electronic control based on machine vision. Background Art

[0002] Through the integrated electronic control indicating device, the operating status information of multiple electrical equipment in the factory can be indicated. For example, the indicating panel of the indicating device can be provided with indicating components such as indicator lights, indicating interfaces and instrument panels, so that the operating status of multiple electrical equipment in the factory can be indicated through these indicating components.

[0003] In order to achieve centralized monitoring of the operating status information of multiple electrical equipment in a factory without modifying or replacing the lines of existing indicating devices, thereby reducing the cost of centralized monitoring of the operating status information of multiple electrical equipment in a factory, the indication interface of the indicating device can be identified by an image recognition device to monitor the operating status of the electrical equipment indicated by the indicating interface.

[0004] For example, in a Chinese patent application document with publication number CN113345036A, a method for identifying indicator light status based on HSV feature transformation is provided, including: obtaining an image of the indicator light area to be identified, using perspective transformation technology to perform distortion correction on the image to obtain a corrected image; segmenting the corrected image based on a single indicator light as a unit, and segmenting the complete indicator light area into several single indicator light areas; using the S component clustering method of the HSV space to locate the indicator light within the single indicator light area; and identifying the indicator light status based on the HSV threshold combined with the proportion and distribution of each color component.

[0005] However, when identifying the indication status of the indication area in the image of the indication area, since there may be noise data points in the acquired image, the success rate of identifying the indication status is low, making it difficult to accurately achieve centralized monitoring of the operating status information of multiple electrical equipment. Summary of the invention

[0006] In order to overcome the problem in the related art that it is difficult to accurately realize centralized monitoring of the operating status information of multiple electrical devices, the present application provides an integrated monitoring system for electric control integration based on machine vision, including:

[0007] An acquisition module is configured to acquire an initial image of an indication panel of an indicating device, wherein the indication panel is used to indicate operating status information of multiple electrical devices to be monitored; a first determination module is configured to determine a search window centered on a target pixel in the initial image, and determine the edge similarity between a first area and a second area; the first area is a neighborhood centered on the target pixel, and the second area is a neighborhood centered on other pixels in the search window; a second determination module is configured to acquire an edge complexity value of the initial image, and determine the regional similarity between the first area and the second area based on the edge similarity and the edge complexity value between the first area and the second area; the edge complexity value is used to characterize the complexity of the edge line in the initial image; a filtering module is configured to determine a weight value of the center of the second area relative to the target pixel using the regional similarity between the first area and the second area, and filter the target pixel according to the weight value of the pixel in the search window to obtain a target image after filtering the initial image; a monitoring module is configured to monitor the operating status information based on the target image.

[0008] In this way, by acquiring an initial image of an indication panel indicating the operating status information of multiple electrical devices to be monitored, and determining the edge similarity between the area where the target pixel point is located and other areas in the search window centered on the target pixel point in the initial image, combined with the overall edge complexity value of the initial image, the edge similarity between the area where the target pixel point is located and other areas is determined to achieve targeted filtering processing of the target pixel point to obtain the target image. Since the situation of the pixel points corresponding to the operating status information in the initial image is more complicated, the pixel features of the pixel points corresponding to the operating status information can be retained in the obtained target image, and the noise pixels in the initial image can be filtered out. Through the target image, centralized monitoring of the operating status information of multiple electrical devices to be monitored can be more accurately achieved.

[0009] At the same time, by performing image acquisition on the indication panel indicating the operating status information of multiple electrical devices to be monitored, the positions of different information indicated in the indication panel of the indicating device are relatively fixed. The acquired images can realize centralized monitoring of the operating status information of multiple electrical devices to be monitored, and there is no need to modify the circuit structure of the indicating device. While making full use of the indicating device, centralized and remote monitoring of the operating status information of the electrical equipment indicated by the indicating device can be realized, thereby reducing the cost and difficulty of monitoring the operating status information of the electrical equipment.

[0010] Optionally, the edge similarity between the first region and the second region is determined in the following manner: the gradient values ​​of the pixel points in the first region are expanded into a one-dimensional vector in a specified order to obtain the expanded first vector; the gradient values ​​of the pixel points in the second region are expanded into a one-dimensional vector in a specified order to obtain the expanded second vector; and the cosine similarity between the first vector and the second vector is used as the edge similarity between the first region and the second region.

[0011] In this way, the gradient value of the pixel point in the first area is expanded into a one-dimensional vector, and the gradient value of the pixel point in the second area is expanded into a one-dimensional vector. Since the one-dimensional vectors corresponding to the areas with different edge features are different, the edge similarity between the first area and the second area can be obtained by determining the cosine similarity between the two one-dimensional vectors, so as to better characterize the similarity between the first area and the second area on the edge through the edge similarity.

[0012] Optionally, the edge similarity between the first region and the second region is determined in the following manner: 1 is used as the characteristic value of the pixel points in the target region whose gradient value is greater than a preset gradient threshold, and 0 is used as the characteristic value of the pixel points in the target region whose gradient value is less than or equal to the preset gradient threshold; the target region is any one of the first region and the second region; the characteristic values ​​of the pixel points in the first region are expanded into a one-dimensional vector in a specified order to obtain an expanded third vector; and the characteristic values ​​of the pixel points in the second region are expanded into a one-dimensional vector in a specified order to obtain an expanded fourth vector; and the cosine similarity between the third vector and the fourth vector is used as the edge similarity between the first region and the second region.

[0013] In this way, by using 1 as the feature value of the pixel points in the target area whose gradient values ​​are greater than the preset gradient threshold, and using 0 as the feature value of the pixel points in the target area whose gradient values ​​are less than or equal to the preset gradient threshold, when comparing the third vector corresponding to the first area and the fourth vector corresponding to the second area, it is possible to more easily compare the edge similarity between the two image areas while retaining the edge features of the image areas in the vectors.

[0014] Optionally, the edge complexity value is determined in the following manner: , E is the edge complexity value of the initial image, N is the number of edge lines in the edge image of the initial image, is the normalization function, is the number of pixels in the kth edge line in the edge image of the initial image, is an exponential function with a natural constant as base, is the maximum curvature of the pixel points in the kth edge line in the edge image of the initial image.

[0015] In this way, the more pixel points there are in the edge line of the edge image of the initial image and the larger the maximum curvature of the pixel points in the edge line of the edge image of the initial image, the higher the complexity of the edge line of the indicating component of the indicating panel corresponding to the initial image, and the obtained edge complexity value can better characterize the complexity of the edge line of the indicating component of the indicating panel.

[0016] Optionally, the edge complexity value is determined in the following manner: , E is the edge complexity value of the initial image, N is the number of edge lines in the edge image of the initial image, is the normalization function, is the number of pixels in the kth edge line in the edge image of the initial image, W is the width of the initial image, H is the height of the initial image, is the variance of the horizontal coordinates of the edge pixels in the edge image of the initial image in the image coordinate system, is the variance of the ordinate of the edge pixel points in the edge image of the initial image in the image coordinate system.

[0017] Optionally, determining the region similarity between the first region and the second region according to the edge similarity and the edge complexity value between the first region and the second region includes: , is the region similarity between the first region and the i-th second region in the search window, is the edge similarity between the first region and the i-th second region in the search window, and E is the edge complexity value of the initial image.

[0018] In this way, since the edge complexity value of the initial image reflects the overall complexity of the edge line in the initial image, the edge complexity value is used to expand the ability to distinguish the regional similarity between the first area and the second area, which can better achieve the distinction between the two image areas in the initial image with more complex edges, so as to filter the target pixel points in combination with the similarity between the areas.

[0019] Optionally, using the regional similarity between the first region and the second region to determine the weight value of the center of the second region relative to the target pixel point includes: when the regional similarity between the first region and the second region is greater than or equal to a preset similarity threshold, , is the weight value of the center of the i-th second area in the search window relative to the target pixel point, is the region similarity between the first region and the i-th second region in the search window, is an exponential function with a natural constant as the base, K is the side length of the search window, is the mean square error of the pixel values ​​of the pixel points at the corresponding positions in the first area and the i-th second area in the search window.

[0020] Optionally, the regional similarity between the first area and the second area is used to determine the weight value of the center of the second area relative to the target pixel point, including: when the regional similarity between the first area and the second area is less than a preset similarity threshold, 0 is used as the weight value of the center of the second area relative to the target pixel point.

[0021] Optionally, the target pixel point is filtered according to the weight value of the pixel point in the search window, including: performing weighted summation on the pixel values ​​of the pixel points in the search window according to the weight value of the pixel points in the search window, and using the result of the weighted summation as the pixel value after filtering the target pixel point.

[0022] Optionally, the integrated electronic control monitoring system based on machine vision also includes a prompt module, which is configured to perform the following steps: when it is determined based on the target image that there is an abnormality in a target electrical device among the multiple electrical devices to be monitored indicated by the indicating device, output a prompt message; the prompt message is used to prompt that there is an abnormality in the target electrical device.

[0023] The technical solution provided by the embodiments of the present application may include the following beneficial effects: by acquiring an initial image of an indication panel indicating operating status information of multiple electrical devices to be monitored, and determining the edge similarity between the area where the target pixel point is located and other areas in a search window centered on the target pixel point in the initial image, combined with the overall edge complexity value of the initial image, determining the edge similarity between the area where the target pixel point is located and other areas, so as to achieve targeted filtering processing of the target pixel point to obtain a target image; since the situation of the pixel points corresponding to the operating status information in the initial image is more complicated, it is possible to take into account the retention of the pixel features of the pixel points corresponding to the operating status information and the filtering of the noise pixels in the initial image in the obtained target image; through the target image, it is possible to more accurately achieve centralized monitoring of the operating status information of multiple electrical devices to be monitored.

[0024] At the same time, by performing image acquisition on the indication panel indicating the operating status information of multiple electrical devices to be monitored, the positions of different information indicated in the indication panel of the indicating device are relatively fixed. The acquired images can realize centralized monitoring of the operating status information of multiple electrical devices to be monitored, and there is no need to modify the circuit structure of the indicating device. While making full use of the indicating device, centralized and remote monitoring of the operating status information of the electrical equipment indicated by the indicating device can be realized, thereby reducing the cost and difficulty of monitoring the operating status information of the electrical equipment.

[0025] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0027] Figure 1 is a schematic diagram of an initial image of an indication panel of an indication device in an embodiment of the present application;

[0028] Figure 2 is a structural schematic diagram of an electric control integrated monitoring system based on machine vision according to an exemplary embodiment;

[0029] Figure 3 It is a structural schematic diagram of another electronic control integrated monitoring system based on machine vision according to an exemplary embodiment. DETAILED DESCRIPTION

[0030] Here, exemplary embodiments will be described in detail, and examples thereof are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application.

[0031] First, the application scenario of the embodiment of the present application is briefly introduced. In the application scenario of the present application, Figure 1 is a schematic diagram of an initial image of an indication panel of an indication device, such as Figure 1 As shown, the indication panel of the indicating device indicates the operating status information of multiple electrical devices to be monitored. For example, the indicating panel can be provided with indicator lights of different colors corresponding to different operating states, or the indicating panel can be provided with a dashboard indicating different operating parameters to realize the indication of the operating status information of multiple monitored electrical devices.

[0032] In the related art, the status of the indicator light is mainly obtained by identifying the image of the indicator light area. However, there may be noise pixels in the image obtained from the indicator panel of the indicating device, which affects the recognition of the information indicated by the indicator panel, making it difficult to accurately realize centralized monitoring of the operating status information of multiple electrical devices.

[0033] In view of the above technical problems, the present application provides an integrated monitoring system based on machine vision and electronic control. Figure 2 is a schematic diagram of a structure of an electric control integrated monitoring system based on machine vision according to an exemplary embodiment, such as Figure 2As shown, the electronic control integrated monitoring system 1000 based on machine vision includes:

[0034] The acquisition module 1100 is configured to acquire an initial image of an indication panel of an indication device, where the indication panel is used to indicate operating status information of a plurality of electrical devices to be monitored.

[0035] The acquisition module can acquire the initial image of the indication panel of the indication device through the image acquisition device, for example, it can acquire the initial image of the indication panel of the indication device in the front view, so as to acquire the information indicated by the indication panel.

[0036] The first determination module 1200 is configured to determine a search window centered on a target pixel in an initial image, and determine edge similarity between a first region and a second region; the first region is a neighborhood centered on the target pixel, and the second region is a neighborhood centered on other pixels in the search window.

[0037] The target pixel point can be any pixel point in the initial image, and a search window with the target pixel point as the center can be determined in the initial image. The side length of the search window can be set according to actual needs. For example, the search window can be a window of size 11×11, 13×13, and 15×15. The embodiment of the present application does not impose any limitation on the side length of the search window.

[0038] The first area is a neighborhood centered on the target pixel point. For example, the first area can be a 3×3 pixel area centered on the target pixel point. The neighborhood of the pixel point can provide a good reference for the target pixel point. The second area is a neighborhood centered on other pixels in the search window. The center of the second area can be any other pixel point in the search window except the target pixel point. Different second areas correspond to different other pixels in the search window except the target pixel point.

[0039] Compared with other areas except the indicating component in the initial image of the indicating panel, the edges of the indicating component in the initial image of the indicating panel are more complex, so that the areas with more complex edges in the initial image are more likely to correspond to the indicating component in the indicating panel.

[0040] like Figure 1 As shown, other regions except the indicating component in the initial image of the indicating panel have certain similarities, and the region where the indicating component is located in the initial image of the indicating panel is more complex, and the similarity between the region where the indicating component is located in the initial image of the indicating panel and other surrounding regions is lower.

[0041] By determining the edge similarity between the first area and the second area, it is possible to compare the complexity of the neighborhood of the target pixel point with the neighborhood of other pixel points in the same search range, so as to better retain the information indicated by the indicating component in the initial image of the indicating panel and filter out noise pixels in the initial image of the indicating panel.

[0042] The second determination module 1300 is configured to obtain an edge complexity value of the initial image, and determine the region similarity between the first region and the second region based on the edge similarity between the first region and the second region and the edge complexity value; the edge complexity value is used to characterize the complexity of the edge line in the initial image.

[0043] The edge complexity value of the initial image can characterize the overall complexity of the edge lines in the initial image by characterizing the complexity of the edge lines in the initial image; the larger the edge complexity value of the initial image, the higher the overall complexity of the edge lines in the initial image, and the more likely the initial image contains a larger number or more diverse indicating components; on the contrary, the smaller the edge complexity value of the initial image, the lower the overall complexity of the edge lines in the initial image, and the more likely the initial image contains a smaller number or more single indicating components.

[0044] By determining the regional similarity between the first region and the second region based on the edge similarity and edge complexity value between the first region and the second region, the overall complexity of the edge line in the initial image can be combined, so that the regional similarity between the first region and the second region can reflect the overall complexity of the edge line in the initial image, so as to better realize the filtering processing of the target pixel points by utilizing the regional similarity between the first region and different second regions.

[0045] The filtering module 1400 is configured to use the regional similarity between the first area and the second area to determine the weight value of the center of the second area relative to the target pixel point, and filter the target pixel point according to the weight value of the pixel point in the search window to obtain the target image after filtering the initial image.

[0046] In an embodiment of the present application, the regional similarity between the first region and the second region is positively correlated with the weight value of the center of the second region relative to the target pixel point; the greater the regional similarity between the first region and the second region, the greater the weight value of the center of the second region relative to the target pixel point; conversely, the smaller the regional similarity between the first region and the second region, the smaller the weight value of the center of the second region relative to the target pixel point.

[0047] By setting a larger weight value for the center of the second area having greater regional similarity to the first area, the contribution of other areas with similar performance within the same search window to the filtering processing of the target pixel points can be increased, thereby more accurately filtering the target pixel points and enabling the filtered image to better retain the details of the information indicated by the indicating component.

[0048] The monitoring module 1500 is configured to monitor the operation status information according to the target image.

[0049] In the initial image of the indication panel of the indicating device, different positions correspond to different electrical equipment to be monitored indicated by the indicating device, or different positions correspond to different operating status information of the same electrical equipment to be monitored indicated by the indicating device. Therefore, the operating status information of multiple electrical equipment to be monitored can be determined through the correspondence between the position of the pixel point in the initial image and the operating status information of the electrical equipment to be monitored.

[0050] The corresponding relationship between the positions of the pixels in the initial image and the operating status information of the electrical equipment to be monitored can be obtained by pre-calibration.

[0051] For example, the first indicator light located in the upper left corner of the initial image can be used to indicate the first operating status information of the first electrical equipment to be monitored; different operating status information can be, for example, at least one of the ambient temperature information, ambient humidity information, operating voltage information, operating frequency information and operating current information of the electrical equipment to be monitored. The embodiments of the present application do not limit the specific type of operating status information.

[0052] The target image is obtained by filtering the pixel points in the initial image. The correspondence between different position points in the target image and the operating status information of multiple electrical equipment to be monitored has not changed. After filtering the pixel points in the initial image, the information indicated by the indicating component located on the indicating panel in the target image is clearer, and the operating status information of the electrical equipment to be monitored indicated by the indicating panel can be determined more accurately based on the target image.

[0053] Through the integrated electric control monitoring system based on machine vision provided by the embodiment of the present application, an initial image of an indication panel indicating the operating status information of multiple electrical devices to be monitored is obtained, and the edge similarity between the area where the target pixel point is located and other areas in the search window centered on the target pixel point in the initial image is determined. Combined with the overall edge complexity value of the initial image, the edge similarity between the area where the target pixel point is located and other areas is determined to achieve targeted filtering processing of the target pixel point to obtain the target image. Since the situation of the pixel points corresponding to the operating status information in the initial image is more complicated, the pixel features of the pixel points corresponding to the operating status information can be retained in the obtained target image, and the noise pixels in the initial image can be filtered out. Through the target image, centralized monitoring of the operating status information of multiple electrical devices to be monitored can be more accurately achieved.

[0054] At the same time, by performing image acquisition on the indication panel indicating the operating status information of multiple electrical devices to be monitored, the positions of different information indicated in the indication panel of the indicating device are relatively fixed. The acquired images can realize centralized monitoring of the operating status information of multiple electrical devices to be monitored, and there is no need to modify the circuit structure of the indicating device. While making full use of the indicating device, centralized and remote monitoring of the operating status information of the electrical equipment indicated by the indicating device can be realized, thereby reducing the cost and difficulty of monitoring the operating status information of the electrical equipment.

[0055] In one implementation, the edge similarity between the first region and the second region is determined by: expanding the gradient values ​​of the pixel points in the first region into a one-dimensional vector in a specified order to obtain the expanded first vector; expanding the gradient values ​​of the pixel points in the second region into a one-dimensional vector in a specified order to obtain the expanded second vector; and using the cosine similarity between the first vector and the second vector as the edge similarity between the first region and the second region.

[0056] The process of determining the edge similarity between the first region and the second region can be executed by the first determination module 1200; since the gradient values ​​of pixel points in regions with different edge information are expressed differently, by comparing the gradient values ​​of pixel points in the two regions, the detailed information of the edges of the two regions can be better compared, so as to determine the similarity information of the two regions on the edges.

[0057] The gradient value of the pixel point in the initial image can be determined by operators such as the Sobel operator, the Prewitt operator, the Canny operator, and the Laplacian operator.

[0058] The gradient values ​​of the pixel points in the first area are expanded into a one-dimensional vector in a specified order, for example, it can be performed in the order of rows first and then columns. For example, the specified order can be to start from the gradient values ​​of the pixel points located in the first row and first column of the first area, and arrange the gradient values ​​of the pixel points located in the first row and second column of the first area in sequence to complete the arrangement of the gradient values ​​of all the pixel points in the first row of the first area. When the arrangement of the gradient values ​​of the pixel points in the previous row of the first area is completed, the gradient values ​​of the pixel points in the next row of the first area are arranged.

[0059] When the size of the neighborhood is 3×3, the first area includes 9 pixels. Therefore, after arranging the gradient values ​​of the pixels in the first area, a one-dimensional vector with a length of 9 can be obtained; referring to the step of arranging the gradient values ​​of the pixels in the first area to obtain the one-dimensional first vector, a one-dimensional second vector obtained after arranging the gradient values ​​of the pixels in the second area can be obtained, so that the cosine similarity between the first vector and the second vector is used as the edge similarity between the first area and the second area.

[0060] In this way, the gradient value of the pixel point in the first area is expanded into a one-dimensional vector, and the gradient value of the pixel point in the second area is expanded into a one-dimensional vector. Since the one-dimensional vectors corresponding to the areas with different edge features are different, the edge similarity between the first area and the second area can be obtained by determining the cosine similarity between the two one-dimensional vectors, so as to better characterize the similarity between the first area and the second area on the edge through the edge similarity.

[0061] In one implementation, the edge similarity between the first region and the second region is determined in the following manner: 1 is used as the feature value of the pixel points in the target region whose gradient value is greater than a preset gradient threshold, and 0 is used as the feature value of the pixel points in the target region whose gradient value is less than or equal to the preset gradient threshold; the target region is any one of the first region and the second region; the feature values ​​of the pixel points in the first region are expanded into a one-dimensional vector in a specified order to obtain an expanded third vector; and the feature values ​​of the pixel points in the second region are expanded into a one-dimensional vector in a specified order to obtain an expanded fourth vector; and the cosine similarity between the third vector and the fourth vector is used as the edge similarity between the first region and the second region.

[0062] The process of determining the edge similarity between the first area and the second area can be executed by the first determination module 1200; the pixel points located at the edge of the image mainly show a large gradient value. When the gradient value of the pixel point is greater than the preset gradient threshold, it means that the pixel point is an edge pixel point in the image. The preset gradient threshold can be set according to actual needs. For example, the preset gradient threshold can be between 20 and 30.

[0063] When the gradient value of a pixel point is less than or equal to a preset gradient threshold, it indicates that the pixel point is a non-edge pixel point in the image. By setting different eigenvalues ​​for edge pixels and non-edge pixels in the image area, the number of non-zero elements contained in the two one-dimensional vectors corresponding to the two image areas is smaller, which facilitates comparison of the two one-dimensional vectors corresponding to the two image areas.

[0064] In this way, by using 1 as the feature value of the pixel points in the target area whose gradient values ​​are greater than the preset gradient threshold, and using 0 as the feature value of the pixel points in the target area whose gradient values ​​are less than or equal to the preset gradient threshold, when comparing the third vector corresponding to the first area and the fourth vector corresponding to the second area, it is possible to more easily compare the edge similarity between the two image areas while retaining the edge features of the image areas in the vectors.

[0065] In one embodiment, the edge complexity value may be determined in the following manner: , E is the edge complexity value of the initial image, N is the number of edge lines in the edge image of the initial image, is the normalization function, is the number of pixels in the kth edge line in the edge image of the initial image, is an exponential function with a natural constant as base, is the maximum curvature of the pixel points in the kth edge line in the edge image of the initial image.

[0066] The process of determining the edge complexity value of the initial image can be executed by the second determination module 1300; the more pixels in the edge line in the edge image of the initial image, the longer the length of the single edge line; for example, the more diverse the elements such as indicator lights, text, patterns, etc. on the indicator panel used to indicate the operating status information.

[0067] The maximum curvature of the pixel points in the edge line in the edge image of the initial image can reflect the complexity of the single edge line in the initial image; the larger the maximum curvature of the pixel points in the edge line in the edge image of the initial image, the more diverse the direction changes of different edge pixel points on the same edge line are compared with the edge line with a straight line shape.

[0068] In the indication panel of the indication device, the edge lines of the area where the indicator lights of the indication panel are located are more complex than the blank area of ​​the indication panel. It is a value normalized to the range of 0 to 1. The larger the exponential term of the value in the range of 0 to 1, the smaller the value of the exponential function.

[0069] Here, the greater the maximum curvature of the pixel points in the edge line in the edge image, the more diverse the direction of the edge line changes. The smaller the value of is, the smaller the The larger the edge complexity value obtained by exponential operation, the more pixels in the edge line of the edge image or the maximum curvature of the pixels in the edge line of the edge image, the larger the edge complexity value of the initial image. The edge complexity value of the initial image can better characterize the complexity of the edge line in the initial image.

[0070] By normalizing the maximum curvature of the pixel points in the edge line, the normalized maximum curvature is in the range of 0 to 1, and the inverse of the exponential function of the normalized maximum curvature is used in the calculation of the edge complexity value. This can expand the contrast of the edge complexity values ​​of edge lines with different maximum curvatures, so that the complexity of the edge line of the corresponding indicating component itself in the initial image can be better characterized by the edge complexity value of the initial image.

[0071] In this way, the more pixel points there are in the edge line of the edge image of the initial image and the larger the maximum curvature of the pixel points in the edge line of the edge image of the initial image, the higher the complexity of the edge line of the indicating component of the indicating panel corresponding to the initial image, and the obtained edge complexity value can better characterize the complexity of the edge line of the indicating component of the indicating panel.

[0072] In another embodiment, the edge complexity value may be determined in the following manner: , E is the edge complexity value of the initial image, N is the number of edge lines in the edge image of the initial image, is the normalization function, is the number of pixels in the kth edge line in the edge image of the initial image, W is the width of the initial image, H is the height of the initial image, is the variance of the horizontal coordinates of the edge pixels in the edge image of the initial image in the image coordinate system, is the variance of the ordinate of the edge pixel points in the edge image of the initial image in the image coordinate system.

[0073] The process of determining the edge complexity value of the initial image can be executed by the second determination module 1300; compared with other parts of the indication panel of the indicating device except the indicating component, such as the blank part of the indication panel, the edge line of the indicating component in the indication panel of the indicating device is more complex, which can be specifically manifested as the edge lines being distributed at different positions in the initial image; in the image coordinate system of the edge image of the initial image, the changes in the horizontal or vertical coordinates of the edge pixel points in the image coordinate system are more diverse; or, the number of pixel points on the edge line is greater.

[0074] In this way, by combining the variance of the vertical coordinates of the edge pixels in the edge image of the initial image in the image coordinate system, the variance of the horizontal coordinates of the edge pixels in the image coordinate system, and the number of pixels on the edge line, the complexity of the edge line in the initial image can be better characterized by the edge complexity value.

[0075] In one embodiment, determining the region similarity between the first region and the second region according to the edge similarity and the edge complexity value between the first region and the second region includes: , is the region similarity between the first region and the i-th second region in the search window, is the edge similarity between the first region and the i-th second region in the search window, and E is the edge complexity value of the initial image.

[0076] The process of determining the regional similarity between the first region and the second region can be executed by the second determination module 1300; by adding 0.5 to the edge complexity value of the initial image as an exponential term, the influence of the edge complexity value on the regional similarity can be expanded, and the ability to distinguish the similarities between different regions can be improved, so as to achieve targeted filtering processing of different pixel points in the initial image.

[0077] Since the edge similarity is equal to the cosine similarity between two vectors, the edge similarity between the first area and the second area of ​​the search window is added to 1 to obtain the sum result, and the sum result is divided by 2. The value of the edge similarity can be normalized to the range of 0 to 1, so as to compare the regional similarities between different areas.

[0078] In this way, since the edge complexity value of the initial image reflects the overall complexity of the edge line in the initial image, the edge complexity value is used to expand the ability to distinguish the regional similarity between the first area and the second area, which can better achieve the distinction between the two image areas in the initial image with more complex edges, so as to filter the target pixel points in combination with the similarity between the areas.

[0079] In one embodiment, using the regional similarity between the first region and the second region to determine the weight value of the center of the second region relative to the target pixel point includes: when the regional similarity between the first region and the second region is greater than or equal to a preset similarity threshold, , is the weight value of the center of the i-th second area in the search window relative to the target pixel point, is the region similarity between the first region and the i-th second region in the search window, is an exponential function with a natural constant as the base, K is the side length of the search window, is the mean square error of the pixel values ​​of the pixel points at the corresponding positions in the first area and the i-th second area in the search window.

[0080] The process of determining the weight value of the center of the second area relative to the target pixel point can be executed by the filtering module 1400; when the regional similarity between the first area and the second area is greater than or equal to the preset similarity threshold, it means that the neighborhood of the target pixel point is relatively similar to the neighborhoods of other pixel points in the search window, and the first area and the second area may both be blank areas of the indication panel; or, the first area and the second area may be areas corresponding to the same type of indicator lights in the indication panel; or, the first area and the second area may include the same partial area of ​​the same type of indicator lights in the indication panel.

[0081] The second area can provide a good reference for filtering out noise pixels in the first area, and the center of the first area is the target pixel, so that the second area can provide a good reference for filtering the target pixel.

[0082] By using the mean square error of the pixel values ​​of the pixel points at corresponding positions in the first area and the second area of ​​the search window, further comparison between the pixel values ​​of the pixel points in the first area and the second area can be achieved. The comparison process of the mean square error is performed when the regional similarity between the first area and the second area is greater than or equal to a preset similarity threshold, and is performed when the regional similarity between the first area and the second area is less than the preset similarity threshold, which can reduce the amount of calculation.

[0083] The mean square error of the pixel values ​​of the pixel points at the corresponding positions in the first area and the second area of ​​the search window may refer to the average of the difference between the two pixel points at the same position in the first area and the second area. For example, the square of the difference between the two pixel points located in the first row and the first column in the first area and the second area is determined respectively, and the sum of the squares of the differences corresponding to different positions is determined, and the ratio of the sum to the number of pixel points in the neighborhood is used as the mean square error of the pixel values ​​of the pixel points at the corresponding positions in the first area and the second area of ​​the search window.

[0084] The larger the mean square error of the pixel values ​​of the pixel points at the corresponding positions in the first area and the second area within the search window, the greater the difference in pixel values ​​between the first area and the second area, and the less similar the pixel values ​​of the pixel points in the first area and the second area are; conversely, the smaller the mean square error of the pixel values ​​of the pixel points at the corresponding positions in the first area and the second area within the search window, the smaller the difference in pixel values ​​between the first area and the second area, and the more similar the pixel values ​​of the pixel points in the first area and the second area are.

[0085] By using the denominator in the calculation formula for the weight value of the center of the second area relative to the target pixel point, the numerator in the calculation formula for the weight value of the center of the second area relative to the target pixel point can be normalized so that the obtained weight value is between 0 and 1.

[0086] In this way, when the regional similarity between the first area and the second area is greater than or equal to the preset similarity threshold, the pixel values ​​of the pixel points in the first area and the second area can be further compared, so as to determine the weight value of the second area relative to the target pixel point based on the mean square error of the pixel values ​​of the two areas, so as to better realize the filtering processing of the target pixel point.

[0087] In one embodiment, the regional similarity between the first area and the second area is used to determine the weight value of the center of the second area relative to the target pixel point, including: when the regional similarity between the first area and the second area is less than a preset similarity threshold, 0 is used as the weight value of the center of the second area relative to the target pixel point.

[0088] The process of determining the weight value of the center of the second area relative to the target pixel point can be executed by the filtering module 1400; when the regional similarity between the first area and the second area is less than the preset similarity threshold, it means that the similarity between the first area and the second area is low, and the information included in the first area and the second area is likely to be completely different information.

[0089] For example, the first area is a blank area in the indication panel, and the second area is the area where the indicator light in the indication panel is located; or, for example, the first area is the area where the indicator light in the indication panel is located, and the second area is a blank area in the indication panel; the second area has a lower reference value for filtering the first area.

[0090] In order to reduce the contribution of the second area to the filtering of the first area, 0 can be used as the weight value of the center of the second area relative to the target pixel point. Directly using 0 as the weight value of the center of the second area relative to the target pixel point also avoids further comparison of the pixels in the first area and the second area, which can reduce the amount of calculation in the weight value calculation process.

[0091] The preset similarity threshold can be set according to actual needs. When the value range of the region similarity is between 0 and 1, the preset similarity threshold can be between 0.3 and 0.4.

[0092] In this way, when the regional similarity between the first area and the second area is less than the preset similarity threshold, the weight value of the center of the second area relative to the target pixel point can be determined more easily, which can not only reduce the weight value corresponding to the second area with lower similarity to the first area, but also reduce the amount of calculation of the weight value corresponding to the second area.

[0093] In one embodiment, the target pixel point is filtered according to the weight value of the pixel point in the search window, including: according to the weight value of the pixel point in the search window, the pixel value of the pixel point in the search window is weighted summed, and the result of the weighted summation is used as the pixel value after filtering the target pixel point.

[0094] The process of filtering the target pixel point according to the weight value of the pixel point in the search window can be executed by the filtering module 1400; the larger the weight value of other pixel points except the target pixel point in the search window, the more consistent the pixel conditions of other pixel points are with those of the target pixel point in the neighborhood, and the greater the reference value of other pixel points to the pixel value of the target pixel point after filtering.

[0095] By performing a weighted summation on the pixel values ​​of the pixel points in the search window according to their weight values, the contribution of the pixel values ​​of other pixel points with higher weight values ​​in the search window to the pixel value of the target pixel point after filtering can be increased, and the contribution of the pixel values ​​of other pixel points with lower weight values ​​in the search window to the pixel value of the target pixel point after filtering can be reduced.

[0096] When there is noise in the initial image, a smaller similarity value will be assigned between the second area where the noise pixel is located and the first area where the target pixel is located, so that the center of the second area where the noise pixel is located has a smaller weight value relative to the first area.

[0097] When performing weighted summation of the pixel values ​​of other pixels except the target pixel within the search range, the pixel value of the noise pixel has little influence on the weighted summation result, or when the weight value corresponding to the noise pixel is 0, the adverse effect of the pixel value of the noise pixel on the weighted summation result can be avoided. Therefore, by performing weighted summation of the pixel values ​​of the pixels in the search window according to the weight values ​​of the pixels in the search window, the pixel value of the target pixel after filtering can exclude the influence of the existing noise pixels.

[0098] Since the size of the search window is larger than the neighborhood range of the pixel point, compared with filtering the target pixel point through the neighborhood of the target pixel point, it is more likely to find other neighborhoods with similar pixel conditions to the neighborhood of the target pixel point within the search window. Therefore, the pixel value of the target pixel point after filtering can be more accurate, and the pixel value of the pixel point corresponding to the indicator panel in the initial image can be more accurate after filtering, so that the monitoring of the operating status information of the monitored electrical equipment can be achieved more accurately through the target image.

[0099] In this way, since the weight value of the pixel points in the search window is determined based on the comparison between the first area corresponding to the target pixel point and the second area corresponding to other pixel points in the search window, the pixel values ​​of the pixel points in the search window are weightedly summed according to the weight values ​​of the pixel points in the search window, and the result of the weighted summation is used as the pixel value after filtering of the target pixel point. This enables the pixel value of the target pixel point after filtering to take into account both the filtering of noise pixels and the retention of pixels corresponding to the running status information in the indication panel.

[0100] In one embodiment, Figure 3 As shown, the integrated electrical control monitoring system 1000 based on machine vision also includes a prompt module 1600, which is configured to perform the following steps: when it is determined based on the target image that a target electrical device among the multiple electrical devices to be monitored indicated by the indicating device has an abnormality, output a prompt message; the prompt message is used to prompt that the target electrical device has an abnormality.

[0101] Since different positions of the indication panel correspond to different operating status information indicated by the indication panel, the different operating status information indicated by the indication panel can be determined according to the target image, thereby realizing centralized monitoring of the operating status information of multiple electrical devices to be monitored.

[0102] For the target electrical equipment among the multiple electrical equipment to be monitored indicated by the indicating device, the indicating panel of the indicating device will indicate the operating status information of the target electrical equipment at the corresponding position. According to the image area corresponding to the position of the target electrical equipment in the target image, the operating status information of the target electrical equipment can be obtained to determine whether there is any abnormality in the target electrical equipment, thereby outputting prompt information when there is any abnormality in the target electrical equipment.

[0103] In the event that an abnormality occurs in the target electrical equipment, the indicator light at the corresponding position in the indicator panel may flash, or the warning light at the corresponding position in the indicator panel may be constantly lit in warning colors such as red, yellow, and orange, or the numbers at the corresponding position in the indicator panel may exceed the range allowed by the target electrical equipment. Therefore, it is possible to determine whether an abnormality occurs in the target electrical equipment based on the target image.

[0104] The prompt information may be, for example, text prompt information, sound prompt information, vibration prompt information, and at least one of vibration prompt information. The prompt module 1600 may output the prompt information through components such as a display, a speaker, a vibrator, and a light emitting mechanism. Alternatively, the machine vision-based integrated electrical control monitoring system 1000 may send prompt information to a terminal device pre-bound by the user to prompt the user that there is an abnormality in the target electrical equipment.

[0105] It should be understood that the features of some embodiments of the various applications described herein may be combined with each other, unless specifically stated otherwise.

[0106] Although terms such as "first", "second", and "third" may be used herein to describe various components, parts, regions, layers, or sections, these components, parts, regions, layers, or sections are not limited to these terms. Instead, these terms are only used to distinguish one component, part, region, layer, or section from another component, part, region, layer, or section. Therefore, without departing from the teachings of the examples described herein, the first component, part, region, layer, or section mentioned in the examples may also be referred to as the second component, part, region, layer, or section.

[0107] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description herein, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0108] In addition, the word "exemplary" is used herein to mean serving as an example, instance, or diagram. Any aspect or design described herein as "exemplary" is not necessarily to be understood as being advantageous over other aspects or designs. Instead, the use of the word exemplary is intended to present concepts in a concrete manner. As used herein, the term "or" is intended to mean an inclusive "or" rather than an exclusive "or".

[0109] Likewise, although the present application has been shown and described with respect to one or more implementations, equivalent variations and modifications will occur to those skilled in the art upon reading and understanding this specification and the accompanying drawings. With particular regard to the various functions performed by the components (e.g., elements, resources, etc.) described above, unless otherwise noted, the terms used to describe such components are intended to correspond to any component (functionally equivalent) that performs the specific functions of the described components, even if not structurally equivalent to the disclosed structures.

[0110] In addition, while a particular feature of the application may have been disclosed with respect to only one of several implementations, such feature may be combined with one or more other features of other implementations.

[0111] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention disclosed herein. The present application is intended to cover any modification, use or adaptation of the present application, which follows the general principles of the present application and includes common knowledge or customary technical means in the art that are not disclosed in the present application, and the specification and embodiments are only considered as exemplary.

[0112] It should be understood that the present application is not limited to the exact construction that has been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof.

Claims

1. An integrated monitoring system for electric control based on machine vision, characterized in that: include: An acquisition module is configured to acquire an initial image of an indication panel of an indication device, wherein the indication panel is used to indicate operating status information of a plurality of electrical devices to be monitored; A first determination module is configured to determine a search window centered on a target pixel in an initial image, and determine edge similarity between a first region and a second region; the first region is a neighborhood centered on the target pixel, and the second region is a neighborhood centered on other pixels in the search window; A second determination module is configured to obtain an edge complexity value of the initial image, and determine a region similarity between the first region and the second region according to an edge similarity and an edge complexity value between the first region and the second region; The edge complexity value is used to characterize the complexity of the edge line in the initial image; A filtering module is configured to determine a weight value of the center of the second area relative to the target pixel point by using the area similarity between the first area and the second area, and filter the target pixel point according to the weight value of the pixel point in the search window to obtain a target image after filtering the initial image; A monitoring module, configured to monitor the operation status information according to the target image; The edge complexity value is determined in the following way: , E is the edge complexity value of the initial image, N is the number of edge lines in the edge image of the initial image, is the normalization function, is the number of pixels in the kth edge line in the edge image of the initial image, is an exponential function with a natural constant as base, is the maximum curvature of the pixel points in the kth edge line in the edge image of the initial image.

2. The electronic control integrated monitoring system based on machine vision according to claim 1 is characterized in that: The edge similarity between the first region and the second region is determined as follows: Expand the gradient values ​​of the pixel points in the first region into a one-dimensional vector in a specified order to obtain an expanded first vector; Expand the gradient values ​​of the pixel points in the second region into a one-dimensional vector in a specified order to obtain an expanded second vector; The cosine similarity between the first vector and the second vector is used as the edge similarity between the first region and the second region.

3. The electronic control integrated monitoring system based on machine vision according to claim 1 is characterized in that: The edge similarity between the first region and the second region is determined as follows: 1 is used as the feature value of the pixel point in the target area whose gradient value is greater than the preset gradient threshold, and 0 is used as the feature value of the pixel point in the target area whose gradient value is less than or equal to the preset gradient threshold; The target area is any one of the first area and the second area; Expand the eigenvalues ​​of the pixels in the first region into a one-dimensional vector in a specified order to obtain an expanded third vector; and expand the eigenvalues ​​of the pixels in the second region into a one-dimensional vector in a specified order to obtain an expanded fourth vector; The cosine similarity between the third vector and the fourth vector is used as the edge similarity between the first region and the second region.

4. The electronic control integrated monitoring system based on machine vision according to claim 1 is characterized in that: The edge complexity value can also be determined in the following manner: , E is the edge complexity value of the initial image, N is the number of edge lines in the edge image of the initial image, is the number of pixels in the kth edge line in the edge image of the initial image, W is the width of the initial image, H is the height of the initial image, is the variance of the horizontal coordinates of the edge pixels in the edge image of the initial image in the image coordinate system, is the variance of the ordinate of the edge pixel points in the edge image of the initial image in the image coordinate system.

5. The electronic control integrated monitoring system based on machine vision according to claim 1 is characterized in that: Determining the region similarity between the first region and the second region according to the edge similarity and the edge complexity value between the first region and the second region includes: , is the region similarity between the first region and the i-th second region in the search window, is the edge similarity between the first region and the i-th second region in the search window, and E is the edge complexity value of the initial image.

6. The electronic control integrated monitoring system based on machine vision according to claim 1 is characterized in that: Determining a weight value of a center of the second region relative to a target pixel by using the region similarity between the first region and the second region includes: When the region similarity between the first region and the second region is greater than or equal to a preset similarity threshold, , is the weight value of the center of the i-th second area in the search window relative to the target pixel point, is the region similarity between the first region and the i-th second region in the search window, is an exponential function with a natural constant as the base, K is the side length of the search window, is the mean square error of the pixel values ​​of the pixel points at the corresponding positions in the first area and the i-th second area in the search window.

7. The electronic control integrated monitoring system based on machine vision according to claim 1 is characterized in that: Determining a weight value of a center of the second region relative to a target pixel by using the region similarity between the first region and the second region includes: When the region similarity between the first region and the second region is less than a preset similarity threshold, 0 is used as the weight value of the center of the second region relative to the target pixel.

8. The electronic control integrated monitoring system based on machine vision according to claim 1 is characterized in that: According to the weight values ​​of the pixels in the search window, the target pixels are filtered, including: According to the weight values ​​of the pixels in the search window, the pixel values ​​of the pixels in the search window are weighted summed, and the result of the weighted summation is used as the pixel value after filtering of the target pixel.

9. The electronic control integrated monitoring system based on machine vision according to claim 1 is characterized in that: Also included is a prompt module, the prompt module is configured to perform the following steps: When it is determined according to the target image that a target electrical device among the plurality of electrical devices to be monitored indicated by the indicating device has an abnormality, a prompt message is output; the prompt message is used to prompt that the target electrical device has an abnormality.

Images