Instrument panel pointer detection method, system, device and storage medium

Abstract

The present invention discloses a method, system, device and storage medium for detecting a dashboard pointer, and relates to the technical field of computer vision recognition. The method comprises: obtaining video frame information of a monitored area; preprocessing the video frame information to obtain image information that meets the detection conditions; inputting the image information into a target detection model to obtain target information; wherein the detection information comprises target category, rectangular frame coordinates and score; filtering the target information according to a preset detection threshold to obtain dashboard information as a candidate identification area; detecting the dashboard information by a Hough circle detection algorithm to obtain inner and outer circle information of the dashboard; and obtaining the indicated position of the pointer tip of the dashboard by performing image channel separation on the inner and outer circle information. The present invention obtains video frame information of the monitored area, thereby realizing real-time detection of the dashboard status, thereby solving the problem that fire-fighting equipment cannot be centrally managed, and improving the intelligent management level of fire-fighting equipment.

Description

Technical Field

[0001] The present invention relates to the field of computer vision recognition technology, and in particular to an instrument panel pointer detection method, system, device and storage medium. Background Art

[0002] In industrial equipment management, many machines and equipment or switch valves will use dashboards to display data information, such as fire extinguisher equipment. In order to ensure the safe, reliable and economical operation of the power grid room, the room is equipped with a certain number of fire extinguishers. During daily inspections and maintenance, maintenance personnel are required to manually check the indications on the dashboard to determine the status of the fire extinguisher and replace the problematic fire extinguisher equipment in a timely manner. However, manual inspections to check the status of fire-fighting equipment and discover faults in fire-fighting equipment require a high labor intensity for inspection personnel, resulting in low maintenance efficiency, low intelligence level, and inability to centrally manage. Summary of the invention

[0003] The purpose of the present invention is to provide a dashboard pointer detection method, system, device and storage medium to solve the problems of manual inspection of the dashboard status of equipment, high labor intensity, low maintenance efficiency, low intelligence level, and inability to centrally manage fire-fighting equipment.

[0004] To achieve the above object, an embodiment of the present invention provides a method for detecting a pointer on an instrument panel, comprising:

[0005] Obtain video frame information of the monitored area;

[0006] Preprocessing the video frame information to obtain image information that meets the detection conditions;

[0007] Inputting the image information into a target detection model to obtain target information; wherein the detection information includes target category, rectangular frame coordinates and score;

[0008] Filtering the target information according to a preset detection threshold to obtain instrument panel information as a candidate recognition area;

[0009] Detecting the instrument panel information by using a Hough circle detection algorithm to obtain inner and outer circle information of the instrument panel;

[0010] The indicated position of the pointer tip of the instrument panel is obtained by performing image channel separation on the inner and outer circle information.

[0011] Preferably, the step of performing image channel separation on the inner and outer circle information to obtain the indicated position of the pointer tip of the instrument panel includes:

[0012] By performing image channel separation on the inner and outer circle information, a plurality of different color regions are obtained;

[0013] According to different color areas, the coordinates of the area boundaries are obtained;

[0014] Obtaining the pointer coordinates according to the inner and outer circle information and a number of different color areas;

[0015] The indicated position of the pointer tip of the instrument panel is obtained according to the area boundary coordinates and the pointer coordinates.

[0016] Preferably, before the image information is input into the target detection model to obtain the target information, the following steps are included:

[0017] Obtain a large number of dashboard images and mark the dashboard positions to obtain dashboard training images;

[0018] After enhancing the instrument panel training images, a part is selected as a training set and another part is selected as a test set;

[0019] The training set and the test set are input into a preliminary model for training to obtain a target detection model.

[0020] Preferably, the target detection model adopts the yolov3 target detection model.

[0021] The embodiment of the present invention further provides an instrument panel pointer detection system, comprising:

[0022] An information acquisition module is used to acquire video frame information of the monitored area;

[0023] A preprocessing module, used to preprocess the video frame information to obtain image information that meets the detection conditions;

[0024] A target detection module, used to input the image information into a target detection model to obtain target information; wherein the detection information includes target category, rectangular frame coordinates and score;

[0025] A filtering module, used to filter the target information according to a preset detection threshold, and obtain the instrument panel information as a candidate identification area;

[0026] A Hough circle detection module, used to detect the instrument panel information by using a Hough circle detection algorithm to obtain the inner and outer circle information of the instrument panel;

[0027] The channel separation module is used to obtain the indicated position of the pointer tip of the instrument panel by performing image channel separation on the inner and outer circle information.

[0028] Preferably, the channel separation module is also used for: obtaining a number of different color areas by performing image channel separation on the inner and outer circle information; obtaining area boundary coordinates based on the different color areas; obtaining pointer coordinates based on the inner and outer circle information and the several different color areas; obtaining the indicated position of the pointer tip of the instrument panel based on the area boundary coordinates and the pointer coordinates.

[0029] Preferably, the dashboard pointer detection system also includes a target detection model training module, which is used to acquire a large number of dashboard images and mark the dashboard positions to obtain dashboard training images; after enhancing the dashboard training images, a part is selected as a training set and the other part is selected as a test set; the training set and the test set are input into the preliminary model for training to obtain a target detection model.

[0030] Preferably, the target detection model adopts the yolov3 target detection model.

[0031] An embodiment of the present invention further provides a computer terminal device, comprising one or more processors and a memory. The memory is coupled to the processor and is used to store one or more programs; when the one or more programs are executed by the one or more processors, the one or more processors implement the instrument panel pointer detection method as described in any of the above embodiments.

[0032] An embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the instrument panel pointer detection method as described in any of the above embodiments is implemented.

[0033] Compared with the prior art, the present invention has the following beneficial effects:

[0034] The instrument panel pointer detection method provided by the present invention includes: obtaining video frame information of the monitored area; preprocessing the video frame information to obtain image information that meets the detection conditions; inputting the image information into the target detection model to obtain target information; wherein the detection information includes target category, rectangular frame coordinates and score; filtering the target information according to a preset detection threshold to obtain instrument panel information as a candidate identification area; detecting the instrument panel information by a Hough circle detection algorithm to obtain inner and outer circle information of the instrument panel; and obtaining the indicated position of the pointer tip of the instrument panel by performing image channel separation on the inner and outer circle information. The present invention obtains video frame information of the monitored area, thereby realizing real-time detection of the instrument panel status. Once an abnormality occurs, an alarm can be issued in time to remind the patrol personnel to conduct on-site inspections, and the fire safety hazards in the machine room can be eliminated in time, reducing the labor intensity of maintenance personnel, solving the problem that fire equipment cannot be centrally managed, and improving the intelligent management level of fire equipment. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the technical solution of the present invention, the drawings required for use in the implementation mode will be briefly introduced below. Obviously, the drawings described below are only some implementation modes of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0036] Figure 1 It is a flowchart of a method for detecting a pointer on an instrument panel provided by a certain embodiment of the present invention;

[0037] Figure 2 It is a structural schematic diagram of an instrument panel pointer detection system provided by a certain embodiment of the present invention;

[0038] Figure 3 It is a schematic diagram of the structure of a computer terminal device provided by a certain embodiment of the present invention. DETAILED DESCRIPTION

[0039] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0040] It should be understood that the step numbers used in this article are only for the convenience of description and are not intended to limit the order in which the steps are executed.

[0041] It should be understood that the terms used in the present specification are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the present specification and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0042] The terms “include” and “comprising” indicate the presence of described features, integers, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or combinations thereof.

[0043] The term "and / or" means and includes any and all possible combinations of one or more of the associated listed items.

[0044] See also Figure 1 , Figure 1 1 is a flow chart of a method for detecting a pointer on an instrument panel provided by an embodiment of the present invention. In this embodiment, the method for detecting a pointer on an instrument panel comprises the following steps:

[0045] S110, obtaining video frame information of the monitored area.

[0046] In one embodiment, a number of fire extinguishers are configured in the power grid room, and each fire extinguisher has a pressure valve (dashboard). In order to reduce the amount of data acquired by image information, only one camera is needed in the same room, and the camera is used to monitor the pressure valves of all fire extinguishers in the room in real time.

[0047] S120, pre-processing the video frame information to obtain image information that meets the detection conditions.

[0048] In a certain embodiment, there is a large amount of image information in the video frame information. However, if each frame of image is imported into the target detection model, the amount of data is too complicated and large, which is not conducive to calculation. Therefore, images in the video frame can be intermittently selected as detected images according to a preset time period. This can reduce the amount of data for calculation and improve the calculation efficiency. For example, the video frame information is preprocessed, and an image is extracted every ten minutes as image information that meets the detection conditions to enter the next step of calculation.

[0049] S130, inputting the image information into a target detection model to obtain target information; wherein the detection information includes target category, rectangular frame coordinates and score.

[0050] In the embodiment of the present invention, the target information includes a dashboard area of ​​interest.

[0051] S140, filtering the target information according to a preset detection threshold to obtain instrument panel information as a candidate recognition area.

[0052] In a certain embodiment, filtering is performed for the purpose of: eliminating noise mixed in the image; extracting image features for image recognition. Due to the imperfections of imaging systems, transmission media and recording equipment, digital images are often contaminated by various noises during their formation, transmission and recording. In addition, in certain links of image processing, when the input object is not as expected, noise will also be introduced into the result image. These noises often appear as isolated pixels or pixel blocks that cause strong visual effects on the image. Generally, the noise signal is irrelevant to the object to be studied. It appears in the form of useless information and disturbs the observable information of the image. For digital image signals, the noise table is a large or small extreme value. These extreme values ​​act on the real grayscale values ​​of the image pixels through addition and subtraction, causing interference of bright and dark spots on the image, greatly reducing the image quality, and affecting the subsequent work such as image restoration, segmentation, feature extraction, and image recognition. To construct a filter that effectively suppresses noise, two basic issues must be considered: it can effectively remove noise in the target and background; at the same time, it can well protect the shape, size and specific geometric and topological structure characteristics of the image target.

[0053] S150, detecting the instrument panel information by using a Hough circle detection algorithm to obtain inner and outer circle information of the instrument panel.

[0054] S160, obtaining the indicated position of the pointer tip of the instrument panel by performing image channel separation on the inner and outer circle information.

[0055] In an embodiment of the present invention, step S160, obtaining the indicated position of the pointer tip of the instrument panel by performing image channel separation on the inner and outer circle information, includes: obtaining a number of different color areas by performing image channel separation on the inner and outer circle information; obtaining area boundary coordinates based on the different color areas; obtaining pointer coordinates based on the inner and outer circle information and the number of different color areas; obtaining the indicated position of the pointer tip of the instrument panel based on the area boundary coordinates and the pointer coordinates.

[0056] In a certain embodiment, the instrument panel mainly includes a green area and a red area. After obtaining the area of ​​interest of the instrument panel, the Hough circle detection algorithm is called to obtain the inner and outer circles of the instrument panel, and the smallest circle is obtained by diameter screening; the green and red areas of the instrument panel are divided by image channel separation; through image channel separation, combined with the position of the center of the circle, it is determined whether the black pointer tip is located in the green or red area; finally, it is determined whether the instrument panel pointer points to the green area or the red area.

[0057] In a certain embodiment, after obtaining the instrument panel's area of ​​interest (candidate identification area), the Hough circle detection algorithm is called to obtain the inner and outer circles of the instrument panel, and the smallest circle is obtained by diameter screening; the green and red areas within the smallest circle of the instrument panel are separated by image channel separation, and the number of pixels in the green and red areas within the smallest circle of the instrument panel is counted; when the black pointer points to the green and red areas, the number and proportion of pixel values ​​in the red and green areas on the instrument panel are different, and by calculating and comparing the pixel values ​​of the red and green areas, it is finally determined whether the instrument panel pointer points to the green or red area.

[0058] In a certain embodiment, after obtaining the instrument panel's area of ​​interest (candidate identification area), the Hough circle detection algorithm is called to obtain the inner and outer circles of the instrument panel, and the minimum circle is obtained by screening the diameters of the two circles, and the minimum circle area is cut out; the cut-out instrument panel minimum circle area is separated by image channel separation, the green and red indication areas within the instrument panel's minimum circle are segmented, and the number of pixels and the ratio of the green and red areas within the instrument panel's minimum circle are counted; the area of ​​the black pointer tip of the instrument panel is obtained by image channel separation, and the black pointer tip area of ​​the instrument panel includes the area where the center of the instrument panel's minimum circle is located, and the black pointer tip is determined based on the center position to determine whether the black pointer tip is in the green or red area; when the black pointer points to the green and red areas, the number and ratio of pixel values ​​in the red and green areas on the instrument panel are different, and by calculating and comparing the pixel values ​​of the red and green areas, it is finally determined whether the instrument panel pointer is pointing to the green or red area.

[0059] In an embodiment of the present invention, the image information is input into a target detection model, and before the target information is obtained, the process includes: acquiring a large number of dashboard images and marking the locations of the dashboards to obtain dashboard training images; performing enhancement processing on the dashboard training images, selecting a part as a training set and another part as a test set; and inputting the training set and the test set into a preliminary model for training to obtain a target detection model.

[0060] In an embodiment of the present invention, the target detection model adopts the yolov3 target detection model.

[0061] In one embodiment, the pre-trained target detection model inputs the pre-processed image into the yolov3 target detection model; by annotating the collected dashboard image, rotating, mirroring, adjusting the brightness and contrast, the data set size is multiplied, and then the model is trained based on the yolov3 algorithm. The principle of the yolov3 algorithm is to divide an input image into S*S grids (grid cells). If the center of an object falls in this grid, then this grid is responsible for predicting the object. Each grid predicts B bounding boxes. In addition to returning to its own position, each bounding box also predicts a confidence. This confidence represents the confidence of the predicted box containing the object and how accurate the prediction of this box is. The value calculation principle is: if an object falls in a grid, the first item takes 1, otherwise it takes 0. The second item is the overlap between the predicted bounding box and the actual classification accuracy (IoU value). Each bounding box predicts a total of 5 values ​​(x, y, w, h) and confidence, and each grid also predicts a category information, recorded as C category. There are S*S grids, and each grid needs to predict B borders and C categories. The output is a tensor of SxSx(5*B+C), which then passes through multiple fully connected layers corresponding to the number of categories of the target to be detected. The trained algorithm model can be deployed in a cloud server or in an edge device (such as Jetson Nano and TX2) to achieve real-time detection of targets.

[0062] See also Figure 2 , Figure 2 : is a schematic diagram of the structure of an instrument panel pointer detection system provided by a certain embodiment of the present invention. In this embodiment, the instrument panel pointer detection system includes:

[0063] The information acquisition module 210 is used to acquire the video frame information of the monitored area;

[0064] A preprocessing module 220, used to preprocess the video frame information to obtain image information that meets the detection conditions;

[0065] The target detection module 230 is used to input the image information into the target detection model to obtain target information; wherein the detection information includes target category, rectangular frame coordinates and score;

[0066] A filtering module 240, configured to filter the target information according to a preset detection threshold, and obtain instrument panel information as a candidate identification area;

[0067] A Hough circle detection module 250, configured to detect the instrument panel information by using a Hough circle detection algorithm to obtain inner and outer circle information of the instrument panel;

[0068] The channel separation module 260 is used to obtain the indicated position of the pointer tip of the instrument panel by performing image channel separation on the inner and outer circle information.

[0069] In an embodiment of the present invention, the channel separation module is also used to: obtain a number of different color areas by performing image channel separation on the inner and outer circle information; obtain area boundary coordinates based on the different color areas; obtain pointer coordinates based on the inner and outer circle information and the several different color areas; obtain the indicated position of the pointer tip of the instrument panel based on the area boundary coordinates and the pointer coordinates.

[0070] In an embodiment of the present invention, the dashboard pointer detection system also includes a target detection model training module, which is used to acquire a large number of dashboard images and mark the dashboard positions to obtain dashboard training images; after enhancing the dashboard training images, a part is selected as a training set and another part is selected as a test set; the training set and the test set are input into the preliminary model for training to obtain a target detection model.

[0071] In an embodiment of the present invention, the target detection model adopts the yolov3 target detection model.

[0072] For the specific definition of the instrument panel pointer detection system, please refer to the definition of the instrument panel pointer detection system mentioned above, which will not be repeated here. Each module in the above instrument panel pointer detection system can be implemented in whole or in part by software, hardware and their combination. The above modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0073] See also Figure 3 An embodiment of the present invention provides a computer terminal device, comprising one or more processors and a memory. The memory is coupled to the processor and is used to store one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the instrument panel pointer detection method in any of the above embodiments.

[0074] The processor is used to control the overall operation of the computer terminal device to complete all or part of the steps of the above-mentioned instrument panel pointer detection method. The memory is used to store various types of data to support the operation of the computer terminal device, and these data may include, for example, instructions for any application or method used to operate on the computer terminal device, as well as application-related data. The memory can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (Static Random Access Memory, referred to as SRAM), electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, referred to as EEPROM), erasable programmable read-only memory (Erasable Programmable Read-Only Memory, referred to as EPROM), programmable read-only memory (Programmable Read-Only Memory, referred to as PROM), read-only memory (Read-Only Memory, referred to as ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0075] In an exemplary embodiment, the computer terminal device can be implemented by one or more application specific integrated circuits (Application Specific 1ntegrated Circuit, referred to as ASC), digital signal processors (Digital Signal Processor, referred to as DSP), digital signal processing devices (Digital Signal Processing Device, referred to as DSPD), programmable logic devices (Programmable Logic Device, referred to as PLD), field programmable gate arrays (Field Programmable Gate Array, referred to as FPGA), controllers, microcontrollers, microprocessors or other electronic components to execute the above-mentioned instrument panel pointer detection method and achieve the same technical effect as the above-mentioned method.

[0076] In another exemplary embodiment, a computer-readable storage medium including program instructions is also provided, and when the program instructions are executed by a processor, the steps of the instrument panel pointer detection method in any of the above embodiments are implemented. For example, the computer-readable storage medium can be the above-mentioned memory including program instructions, and the above-mentioned program instructions can be executed by a processor of a computer terminal device to complete the above-mentioned instrument panel pointer detection method and achieve the same technical effect as the above-mentioned method.

[0077] The above is a preferred embodiment of the present invention. It should be pointed out that a person skilled in the art can make several improvements and modifications without departing from the principle of the present invention. These improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A method for detecting a pointer on an instrument panel, characterized in that: include: Obtain video frame information of the monitored area; Preprocessing the video frame information to obtain image information that meets the detection conditions; Inputting the image information into a target detection model to obtain target information; wherein the target information includes target category, rectangular frame coordinates and score; Filtering the target information according to a preset detection threshold to obtain instrument panel information as a candidate recognition area; Detecting the instrument panel information by using a Hough circle detection algorithm to obtain inner and outer circle information of the instrument panel; By performing image channel separation on the inner and outer circle information, the indicated position of the pointer tip of the instrument panel is obtained; The step of performing image channel separation on the inner and outer circle information to obtain the indicated position of the pointer tip of the instrument panel includes: By performing image channel separation on the inner and outer circle information, a plurality of different color regions are obtained; According to different color areas, the coordinates of the area boundaries are obtained; Obtaining the pointer coordinates according to the inner and outer circle information and a number of different color areas; Calculating the pixel value ratio in the area according to the area boundary coordinates and the pointer coordinates, and obtaining the indicated position of the pointer tip of the instrument panel according to the pixel value ratio; By performing image channel separation on the inner and outer circle information, several different color areas are obtained, including: Screening is performed according to the inner and outer circle information and the preset diameter to obtain the minimum circle information of the instrument panel; Image channels are separated according to the smallest circle information of the instrument panel to obtain several different color areas.

2. The instrument panel pointer detection method according to claim 1, characterized in that: The image information is input into the target detection model to obtain the target information, including: Obtain a large number of dashboard images and mark the dashboard positions to obtain dashboard training images; After enhancing the instrument panel training images, a part is selected as a training set and another part is selected as a test set; The training set and the test set are input into a preliminary model for training to obtain a target detection model.

3. The instrument panel pointer detection method according to claim 1, characterized in that: The target detection model adopts the yolov3 target detection model.

4. An instrument panel pointer detection system, characterized in that: include: An information acquisition module is used to acquire video frame information of the monitored area; A preprocessing module, used to preprocess the video frame information to obtain image information that meets the detection conditions; A target detection module, used to input the image information into a target detection model to obtain target information; wherein the target information includes target category, rectangular frame coordinates and score; A filtering module, used to filter the target information according to a preset detection threshold, and obtain the instrument panel information as a candidate identification area; A Hough circle detection module, used to detect the instrument panel information by using a Hough circle detection algorithm to obtain the inner and outer circle information of the instrument panel; A channel separation module is used to obtain the indicated position of the pointer tip of the instrument panel by performing image channel separation on the inner and outer circle information, including: obtaining a number of different color areas by performing image channel separation on the inner and outer circle information; obtaining area boundary coordinates based on the different color areas; obtaining pointer coordinates based on the inner and outer circle information and the number of different color areas; calculating the pixel value ratio in the area based on the area boundary coordinates and the pointer coordinates, and obtaining the indicated position of the pointer tip of the instrument panel based on the pixel value ratio; wherein, obtaining a number of different color areas by performing image channel separation on the inner and outer circle information includes: obtaining the minimum circle information of the instrument panel by screening according to the inner and outer circle information and a preset diameter; performing image channel separation based on the minimum circle information of the instrument panel to obtain a number of different color areas.

5. The instrument panel pointer detection system according to claim 4, characterized in that: It also includes a target detection model training module, which is used to obtain a large number of dashboard images and mark the dashboard positions to obtain dashboard training images; after enhancing the dashboard training images, a part is selected as a training set and the other part is selected as a test set; the training set and the test set are input into the preliminary model for training to obtain a target detection model.

6. The instrument panel pointer detection system according to claim 4, characterized in that: The target detection model adopts the yolov3 target detection model.

7. A computer terminal device, characterized in that: include: one or more processors; A memory, coupled to the processor, for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the instrument panel pointer detection method as described in any one of claims 1 to 3.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the instrument panel pointer detection method according to any one of claims 1 to 3 is implemented.

Images