Knob switch state identification method, computer readable storage medium and identification system

Abstract

The invention discloses a knob switch state identification method, a computer readable storage medium and an identification system. The knob switch state identification method comprises the following steps: acquiring an original image of a knob switch; and monocular depth estimation processing is carried out on the original image to obtain a depth image. And performing binarization processing on the depth image, and extracting a part of the knob switch to obtain a binarization image of the knob switch. And carrying out contour detection and screening on the binarized image to obtain a rectangular frame of the knob switch and a rotation angle of the rectangular frame. And judging the rotation angle of the rectangular frame, and identifying the state of the rotary switch according to a judgment result. According to the recognition method, manual marking from the coordinate position of the knob switch in the original image is not needed, the labor cost is reduced, the knob switch state recognition efficiency is improved, and the automation degree is high.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a method for identifying a knob switch state, a computer-readable storage medium and an identification system. Background Art

[0002] A large number of knob switches are equipped in distribution rooms and other related industrial sites. When the knob switch is in different positions, it represents different working states of the equipment. When the inspection robot inspects the distribution room, it needs to automatically identify the position status of various knob switches in the distribution room, so that it can automatically and timely understand and grasp the working status of the equipment. In the related technology, it is necessary to manually mark the coordinate position of the knob switch in the original image first, and then segment the knob switch from the original image according to the marked content. Manually marking the coordinate position of the knob switch in the original image increases labor costs, reduces the efficiency of knob switch state recognition, and has a low degree of automation. Summary of the invention

[0003] The main purpose of the present invention is to provide a method for identifying the state of a knob switch, a computer-readable storage medium and an identification system, aiming to solve the technical problems in the related art of manually marking the coordinate position of the knob switch in the original image, increasing labor costs, reducing the efficiency of knob switch state identification, and having a low degree of automation.

[0004] In order to achieve the above-mentioned object of the invention, a first aspect of the present invention provides a method for identifying a knob switch state.

[0005] A method for identifying a knob switch state, comprising:

[0006] Get the original image of the knob switch;

[0007] Performing monocular depth estimation processing on the original image to obtain a depth image;

[0008] Binarizing the depth image, extracting the portion of the rotary switch, and obtaining a binary image of the rotary switch;

[0009] Performing contour detection and screening on the binary image to obtain a rectangular frame of the knob switch and a rotation angle of the rectangular frame;

[0010] The rotation angle of the rectangular frame is judged, and the state of the rotation switch is identified according to the judgment result.

[0011] In one embodiment, the step of binarizing the depth image, extracting the portion of the knob switch, and obtaining a binarized image of the knob switch comprises:

[0012] The pixel value of the depth image is a relative depth value of an object at the same position in the depth image from the camera, the knob switch is a raised portion, and the raised portion is closer to the camera than other portions;

[0013] The pixel value of the raised portion from the camera is greater than a threshold;

[0014] Determining each pixel value of the depth image;

[0015] If the pixel value is greater than the threshold, display the pixel in the first color;

[0016] If the pixel value is less than the threshold, the pixel point is displayed in a second color to obtain a binary image of the knob switch.

[0017] In one embodiment, the determination of the threshold comprises the following steps:

[0018] measuring the relative height of the raised portion;

[0019] If the relative height of the raised portion is higher, the threshold is reduced;

[0020] If the relative height of the protruding portion is lower, the threshold is increased.

[0021] In one embodiment, the step of binarizing the depth image, extracting the portion of the knob switch, and obtaining a binarized image of the knob switch comprises:

[0022] Binarizing the depth image after the grayscale processing;

[0023] Performing morphological processing on the depth image;

[0024] Performing an erosion operation according to image features of the depth image to eliminate noise portions;

[0025] A dilation operation is performed on the depth image with the noise removed to restore the original contour of the depth image, thereby obtaining a binary image composed of two colors.

[0026] In one embodiment, the step of determining the rotation angle of the rectangular frame and identifying the state of the rotation switch according to the determination result includes:

[0027] The vertical angle of the rectangular frame is set to zero degrees, the angle of the rectangular frame deflected in the counterclockwise direction is set to a positive angle, and the angle of the rectangular frame deflected in the clockwise direction is set to a negative angle;

[0028] When the rectangular frame is at a positive angle, the knob switch is in a first state;

[0029] When the rectangular frame has a negative angle, the rotary switch is in the second state.

[0030] In one embodiment, before the step of performing monocular depth estimation processing on the original image to obtain the depth image, the step further includes:

[0031] Performing denoising processing on the original image;

[0032] The contrast of the original image after denoising is enhanced.

[0033] In one embodiment, the rectangular frame is a minimum circumscribed rotation rectangle surrounding the knob switch.

[0034] In one of the embodiments, the relative depth value of the depth image is 0-255.

[0035] A second aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the above-mentioned method for identifying the state of a knob switch when executed by a computer processor.

[0036] A third aspect of the present invention provides a system for identifying a knob switch state, comprising:

[0037] An image acquisition unit, used for acquiring an original image of the knob switch;

[0038] A first image processing unit, configured to perform monocular depth estimation processing on the original image to obtain a depth image;

[0039] A second image processing unit is used to perform binarization processing on the depth image, extract the part of the knob switch, and obtain a binary image of the knob switch;

[0040] A knob contour acquisition unit, used to perform contour detection and screening on the binary image to obtain a rectangular frame of the knob switch and a rotation angle of the rectangular frame;

[0041] The judging and identifying unit is used to judge the rotation angle of the rectangular frame and identify the state of the rotation switch according to the judging result.

[0042] Beneficial effects:

[0043] The method for identifying the state of a rotary switch of the present invention obtains an original image of the rotary switch;

[0044] The original image is processed by monocular depth estimation to obtain a depth image; the depth image is binarized to extract the knob switch part to obtain a binarized image of the knob switch; the binarized image is subjected to contour detection and screening to obtain a rectangular frame of the knob switch and the rotation angle of the rectangular frame; the rotation angle of the rectangular frame is judged, and the state of the rotary switch is identified according to the judgment result. The recognition method does not need to manually mark the coordinate position of the knob switch in the original image, which reduces labor costs, improves the efficiency of knob switch state recognition, and has a high degree of automation. In addition, the recognition method can accurately identify the rotation angle of the knob switch, so as to further identify the state of the knob switch, and realize automatic recognition of the state of the knob switch. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 The present invention is a flowchart of a method for identifying a knob start state according to an embodiment of the present invention.

[0046] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0047] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0048] In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", "clockwise", "counterclockwise" and the like indicate positions or positional relationships based on the positions or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first" and "second" are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, the meaning of "multiple" is two or more, unless otherwise clearly and specifically defined.

[0049] In the description of the present invention, it should be noted that, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection, a direct connection, or an indirect connection through an intermediate medium, it can be the internal connection of two elements or the interaction relationship between two elements. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0050] In the present invention, unless otherwise clearly specified and limited, a first feature being "above" or "below" a second feature may include that the first and second features are in direct contact, or may include that the first and second features are not in direct contact but are in contact through another feature between them. Moreover, a first feature being "above", "above" and "above" a second feature includes that the first feature is directly above and obliquely above the second feature, or simply indicates that the first feature is higher in level than the second feature. A first feature being "below", "below" and "below" a second feature includes that the first feature is directly below and obliquely below the second feature, or simply indicates that the first feature is lower in level than the second feature.

[0051] like Figure 1 As shown, in some embodiments, a method for identifying a knob switch state includes the following steps:

[0052] S100: Acquire the original image of the knob switch.

[0053] S200: Perform monocular depth estimation processing on the original image to obtain a depth image.

[0054] S300 , binarizing the depth image, extracting the knob switch portion, and obtaining a binarized image of the knob switch.

[0055] S400 , performing contour detection and screening on the binary image to obtain a rectangular frame of the rotary switch and a rotation angle of the rectangular frame.

[0056] S500: Determine the rotation angle of the rectangular frame, and identify the state of the rotation switch according to the determination result.

[0057] It should be noted that in the related art, it is necessary to manually mark the knob switch and manually locate the position of the rotary switch. After the knob position is manually marked, further state recognition is required. When performing state recognition, the binary image is trained using the GhostNet classification network to obtain a multi-state knob switch classification model. When using the GhostNet classification network to train the binary image, a large number of training images need to be prepared in advance, which takes a lot of time and effort.

[0058] The recognition method in this embodiment uses the inspection robot camera to capture the original image of the knob switch, and inputs the original image into the monocular depth estimation pre-training model for inference prediction to generate a corresponding depth image. The depth image is then binarized to obtain a binary image containing only the raised part of the knob switch, and then the binary image is contour detected and screened, and the minimum circumscribed rotation rectangle of the remaining contour is calculated. Finally, the angle of the minimum circumscribed rectangle is logically judged accordingly to identify the current state of the knob switch.

[0059] The recognition method in this embodiment can realize automatic detection and positioning of the knob switch, and can also accurately identify the rotation angle of the knob switch, without manually marking the coordinate position of the knob switch in the original image, thereby reducing labor costs, improving the efficiency of knob switch state recognition, and having a high degree of automation. The recognition method in this embodiment uses a pre-trained monocular depth estimation model for depth estimation, does not require model training, does not require preparation of corresponding training images, and does not require data annotation, which can save a lot of time and energy.

[0060] In some embodiments, before the step of performing monocular depth estimation on the original image in S200 to obtain a relative depth image, the step further includes:

[0061] S210: De-noising the original image.

[0062] S220, performing contrast enhancement processing on the original image after the denoising processing. The accuracy of monocular depth estimation can be improved by image denoising and contrast enhancement processing.

[0063] Specifically, the median filtering algorithm is used to denoise the original image, covering each pixel of the original image, taking a neighborhood window of a preset size with the pixel as the center, sorting the pixel values ​​in the neighborhood window, and taking the median value as the new pixel value of the pixel, thereby effectively removing noise and other interference in the original image.

[0064] Specifically, the histogram equalization method can be used to enhance the contrast of the original image, the grayscale histogram of the original image can be counted, the cumulative distribution function can be calculated, and the grayscale value of each pixel of the original image can be mapped according to the cumulative distribution function, so that the grayscale distribution of the image is more uniform, highlighting the difference between the knob switch and the background. After the above-mentioned image denoising and contrast enhancement preprocessing operations, the monocular depth estimation processing can be performed, which can significantly improve the accuracy and reliability of depth estimation, and thus more accurately identify the state of the knob switch.

[0065] In some embodiments, before binarizing the depth image, the depth image is also included in brightness adjustment to make the binarized image clearer. The specific brightness adjustment method is as follows: calculate the average brightness of the pixels of the depth image, and in response to the average brightness being less than the preset brightness value, adjust the average brightness of the pixels of the depth image to increase the brightness of the depth image. Adjusting the average brightness of the pixels of the depth image to increase the brightness of the depth image is specifically as follows: taking 10% of the brightness value of the depth image as the minimum value and 90% as the maximum value, in response to the brightness value of the pixel in the depth image being less than the minimum value, the minimum value is used to replace the brightness value of the pixel; in response to the brightness value of the pixel in the depth image being greater than the maximum value, the maximum value is used to replace the brightness value of the pixel; the pixels of the depth image are stretched to 0-255. The above method can effectively improve the overall brightness of the picture without affecting the subsequent recognition process.

[0066] Monocular depth estimation refers to estimating the depth information in a scene through a single-view image (usually an RGB image) captured by a single camera. That is, the depth value of each pixel in the image is estimated. The depth value refers to the distance from the surface of the object to the camera. This distance information is usually represented by a relative depth image. The value of each pixel corresponds to the depth value of a certain point in the scene. The depth value includes relative depth and absolute depth. Relative depth refers to the depth relationship between different objects or areas in the image. This relative depth representation method helps to understand the spatial layout of the scene and the relative positions between objects. In the relative depth map, the depth difference between different objects or areas is usually represented by the change of color depth or hue. Absolute depth refers to the actual distance from the object in the scene to the camera, usually expressed in physical units such as meters (m) or centimeters (cm).

[0067] This embodiment uses the open source Depth-Anything-V2 pre-trained model to estimate the relative depth of the knob switch image and generate a relative depth map. The relative depth map is a single-channel grayscale image with the same resolution as the original image. The pixel value of each pixel in the image represents the relative depth value of the object at the same position in the original image from the camera. The relative depth value range is between 0-255. The smaller the relative depth value, the greater the relative depth, indicating that the farther the point is from the camera, and the darker the color in the corresponding depth image. The larger the relative depth value, the smaller the relative depth, indicating that the closer the point is from the camera, and the brighter the color in the corresponding relative depth map.

[0068] Since the rotary switch has a very obvious raised part, the relative distance between the raised part and the camera is the shortest, so the pixel value in the depth image is the largest, the corresponding color is the brightest, and it appears white. In addition to the raised part of the rotary switch, the other areas are relatively far away from the camera, so the pixel value in the relative depth map is small, and the corresponding color is darker. Among them, the background panel area is the farthest away, so the color in the relative depth map is the darkest, and it appears black.

[0069] Specifically, S300, the step of performing binarization processing on the relative depth image, extracting the knob switch portion, and obtaining a binarized image of the knob switch includes:

[0070] S310: The pixel value of the relative depth image is a relative depth value of an object at the same position in the relative depth image from the camera. The knob switch is a raised portion, and the raised portion is closer to the camera than other portions.

[0071] S320: The pixel value of the raised portion from the camera is greater than a threshold.

[0072] S330: Determine each pixel value of the relative depth image.

[0073] S340: If the pixel value is greater than the threshold, display the pixel in the first color.

[0074] S350: If the pixel value is less than the threshold, the pixel point is displayed in a second color to obtain a binary image of the knob switch.

[0075] It should be noted that the raised part in the original image is relatively close to the camera, so the pixel value of the raised part in the depth image is greater than the threshold. By performing monocular depth estimation on the original image, a single-channel depth image can be obtained. The pixel value of each pixel in the depth image represents the relative distance between the position and the camera. Since the knob switch has a significant bulge, the distance between the raised part and the camera is the shortest, and the raised part appears the whitest in the depth image. For example, the pixel value of the raised part should be above 200, so in order to extract the complete raised part, the threshold can be set to 200. Using this threshold to binarize the relative depth map, a binary image can be obtained. After binarization, the raised part of the knob switch is completely extracted, and except for the raised part, other areas are filtered out.

[0076] More specifically, the determination of the threshold includes the following steps:

[0077] S321, measuring the relative height of the raised portion;

[0078] S322, if the relative height of the raised portion is higher, reduce the threshold;

[0079] S323. If the relative height of the raised portion is lower, increase the threshold.

[0080] It should be noted that the threshold can be appropriately adjusted according to the relative height of the raised part of the knob in the original image. Since the raised part is closest to the camera, it always appears white in the corresponding depth image and the color is always the brightest. When the relative height of the raised part is higher, the difference in pixel value between the raised part and other parts in the depth image is larger. Even if the threshold is appropriately reduced, the raised part can be better extracted. When the relative height of the raised part is lower, the difference in pixel value between the raised part and other parts in the depth image is smaller. At this time, if the threshold is small, it is easy to extract other parts as raised parts, which will affect the subsequent determination of the rotation angle of the knob switch. Therefore, the threshold can be appropriately increased at this time.

[0081] In other embodiments, S300, the step of performing binarization processing on the relative depth image, extracting the portion of the knob switch, and obtaining a binarized image of the knob switch includes:

[0082] S361, binarize the relative depth image after the grayscale processing.

[0083] S362: Perform morphological processing on the relative depth image.

[0084] S363: Perform an erosion operation according to the image features of the relative depth image to eliminate noise portions.

[0085] S364: Perform a dilation operation on the relative depth image with the noise removed to restore the original contour of the relative depth image, and obtain a binary image composed of two colors.

[0086] In some embodiments, S400 , contour detection and screening are performed on the binary image to obtain a rectangular frame of the knob switch and a rotation angle of the rectangular frame.

[0087] It should be noted that the contour of the binary image is detected, and then the minimum circumscribed rotated rectangle of each contour is detected. This process can just surround the contour part with the minimum circumscribed rotated rectangle frame, and the rotation angle of the rotated rectangle can be obtained, and the detected minimum circumscribed rotated rectangle is drawn in the original image. The raised part of the knob switch is framed in the rectangular frame, thereby realizing the automatic detection and positioning of the knob switch.

[0088] In some embodiments, S500, the rotation angle of the rectangular frame is determined, and the state of the rotation switch is identified according to the determination result.

[0089] Specifically, S500, the step of determining the rotation angle of the rectangular frame and identifying the state of the rotation switch according to the determination result includes:

[0090] S510: Set the vertical angle of the rectangular frame to zero degrees, the counterclockwise angle of the rectangular frame to a positive angle, and the clockwise angle of the rectangular frame to a negative angle.

[0091] S520: When the rectangular frame is at a positive angle, the knob switch is in a first state, which can be defined as a "remote" state.

[0092] S530: When the rectangular frame is at a negative angle, the rotary switch is in a second state, which can be defined as a "local" state.

[0093] It should be noted that the rotating rectangular border has the property of rotation angle. The rotation angle is converted, and the converted angle is defined as follows: the vertical direction is zero degrees, the angle deflected in the counterclockwise direction is a positive angle, and the angle deflected in the clockwise direction is a negative angle. For example, the counterclockwise deflection angle is 54 degrees. Through logical judgment, the current state of the knob switch can be quickly determined. The knob switch has two position states: "remote" and "local". When the angle value is positive, it means that the knob switch is deflected in the counterclockwise direction, and it is currently in the "remote" state. When the angle value is negative, it means that the knob switch is deflected in the clockwise direction, and it is currently in the "local" state.

[0094] In some embodiments, after obtaining the rectangular frame of the knob switch, the rectangular frame is smoothed. The specific steps are as follows:

[0095] Determine the coordinates of the four vertices of the rectangular border: Accurately determine the coordinates of the four vertices of the rectangular border by detecting and filtering the contour of the knob switch in the binary image. This step can be implemented using functions or algorithms in an existing image processing library to ensure the accuracy of the vertex coordinates.

[0096] Select neighborhood range: Based on each vertex, select a preset size of square neighborhood range with it as the center. The preset size can be adjusted according to the image resolution and the size of the knob switch, and can generally be set to a range of 5 pixels × 5 pixels to 15 pixels × 15 pixels. If the image resolution is high or the knob switch is large, the neighborhood range can be appropriately expanded; otherwise, the neighborhood range can be reduced.

[0097] Calculate the average coordinates: For all pixels in the neighborhood of each vertex, extract their horizontal and vertical coordinate values. Calculate the average horizontal coordinates and the average vertical coordinates of all pixels in the neighborhood. You can first add up the horizontal coordinate values ​​of all pixels and then divide them by the total number of pixels to get the average horizontal coordinate. Similarly, calculate the average vertical coordinate. The calculated average horizontal coordinate and average vertical coordinate are used as the new vertex coordinates.

[0098] Form a new rectangular border: Connect the four newly obtained vertices in sequence to form a new rectangular border. The connection method can be straight line connection to ensure that the four sides of the rectangular border are straight. Check whether the newly formed rectangular border meets the geometric characteristics of a rectangle, such as whether the opposite sides are parallel and equal in length, and whether the adjacent sides are perpendicular. If not, there may be an error in calculating the coordinates of the new vertices, which needs to be rechecked and calculated.

[0099] Repeat processing: Repeat the steps of selecting the neighborhood range, calculating the coordinate average, and forming a new rectangular border several times. The number of repetitions can be set according to the actual situation, and can generally be set to 5 to 10 times. A dynamic repetition condition can also be set, for example, when the change of the rectangular border is less than the preset threshold, the repetition is stopped. During each repetition, the vertex coordinates and size information of the newly formed rectangular border are recorded. Calculate the change of the rectangular border after each repetition. The change can be determined by comparing the difference in vertex coordinates, side length or area between two adjacent rectangular borders. For example, the sum of the Euclidean distances between the corresponding vertices of two adjacent rectangular borders can be calculated as an indicator of the change.

[0100] Stop condition judgment: When the change of the rectangular border is less than the preset threshold, stop the repeated processing. The preset threshold can be determined according to the specific application requirements and image accuracy. If the accuracy of the angle calculation is required to be high, a smaller threshold can be set; otherwise, a larger threshold can be set. The preset threshold can be expressed in absolute values, such as the change of the vertex coordinates is less than 1 pixel unit; it can also be expressed in relative values, such as the change of the area of ​​the rectangular border is less than a certain proportion of the initial area.

[0101] Through the above steps, the rectangular frame of the knob switch can be smoothed more accurately, reducing the error in angle calculation, so that the subsequent determination of the state of the knob switch based on the rotation angle of the rectangular frame is more accurate and reliable.

[0102] In another embodiment, a computer-readable storage medium stores a computer program, and when the computer program is executed by a computer processor, the above-mentioned method for identifying the state of a knob switch is implemented.

[0103] The computer-readable storage medium in this embodiment can be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium can be, for example, an electrical, magnetic, optical, electromagnetic, infrared or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media can include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In this embodiment, the computer-readable signal medium can include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal can take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer-readable signal medium may also be any computer-readable storage medium other than a computer-readable storage medium that can send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0104] This embodiment also provides a computer-readable storage medium, which may be included in the electronic device described in the above embodiment; or it may exist independently without being assembled into the electronic device. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed by the electronic device, the electronic device: obtains the original image of the knob switch; performs monocular depth estimation processing on the original image to obtain a depth image; performs binarization processing on the depth image, extracts the part of the knob switch, and obtains the binarization image of the knob switch; performs contour detection and screening on the binarization image to obtain the rectangular frame of the knob switch and the rotation angle of the rectangular frame; judges the rotation angle of the rectangular frame, and identifies the state of the rotary switch according to the judgment result.

[0105] In another embodiment, a system for identifying the state of a knob switch includes an image acquisition unit, a first image processing unit, a second image processing unit, a knob contour acquisition unit, and a judgment and recognition unit. The image acquisition unit is used to acquire an original image of the knob switch. The first image processing unit is used to perform monocular depth estimation processing on the original image to obtain a relative depth image. The second image processing unit is used to perform binarization processing on the relative depth image, extract a portion of the knob switch, and obtain a binarized image of the knob switch. The knob contour acquisition unit is used to perform contour detection and screening on the binarized image to obtain a rectangular border of the knob switch and a rotation angle of the rectangular border. The judgment and recognition unit is used to judge the rotation angle of the rectangular border and identify the state of the rotary switch based on the judgment result.

[0106] The above description is only a preferred embodiment of the present invention, and does not limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made by using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A method for identifying a knob switch state, characterized in that: The following steps are involved: Get the original image of the knob switch; Performing monocular depth estimation processing on the original image to obtain a depth image; Binarizing the depth image, extracting the portion of the rotary switch, and obtaining a binary image of the rotary switch; Performing contour detection and screening on the binary image to obtain a rectangular frame of the knob switch and a rotation angle of the rectangular frame; The rotation angle of the rectangular frame is judged, and the state of the rotation switch is identified according to the judgment result.

2. The identification method according to claim 1, characterized in that: The step of binarizing the depth image, extracting the portion of the knob switch, and obtaining a binarized image of the knob switch comprises: The pixel value of the depth image is a relative depth value of an object at the same position in the original image from the camera, the knob switch is a raised portion, and the raised portion is closer to the camera than other portions; Determining each pixel value of the depth image; If the pixel value is greater than the threshold, display the pixel in the first color; If the pixel value is less than the threshold, the pixel point is displayed in a second color to obtain a binary image of the knob switch; The pixel value of the raised portion from the camera is greater than a threshold.

3. The identification method according to claim 2, characterized in that: Determination of the threshold value comprises the following steps: measuring the relative height of the raised portion; If the relative height of the raised portion is higher, the threshold is reduced; If the relative height of the protruding portion is lower, the threshold is increased.

4. The identification method according to claim 1, characterized in that: The step of binarizing the depth image, extracting the portion of the knob switch, and obtaining a binarized image of the knob switch comprises: Binarizing the depth image after the grayscale processing; Performing morphological processing on the depth image; Performing an erosion operation according to image features of the depth image to eliminate noise portions; A dilation operation is performed on the depth image with the noise removed to restore the original contour of the depth image, thereby obtaining a binary image composed of two colors.

5. The identification method according to claim 1, characterized in that: The step of judging the rotation angle of the rectangular frame and identifying the state of the rotation switch according to the judgment result comprises: The vertical angle of the rectangular frame is set to zero degrees, the angle of the rectangular frame deflected in the counterclockwise direction is set to a positive angle, and the angle of the rectangular frame deflected in the clockwise direction is set to a negative angle; When the rectangular frame is at a positive angle, the knob switch is in a first state; When the rectangular frame has a negative angle, the rotary switch is in the second state.

6. The identification method according to claim 1, characterized in that: Before the step of performing monocular depth estimation processing on the original image to obtain a depth image, the step further includes: Performing denoising processing on the original image; The contrast of the original image after denoising is enhanced.

7. The identification method according to claim 1, characterized in that: The rectangular frame is the smallest circumscribed rotating rectangle surrounding the knob switch.

8. The identification method according to claim 1, characterized in that: The relative depth value of the depth image is 0-255.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a computer processor, the method for identifying the state of a knob switch according to any one of claims 1 to 8 is implemented.

10. A system for identifying the state of a knob switch, characterized in that: include: An image acquisition unit, used for acquiring an original image of the knob switch; A first image processing unit, configured to perform monocular depth estimation processing on the original image to obtain a depth image; A second image processing unit is used to perform binarization processing on the depth image, extract the part of the knob switch, and obtain a binary image of the knob switch; A knob contour acquisition unit, used to perform contour detection and screening on the binary image to obtain a rectangular frame of the knob switch and a rotation angle of the rectangular frame; The judging and identifying unit is used to judge the rotation angle of the rectangular frame and identify the state of the rotation switch according to the judging result.

Images