Inspection robot, switch state recognition method, device and storage medium

Abstract

The embodiments of this specification provide a patrol robot, a switch state recognition method, a device and a storage medium, the method comprising: obtaining a physical mark on a switch action part; the physical mark is used to indicate the current state of the switch action part; querying a table of correspondences between marks and states according to the physical mark; and determining the state of the switch action part based on the query result. The embodiments of this specification can improve the switch state recognition accuracy of the patrol robot.

Description

Technical Field

[0001] The present invention relates to the technical field of inspection robots, and in particular to an inspection robot, a switch state recognition method, a device and a storage medium. Background Art

[0002] With the rapid development of data centers, data center room inspection has become an important task to ensure the operation and maintenance of data centers. With the continuous increase of power distribution cabinet equipment, manual inspection can no longer meet the operation and maintenance needs of large data centers, and inspection robots are gradually deployed in the operation and maintenance of computer rooms.

[0003] The current inspection robot detection module is mainly used to inspect switches, instrument panels, status lights and other devices. However, in actual applications, it is found that when the inspection robot inspects switch devices, it is easy to have the problem of switch status recognition error. Summary of the invention

[0004] The purpose of the embodiments of this specification is to provide an inspection robot, a switch state recognition method, a device and a storage medium to improve the switch state recognition accuracy of the inspection robot.

[0005] To achieve the above objectives, on the one hand, an embodiment of this specification provides a switch state identification method, including:

[0006] Acquire a physical mark on a switch action portion; the physical mark is used to indicate a current state of the switch action portion;

[0007] Query a table of correspondence between a tag and a state according to the entity tag;

[0008] The state of the switch actuating unit is determined based on the query result.

[0009] In a preferred embodiment, the entity mark is a color mark.

[0010] In a preferred embodiment, the method further comprises:

[0011] synchronously performing image recognition on the switch action portion, and determining a state of the switch action portion based on the image recognition result;

[0012] confirming whether the state determined based on the image recognition result is consistent with the state determined based on the query result;

[0013] If the state determined based on the image recognition result is consistent with the state determined based on the query result, it is used as the switch state recognition result.

[0014] In a preferred embodiment, the method further comprises:

[0015] If the state determined based on the image recognition result is inconsistent with the state determined based on the query result, the entity mark on the switch action part is re-acquired and re-recognized accordingly.

[0016] In a preferred embodiment, the synchronously performing image recognition on the switch action part and determining the state of the switch action part based on the image recognition result includes:

[0017] Acquiring an image of the switch action portion;

[0018] Recognize the state of the switch action part based on the image contour to obtain a first recognition result; and recognize the state of the switch action part based on the image color to obtain a second recognition result;

[0019] confirming whether the first recognition result is consistent with the second recognition result;

[0020] If the first recognition result is consistent with the second recognition result, it is regarded as a state determined based on the image recognition result.

[0021] In a preferred embodiment, the synchronously performing image recognition on the switch action part and determining the state of the switch action part based on the image recognition result further includes:

[0022] If the first recognition result is inconsistent with the second recognition result, the image of the switch action portion is reacquired and re-recognized based on the image.

[0023] In a preferred embodiment, the identifying the state of the switch action portion based on the image contour includes:

[0024] Converting the image into a binary image;

[0025] identifying a switch contour and a switch action portion contour from the binary image;

[0026] The switch contour and the switch action portion contour are input into a pre-trained deep learning model to obtain a first recognition result.

[0027] In a preferred embodiment, the identifying the state of the switch action portion based on the image color includes:

[0028] Extracting the color of the switch area in the image;

[0029] determining, among the colors of the switch area, a color belonging to the switch action portion;

[0030] Querying a color-state correspondence table according to the color belonging to the switch action part;

[0031] A second recognition result is obtained according to the query result.

[0032] On the other hand, the embodiments of this specification also provide a switch state identification device, including:

[0033] A mark collector, used for collecting physical marks on the switch action part; the physical marks are used to indicate the current state of the switch action part;

[0034] The processor is used to obtain the entity tag, query the tag-state correspondence table according to the entity tag, and determine the state of the switch action unit based on the query result.

[0035] In a preferred embodiment, the entity mark is a color mark, and the mark collector is a color sensor.

[0036] In a preferred embodiment, the device further comprises:

[0037] An imaging unit, used for synchronously acquiring an image of the switch action unit;

[0038] The processor is also used to: perform image recognition on the image and determine the state of the switch action unit based on the image recognition result; confirm whether the state determined based on the image recognition result is consistent with the state determined based on the query result; if the state determined based on the image recognition result is consistent with the state determined based on the query result, then use it as the switch state recognition result.

[0039] In a preferred embodiment, the processor is further configured to:

[0040] If the state determined based on the image recognition result is inconsistent with the state determined based on the query result, the entity mark on the switch action part is re-acquired and re-recognized accordingly.

[0041] In a preferred embodiment, the synchronously performing image recognition on the switch action part and determining the state of the switch action part based on the image recognition result includes:

[0042] Acquiring an image of the switch action portion;

[0043] Recognize the state of the switch action part based on the image contour to obtain a first recognition result; and recognize the state of the switch action part based on the image color to obtain a second recognition result;

[0044] confirming whether the first recognition result is consistent with the second recognition result;

[0045] If the first recognition result is consistent with the second recognition result, it is regarded as a state determined based on the image recognition result.

[0046] In a preferred embodiment, the synchronously performing image recognition on the switch action part and determining the state of the switch action part based on the image recognition result further includes:

[0047] If the first recognition result is inconsistent with the second recognition result, the image of the switch action portion is reacquired and re-recognized based on the image.

[0048] In a preferred embodiment, the identifying the state of the switch action portion based on the image contour includes:

[0049] Converting the image into a binary image;

[0050] identifying a switch contour and a switch action portion contour from the binary image;

[0051] The switch contour and the switch action portion contour are input into a pre-trained deep learning model to obtain a first recognition result.

[0052] In a preferred embodiment, the identifying the state of the switch action portion based on the image color includes:

[0053] Extracting the color of the switch area in the image;

[0054] determining, among the colors of the switch area, a color belonging to the switch action portion;

[0055] Querying a color-state correspondence table according to the color belonging to the switch action part;

[0056] A second recognition result is obtained according to the query result.

[0057] On the other hand, an embodiment of the present specification further provides a patrol robot, wherein the patrol robot is equipped with the above-mentioned switch state recognition device.

[0058] On the other hand, an embodiment of the present specification further provides a computer storage medium on which a computer program is stored. When the computer program is run by a processor of the inspection robot, the instructions of the above method are executed.

[0059] From the technical solutions provided by the above embodiments of this specification, it can be seen that in the embodiments of this specification, since the physical mark is easier to identify than the switch image, the embodiments of this specification can improve the switch state recognition accuracy of the inspection robot. In addition, through multiple recognition, the embodiments of this specification further improve the switch state recognition accuracy of the inspection robot. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] In order to more clearly illustrate the technical solutions in the embodiments of this specification or the prior art, the following briefly introduces the drawings required for use in the embodiments or the prior art description. Obviously, the drawings described below are only some embodiments recorded in this specification. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative labor. In the drawings:

[0061] Figure 1 A flow chart showing a method for identifying a switch state in some embodiments of this specification is shown;

[0062] Figure 2 A partial structural schematic diagram of an inspection robot in some embodiments of this specification is shown;

[0063] Figure 3 A schematic diagram of the structure of a switch action portion in a closed state in an embodiment of the present specification is shown;

[0064] Figure 4a A schematic diagram of the structure of a switch action unit in an off state in an embodiment of the present specification is shown;

[0065] Figure 4b A schematic diagram of the structure of a switch action unit in another embodiment of the present specification in an off state is shown;

[0066] Figure 5 A flow chart showing a method for identifying a switch state in some other embodiments of this specification is shown;

[0067] Figure 6 A flowchart of determining a switch state based on image recognition in some embodiments of this specification is shown;

[0068] Figure 7 A schematic diagram showing a switch state prediction method using a Faster R-CNN network in an embodiment of the present specification is shown;

[0069] Figure 8 The structural block diagram of the inspection robot in some embodiments of this specification is shown.

[0070] [Description of Reference Numerals]

[0071] 1. Lifting rod;

[0072] 2. Steering PTZ;

[0073] 3. Tag collector;

[0074] 4. Imaging department;

[0075] 5. Switch body;

[0076] 6. Switch action part;

[0077] 802. Inspection robot;

[0078] 804, processor;

[0079] 806. Memory;

[0080] 808, driving mechanism;

[0081] 810, input / output interface;

[0082] 812. Input device;

[0083] 814. Output device;

[0084] 816. Presentation equipment;

[0085] 818. Graphical user interface;

[0086] 820, network interface;

[0087] 822, communication link;

[0088] 824. Communication bus. DETAILED DESCRIPTION

[0089] In order to enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below in conjunction with the drawings in the embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this specification, not all of the embodiments. Based on the embodiments in this specification, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of this specification. For example, in the following description, forming a second component above a first component may include an embodiment in which the first component and the second component are formed in direct contact, and may also include an embodiment in which the first component and the second component are formed in non-direct contact (that is, additional components may be included between the first component and the second component), etc.

[0090] Moreover, for ease of description, some embodiments of this specification may use spatially relative terms such as "above", "below", "top", "below", etc. to describe the relationship between one element or component and another (or other) elements or components as shown in the drawings of the embodiments. It should be understood that in addition to the orientations described in the drawings, the spatially relative terms are also intended to include different orientations of the device in use or operation. For example, if the device in the drawings is turned over, the elements or components described as "below" or "beneath" other elements or components will then be positioned "above" or "on" the other elements or components.

[0091] The embodiments of this specification relate to the technology of automatically inspecting the status of equipment (such as switches, etc.) in a data center using an inspection robot. For the convenience of description, the following uses a switch device (hereinafter referred to as a switch) as an example of an application scenario, but those skilled in the art should understand that the embodiments of this specification can be applied to any other suitable data center equipment status detection field, and this specification does not make a sole limitation to this, and specific selection can be made as needed.

[0092] When using an inspection robot to inspect switches, since the entire switch adopts one color, it is difficult to accurately determine the boundary of the switch action part during image recognition, resulting in a low state recognition rate of the switch action part and failure to meet the predetermined requirements.

[0093] In view of this, the embodiments of this specification have improved the inspection robot and made adaptive changes to the switch. Among them, the adaptive change to the switch means: a physical mark is set on the switch action part, and the physical mark is used to indicate the current state of the switch action part (that is, the physical mark can be used to identify whether the switch action part is in a closed state or an open state). Figure 1 As shown, in some embodiments of the present specification, the inspection robot may include a lifting rod 1, the free end of the lifting rod 1 is connected to a steering platform 2, and a mark collector 3 may be installed on the steering platform 2. When in use, the processor of the inspection robot can control the lifting rod 1 and the steering platform 2 to make the mark collector 3 face the switch action part of the target switch to collect the entity mark on the switch action part and provide it to the processor; the processor can query the mark and state correspondence table according to the entity mark, and determine the state of the switch action part based on the query result, thereby realizing the state recognition of the switch action part (that is, realizing the switch state recognition). Since the entity mark is easier to identify than the switch image, the embodiments of the present specification can improve the switch state recognition accuracy of the inspection robot.

[0094] In the embodiments of this specification, the physical mark may refer to a physical entity (i.e., a physical object) provided on the switch action part. The physical mark may be a pattern, color, or a combination thereof provided on the switch action part. Figure 3 As shown, the switch body 5 is provided with a switch action part 6, and the switch action part 6 is in a closed state. A color mark that can be significantly distinguished from the color of the switch body 5 and other parts of the switch action part 6 can be applied on the outer surface of the switch action part 6 in the closed state. Figure 4a As shown, the outer surface of the switch action part 6 in the closed state can be unchanged. In this way, the state of the switch action part can be identified by identifying whether there is a color mark. Of course, in another embodiment of this specification, combined with Figure 4bAs shown, the outer surface of the switch action part 6 in the disconnected state may also be coated with another color mark that can be significantly different from the color of the switch body 5 and other parts of the switch action part 6; obviously, the color mark on the outer surface of the switch action part 6 in the disconnected state should be significantly different from the color mark on the outer surface of the switch action part 6 in the closed state. In this way, the state of the switch action part can be identified by identifying different color marks.

[0095] It should be pointed out that the above-mentioned facade of the switch action part in the closed state and the facade in the closed state are only exemplary descriptions. This specification does not limit the setting position of the physical mark on the switch action part. The physical mark can be set at any position on the switch action part that can facilitate the identification of its state, and the specific position can be selected according to needs.

[0096] It should also be noted that this specification does not limit the coating area of ​​the physical mark, and the specific area can be selected according to needs; relatively speaking, the larger the coating area, the easier it is to identify, but the higher the coating cost. For example, in an embodiment of this specification, the coating area of ​​the physical mark can be as follows: Figure 3 , Figure 4a and Figure 4b For example, in another embodiment of the present specification, the physical mark may only cover a portion (for example, in the middle of the covered facade).

[0097] In some embodiments of this specification, the mark collector 3 on the inspection robot can be configured according to the adaptability of the entity mark. For example, when the entity mark is a color mark, the mark collector 3 can be a color sensor. A color sensor is also called a color sensor or a color recognition sensor. It is a sensor that detects the color by comparing the color of an object with a previously taught reference color. When the two colors match within a certain error range, the detection result (i.e., the color detection result of the target object) is output.

[0098] In some embodiments of the present specification, the switch action part is an actionable part in the switch and is visible from the outside, and the state conversion control of the switch can be realized through the action of the action part. When the action part performs the closing action, the switch action part is converted to the closed state, and when the action part performs the opening action, the switch action part is converted to the opening state. In some embodiments of the present specification, the action of the switch action part can be realized by automatic control or by manual control, or part of the control is realized by automatic control (for example, opening), and the other part of the control (for example, closing) is realized by manual control. In one embodiment of the present specification, a typical switch action part can be, for example, a knife switch part of a switch.

[0099] In some embodiments of this specification, a tag-state correspondence table may be stored in the memory of the inspection robot. The tag-state correspondence table records the one-to-one correspondence between the entity tag of each switch and the corresponding state. Therefore, by querying the correspondence table, the switch state corresponding to the currently obtained entity tag can be determined. In an exemplary embodiment of this specification, taking color tags as an example, the tag-state correspondence table can be shown in Table 1 below.

[0100] Table 1

[0101] Switch marking Switch action color coding Switch Status Switch 1 yellow closure Switch 1 blue disconnect Switch 2 grey closure Switch 2 black disconnect Switch 3 yellow closure Switch 3 blue disconnect … … …

[0102] For the inspection robot, it can identify different switches by spatial position (or other information), and identify its status on this basis. That is, the switch position information of each switch that needs to be detected in the data center can be pre-stored in the memory of the inspection robot. When the inspection robot finds a switch, it can determine the position information of the switch through the positioning device it carries, and then match it with the pre-stored switch position information, that is, it can identify which switch it is.

[0103] Please continue to refer to Figure 1 As shown, in some embodiments of the present specification, an imaging unit 4 may be further installed on the steering platform 2 of the inspection robot. The imaging unit 4 may be used to synchronously capture images of the switch action unit. Accordingly, the processor of the inspection robot may also perform image recognition on the image captured by the imaging unit 4, and determine the state of the switch action unit based on the image recognition result; confirm whether the state determined based on the image recognition result is consistent with the state determined based on the query result; if the state determined based on the image recognition result is consistent with the state determined based on the query result, it is used as the switch state recognition result; of course, if the state determined based on the image recognition result is inconsistent with the state determined based on the query result, the entity mark on the switch action unit may be re-acquired and re-identified accordingly. In this way, through dual recognition detection, the accuracy of switch state recognition can be further improved.

[0104] In some embodiments of this specification, the imaging unit 4 can be a device that can collect the target image and convert it into an image signal (or image data) suitable for the processor of the inspection robot to process. In one embodiment of this specification, the typical imaging unit 4 can be, for example, a camera. When in use, the processor of the inspection robot can control the lifting rod 1 and the steering platform 2 to make the imaging unit 4 face the switch action part of the target switch, so as to collect the image of the switch action part. In this specification, the imaging unit 4 synchronously collects the image of the switch action part, which is relative to the mark collector 3, that is, the mark collector 3 and the imaging unit 4 can work at the same time, so that the state determined based on the image recognition result and the state determined based on the query result are the states of the same switch action part at the same time, which is conducive to improving the accuracy of switch state recognition. Of course, considering that in most scenarios, the state change of the switch action part is not frequent, the synchronization in the embodiment of this specification means that a certain time deviation can be allowed.

[0105] This manual also provides a switch state recognition method, which can be applied to inspection robots. Figure 2 As shown, in some embodiments of this specification, the switch state identification method may include the following steps:

[0106] S201. Acquire a physical mark on a switch action part; the physical mark is used to indicate a current state of the switch action part.

[0107] S202: Query a tag-state correspondence table according to the entity tag.

[0108] S203: Determine the state of the switch action unit based on the query result.

[0109] In the embodiments of the present specification, since the physical mark is easier to identify than the switch image, the switch state recognition method of the embodiments of the present specification can improve the switch state recognition accuracy of the inspection robot.

[0110] In an embodiment of the present specification, the physical mark on the switch action part may be obtained by passively receiving or actively reading the physical mark on the switch action part collected by the mark collector by the processor of the inspection robot. In some embodiments of the present specification, the physical mark may be a color mark.

[0111] Taking color marking as an example, combined with Figure 5 As shown, in some embodiments of this specification, the switch state identification method may include the following steps:

[0112] S51, obtaining a color mark of a switch action portion, and synchronously obtaining an image of the switch action portion.

[0113] In the embodiments of the present specification, the color mark may be acquired by a color sensor of the inspection robot, and the image may be acquired by an imaging unit such as a camera of the inspection robot.

[0114] S52, querying a color-state correspondence table according to the color mark, and determining the state of the switch action part based on the query result; at the same time, image recognition can also be performed on the switch action part, and the state of the switch action part can be determined based on the image recognition result.

[0115] S53, confirm whether the state determined based on the image recognition result is consistent with the state determined based on the query result. If the state determined based on the image recognition result is consistent with the state determined based on the query result, execute step S54; otherwise, jump to the step of obtaining the color mark of the switch action part in S51, that is, re-obtain the physical mark on the switch action part and re-identify it accordingly.

[0116] In various embodiments of this specification, the state consistency means the same state. For example, if the state determined based on the image recognition result is a closed state, and the state determined based on the query result is also a closed state, then the state determined based on the image recognition result is considered to be consistent with the state determined based on the query result, that is, both are closed states.

[0117] S54: If the state determined based on the image recognition result is consistent with the state determined based on the query result, use it as a switch state recognition result.

[0118] In this way, the accuracy of switch state recognition is further improved through double recognition detection.

[0119] refer to Figure 6 As shown, the above-mentioned performing image recognition on the switch action part and determining the state of the switch action part based on the image recognition result may include the following steps:

[0120] S61, acquiring an image of the switch action portion.

[0121] S62, identifying the state of the switch action part based on the image contour to obtain a first recognition result; and identifying the state of the switch action part based on the image color to obtain a second recognition result.

[0122] S63, confirm whether the first recognition result is consistent with the second recognition result. If the first recognition result is consistent with the second recognition result, execute step S64; otherwise, jump to execute step S61, that is, when the first recognition result is inconsistent with the second recognition result, the image of the switch action part can be re-acquired and re-recognized accordingly.

[0123] S64, using the recognition result as the state determined based on the image recognition result. In the embodiments of this specification, consistent recognition results refer to the same recognition results; therefore, when the recognition results are consistent, the first recognition result or the second recognition result can be used as the state determined based on the image recognition result.

[0124] In this way, the accuracy of image recognition can be improved through dual image recognition, which is conducive to further improving the accuracy of switch state recognition.

[0125] In conjunction with the dotted line portion on the left side in step S62, the identifying the state of the switch action portion based on the image contour may include:

[0126] Convert the image into a grayscale image; for example, the image may be converted into a grayscale image by a weighted average method or the like;

[0127] Convert the grayscale image into a binary image; that is, by selecting a suitable threshold, convert the grayscale image into an image with only two grayscale values ​​of 0 and 255. For example, the grayscale image can be converted into a binary image by using a region adaptive method or the like;

[0128] identifying a switch contour and a switch action portion contour from the binary image;

[0129] The switch contour and the switch action portion contour are input into a pre-trained deep learning model to obtain a first recognition result.

[0130] In one embodiment of the specification, the pre-trained deep learning model may include, but is not limited to, the Faster R-CNN network. Compared with the R-CNN network, which requires a large amount of overlap between candidate frames in an image to facilitate the extraction of redundant feature operations; the Faster R-CNN network can directly send the entire image into the deep network after normalization, and then send it to the candidate area extracted from the image. The first few layers of features of these candidate areas do not need to be repeatedly calculated, thereby improving the training and testing speed. At the same time, unlike the independent classifiers and regressors in the R-CNN network that require a large number of features as training samples, Faster R-CNN unifies the category judgment and position fine-tuning with a deep network, and no additional storage is required. The Faster R-CNN network first frames the switch contour and the switch action part contour. When the switch action part contour is identified to be above the switch contour, it is concluded that the switch is in a closed state; when the switch action part contour is identified to be below the switch contour, it is concluded that the switch is in an open state.

[0131] Combination Figure 7 As shown in the figure, the principle of using Faster R-CNN to predict the switch state is as follows:

[0132] First, input a switch image of any size M×N into the CNN network, and forward propagate through the CNN network to the last shared convolutional layer. On the one hand, a feature map for input to the Region Proposal Network (RPN) is obtained, and on the other hand, it continues to forward propagate to the unique convolutional layer to generate a higher-dimensional feature map. The feature map for input to the RPN network is passed through the RPN network to obtain region proposals and region scores, and non-maximum suppression (for example, a threshold of 0.7) is used for the region score, and the region proposals of its Top-N scores are output to the Region of Interest (RoI) pooling layer. The region proposal is the prediction box identified by the algorithm and the more accurate prediction boxes obtained based on the offset, and the prediction boxes are removed for crossing the boundary and overlapping boxes. The region score retains the local maximum score for each prediction box scene with non-maximum suppression not exceeding 0.7. The prediction boxes with the top N local maximum score values ​​are extracted and enter the next layer of the network.

[0133] The obtained high-dimensional feature map and the output region proposal are simultaneously input into the RoI pooling layer to extract the features of the corresponding region proposal.

[0134] After the obtained region proposal features pass through the fully connected layer, the classification score of the region and the regressed bounding box (bbox) are output. The bbox is the position offset of the switch action prediction box. The final prediction box position can be determined based on the offset. The current state of the switch action part can be judged through the position, and whether the switch action part is closed (i.e., closed).

[0135] In conjunction with the dotted line portion on the right side in step S62, the identifying the state of the switch action portion based on the image color may include:

[0136] Extract the color of the switch area in the image; for example, a computer vision algorithm such as OpenCV can be used to extract the color distribution of the entire switch area including the switch color, the switch action part color, the background color, etc. from the image.

[0137] Determine the color of the switch area, which color belongs to the switch action part; according to the color distribution of the entire switch area, the color of the switch action part can be identified;

[0138] Querying a color-state correspondence table according to the color belonging to the switch action part;

[0139] A second recognition result is obtained based on the query result; for example, when the color of the switch action part is identified as a closed color, the switch action part is judged to be in a closed state at this time; when the color of the switch action part is identified as an open color, the switch action part is judged to be in an open state at this time.

[0140] Although the process flows described above include multiple operations that occur in a particular order, it should be clearly understood that the processes may include more or fewer operations, which may be performed sequentially or in parallel (eg, using parallel processors or a multi-threaded environment).

[0141] The embodiments of this specification provide a switch state recognition device, which can be configured on a patrol robot to form the patrol robot of the embodiments of this specification. In some embodiments of this specification, the switch state recognition device may include a tag collector and a processor. Among them, the tag collector can be used to collect entity tags on the switch action part; the entity tag is used to indicate the current state of the switch action part. The processor can be used to obtain the entity tag, query the tag and state correspondence table according to the entity tag, and determine the state of the switch action part based on the query result.

[0142] In some embodiments of the present specification, the entity mark is a color mark, and the mark collector is a color sensor.

[0143] In some embodiments of the present specification, the switch state recognition device may further include an imaging unit. The imaging unit may be used to synchronously capture an image of the switch action unit. Correspondingly, the processor may further be used to: perform image recognition on the image, and determine the state of the switch action unit based on the image recognition result; confirm whether the state determined based on the image recognition result is consistent with the state determined based on the query result; if the state determined based on the image recognition result is consistent with the state determined based on the query result, use it as the switch state recognition result; if the state determined based on the image recognition result is inconsistent with the state determined based on the query result, re-acquire the entity mark on the switch action unit and re-recognize it accordingly.

[0144] In some embodiments of the present specification, the synchronously performing image recognition on the switch action part and determining the state of the switch action part based on the image recognition result may include:

[0145] Acquiring an image of the switch action portion;

[0146] Recognize the state of the switch action part based on the image contour to obtain a first recognition result; and recognize the state of the switch action part based on the image color to obtain a second recognition result;

[0147] confirming whether the first recognition result is consistent with the second recognition result;

[0148] If the first recognition result is consistent with the second recognition result, it is regarded as a state determined based on the image recognition result.

[0149] In some embodiments of the present specification, the synchronously performing image recognition on the switch action part and determining the state of the switch action part based on the image recognition result further includes:

[0150] If the first recognition result is inconsistent with the second recognition result, the image of the switch action portion is reacquired and re-recognized based on the image.

[0151] In some embodiments of this specification, the identifying the state of the switch action part based on the image contour includes:

[0152] Converting the image into a binary image;

[0153] identifying a switch contour and a switch action portion contour from the binary image;

[0154] The switch contour and the switch action portion contour are input into a pre-trained deep learning model to obtain a first recognition result.

[0155] In some embodiments of this specification, the identifying the state of the switch action unit based on the image color includes:

[0156] Extracting the color of the switch area in the image;

[0157] determining, among the colors of the switch area, a color belonging to the switch action portion;

[0158] Querying a color-state correspondence table according to the color belonging to the switch action part;

[0159] A second recognition result is obtained according to the query result.

[0160] For the convenience of description, the above device is described in various units according to their functions. Of course, when implementing this specification, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0161] like Figure 8As shown, in some embodiments of the present specification, the inspection robot 802 may include one or more processors 804, such as one or more central processing units (CPUs) or graphics processing units (GPUs), and each processing unit may implement one or more hardware threads. The inspection robot 802 may also include any memory 806, which is used to store any kind of information such as code, settings, data, etc. In a specific embodiment, the computer program on the memory 806 and can be run on the processor 804, when the computer program is run by the processor 804, the instructions of the switch state recognition method described in any of the above embodiments can be executed. Non-limiting, for example, the memory 806 may include any one or more combinations of the following: any type of RAM, any type of ROM, flash memory device, hard disk, optical disk, etc. More generally, any memory can use any technology to store information. Further, any memory can provide volatile or non-volatile retention of information. Further, any memory can represent a fixed or removable component of the inspection robot 802. In one case, when the processor 804 executes an associated instruction stored in any memory or a combination of memories, the inspection robot 802 can perform any operation of the associated instruction. The inspection robot 802 also includes one or more drive mechanisms 808 for interacting with any storage, such as a hard disk drive mechanism, an optical disk drive mechanism, etc.

[0162] The inspection robot 802 may also include an input / output interface 810 (I / O) for receiving various inputs (via input devices 812) and for providing various outputs (via output devices 814). A specific output mechanism may include a presentation device 816 and an associated graphical user interface 818 (GUI). In other embodiments, the input / output interface 810 (I / O), input device 812, and output device 814 may not be included, and the robot may be used as only one inspection robot in the network. The inspection robot 802 may also include one or more network interfaces 820 for exchanging data with other devices via one or more communication links 822. One or more communication buses 824 couple the components described above together. Among them, the input device 812 may include, but is not limited to, a camera and a color sensor.

[0163] The communication link 822 may be implemented in any manner, for example, through a local area network, a wide area network (e.g., the Internet), a point-to-point connection, etc., or any combination thereof. The communication link 822 may include any combination of hardwired links, wireless links, routers, gateway functions, name servers, etc. governed by any protocol or combination of protocols.

[0164] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products of some embodiments of the present specification. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processor to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processor generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0165] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processor to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0166] These computer program instructions can also be loaded onto a computer or other programmable data processor so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0167] In a typical configuration, the inspection robot includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0168] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0169] Computer readable media include permanent and non-permanent, removable and non-removable media that can store information by any method or technology. Information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by the inspection robot. As defined in this article, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carriers.

[0170] Those skilled in the art will appreciate that the embodiments of this specification may be provided as methods, systems or computer program products. Therefore, the embodiments of this specification may take the form of complete hardware embodiments, complete software embodiments or embodiments combining software and hardware. Moreover, the embodiments of this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0171] The present specification embodiments may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present specification embodiments may also be practiced in distributed computing environments where tasks are performed by remote processors connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0172] Each embodiment in this specification is described in a progressive manner, and the same and similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment. In the description of this specification, the description of the reference terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the embodiment of this specification. In this specification, the schematic representation of the above terms does not necessarily target the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, in the absence of contradiction, a person skilled in the art can combine and combine the different embodiments or examples described in this specification and the features of different embodiments or examples.

[0173] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. A switch state identification method, characterized in that: include: Acquire a physical mark on the switch action part and an image of the switch action part; the physical mark is a physical entity set on the switch action part, used to indicate the current state of the switch action part; Query a table of correspondence between a tag and a state according to the entity tag; Determine the state of the switch action unit based on the query result; performing image recognition on the image of the switch action portion, and determining a state of the switch action portion based on a result of the image recognition; confirming whether the state determined based on the image recognition result is consistent with the state determined based on the query result, and if so, taking it as the switch state recognition result; The performing of image recognition on the image of the switch action part and determining the state of the switch action part based on the image recognition result includes: Recognize the state of the switch action part based on the image contour to obtain a first recognition result; and recognize the state of the switch action part based on the image color to obtain a second recognition result; confirming whether the first recognition result is consistent with the second recognition result; If the first recognition result is consistent with the second recognition result, it is regarded as a state determined based on the image recognition result.

2. The switch state identification method according to claim 1, characterized in that: The entity markings are color markings.

3. The switch state identification method according to claim 1, characterized in that: Also includes: If the state determined based on the image recognition result is inconsistent with the state determined based on the query result, the entity mark on the switch action part is re-acquired and re-recognized accordingly.

4. The switch state identification method according to claim 1, characterized in that: The performing image recognition on the image of the switch action part and determining the state of the switch action part based on the image recognition result further includes: If the first recognition result is inconsistent with the second recognition result, the image of the switch action portion is reacquired and re-recognized based on the image.

5. The switch state identification method according to claim 1, characterized in that: The identifying the state of the switch action part based on the image contour includes: Converting the image into a binary image; identifying a switch contour and a switch action portion contour from the binary image; The switch contour and the switch action portion contour are input into a pre-trained deep learning model to obtain a first recognition result.

6. The switch state identification method according to claim 1, characterized in that: The identifying the state of the switch action unit based on the image color includes: Extracting the color of the switch area in the image; determining, among the colors of the switch area, a color belonging to the switch action portion; Querying a color-state correspondence table according to the color belonging to the switch action part; A second recognition result is obtained according to the query result.

7. A switch state identification device, characterized in that: include: A mark collector, used to collect physical marks on the switch action part and images of the switch action part; the physical mark is a physical entity set on the switch action part, used to indicate the current state of the switch action part; A processor, configured to obtain the entity tag, query a tag-state correspondence table according to the entity tag, and determine the state of the switch action unit based on the query result; performing image recognition on the image of the switch action portion, and determining a state of the switch action portion based on a result of the image recognition; confirming whether the state determined based on the image recognition result is consistent with the state determined based on the query result, and if so, taking it as the switch state recognition result; The performing of image recognition on the image of the switch action part and determining the state of the switch action part based on the image recognition result includes: Recognize the state of the switch action part based on the image contour to obtain a first recognition result; and recognize the state of the switch action part based on the image color to obtain a second recognition result; confirming whether the first recognition result is consistent with the second recognition result; If the first recognition result is consistent with the second recognition result, it is regarded as a state determined based on the image recognition result.

8. The switch state recognition device according to claim 7, characterized in that: The entity mark is a color mark, and the mark collector is a color sensor.

9. The switch state recognition device according to claim 7, characterized in that: The processor is further configured to: If the state determined based on the image recognition result is inconsistent with the state determined based on the query result, the entity mark on the switch action part is re-acquired and re-recognized accordingly.

10. The switch state recognition device according to claim 7, characterized in that: The performing image recognition on the image of the switch action part and determining the state of the switch action part based on the image recognition result further includes: If the first recognition result is inconsistent with the second recognition result, the image of the switch action portion is reacquired and re-recognized based on the image.

11. The switch state recognition device according to claim 7, characterized in that: The identifying the state of the switch action part based on the image contour includes: Converting the image into a binary image; identifying a switch contour and a switch action portion contour from the binary image; The switch contour and the switch action portion contour are input into a pre-trained deep learning model to obtain a first recognition result.

12. The switch state recognition device according to claim 7, characterized in that: The identifying the state of the switch action unit based on the image color includes: Extracting the color of the switch area in the image; determining, among the colors of the switch area, a color belonging to the switch action portion; Querying a color-state correspondence table according to the color belonging to the switch action part; A second recognition result is obtained according to the query result.

13. A patrol robot, characterized in that: The inspection robot is equipped with a switch state recognition device as described in any one of claims 7-12.

14. A computer storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by the processor of the inspection robot, the computer program executes the instructions of the method according to any one of claims 1 to 6.

Images