Substation switch state recognition method and device, computer device, and storage medium

Abstract

The application relates to a substation switch state recognition method and device, computer equipment, a storage medium and a computer program product. The method comprises the following steps: acquiring an inspection image collected by an inspection device in a preset collection area; performing an image alignment operation on the inspection image and a standard image based on an inspection key point of the inspection image and a standard key point of the standard image, to obtain an aligned inspection image; acquiring a region of interest of the aligned inspection image, performing knob switch angle recognition on a knob switch image in the region of interest, and obtaining a knob switch state of a substation. The method can effectively avoid the situation that the inspection image collected by the inspection device and the standard image have deviations, so that the corresponding target cannot be correctly detected, and the substation switch state recognition precision is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of robots, in particular to a substation switch state recognition method and device, computer equipment, a storage medium and a computer program product. BACKGROUND

[0002] Periodic inspection of the electrical control cabinet of the substation is an important step to ensure the safe and reliable operation of the substation. Currently, the substation mainly uses manual inspection, suspended rail inspection robots and ground inspection robots to monitor the operating state of the electrical cabinet switch. Among them, the manual method requires the staff to spend a lot of manpower and material resources to periodically inspect the electrical control cabinet; the suspended rail inspection robot is complex to lay the guide rail, the inspection line is fixed, and it is difficult to transform and replace; the ground inspection robot uses laser navigation or visual navigation, can move autonomously and flexibly, is easy to deploy, periodically automatically inspects the key power equipment, reduces the cost of manual inspection and timely detects equipment failure.

[0003] However, due to the limitations of current robot navigation technology, using an inspection robot to recognize the state of the switch of the substation can easily cause the images collected by the robot to deviate from the standard images, so that the inspection robot cannot correctly detect the corresponding target, and the recognition accuracy of the switch state is poor. SUMMARY

[0004] Therefore, it is necessary to provide a substation switch state recognition method, device, computer equipment, computer readable storage medium and computer program product capable of improving the recognition accuracy of the switch state of the substation.

[0005] In a first aspect, the present application provides a substation switch state recognition method. The method comprises:

[0006] acquiring an inspection image collected by an inspection device in a preset collection area;

[0007] performing an image alignment operation on the inspection image and a standard image based on an inspection key point of the inspection image and a standard key point of the standard image, to obtain an aligned inspection image;

[0008] acquiring a region of interest of the aligned inspection image, and performing knob switch angle recognition on a knob switch image in the region of interest to obtain a knob switch state of the substation.

[0009] In one embodiment, the method further comprises:

[0010] obtain a homography matrix of the inspection image and the standard image based on the inspection key points of the inspection image and the standard key points of the pre-deployed standard image, the homography matrix being used to represent a corresponding relationship between the inspection key points and the standard key points;

[0011] obtain a region of interest of the standard image, the region of interest of the standard image having a label frame of a rotary switch pre-deployed thereon;

[0012] perform perspective transformation on the inspection image according to the homography matrix and the label frame of the rotary switch, map the inspection image onto the standard image, and obtain an aligned inspection image.

[0013] In one of the embodiments, the obtaining of the homography matrix of the inspection image and the standard image based on the inspection key points of the inspection image and the standard key points of the pre-deployed standard image comprises:

[0014] obtain initial inspection key points of the inspection image and standard key points of the pre-deployed standard image;

[0015] calculate a similarity between the initial inspection key points and the standard key points, and determine key points with a similarity greater than a preset similarity threshold as the inspection key points;

[0016] obtain the homography matrix of the inspection image and the standard image based on the inspection key points of the inspection image and the standard key points of the pre-deployed standard image.

[0017] In one of the embodiments, the obtaining of the homography matrix of the inspection image and the standard image based on the inspection key points of the inspection image and the standard key points of the pre-deployed standard image comprises:

[0018] perform a specific image extraction operation on each of the inspection key points and the standard key points of the pre-deployed standard image to obtain an inspection key image corresponding to each of the inspection key points and a standard key image corresponding to each of the standard key points;

[0019] perform an image matching operation according to the inspection key image and the standard key image to obtain a matching score of each matching key group;

[0020] determine a target matching key group from matching key groups with a matching score greater than a preset score threshold;

[0021] obtain the homography matrix of the inspection image and the standard image based on each of the target matching key groups.

[0022] In one of the embodiments, the knob switch angle recognition on the knob switch image in the region of interest obtains the knob switch state of the substation, which comprises:

[0023] The region of interest is subjected to fuzzy and binaryzation processing to obtain a processed region of interest image;

[0024] The switch connected region of the processed region of interest image is searched;

[0025] The switch connected region angle is calculated through the geometric second moment to obtain the knob switch angle;

[0026] The knob switch angle is compared with a preset angle range, and the knob switch state of the substation is obtained according to the comparison result.

[0027] In one of the embodiments, after the step of obtaining the region of interest of the aligned inspection image, the method further comprises:

[0028] The region of interest is input into a preset neural network to identify whether there is a knob switch image in the region of interest;

[0029] If not, a prompt information is generated to prompt that the image alignment fails, and the switch state recognition is ended.

[0030] In a second aspect, the application further provides a substation switch state recognition device. The device comprises:

[0031] An image acquisition module is configured to acquire an inspection image collected by an inspection device in a preset collection area;

[0032] An image alignment module is configured to perform an image alignment operation on the inspection image and a standard image based on an inspection key point of the inspection image and a standard key point of the standard image to obtain an aligned inspection image;

[0033] A switch state recognition module is configured to obtain a region of interest of the aligned inspection image, and to perform knob switch angle recognition on a knob switch image in the region of interest to obtain a knob switch state of the substation.

[0034] In a third aspect, the application further provides a computer device. The computer device comprises a memory and a processor, the memory stores a computer program, and the processor implements the steps of the substation switch state recognition method when executing the computer program.

[0035] In a fourth aspect, the application further provides a computer readable storage medium. The computer readable storage medium stores a computer program, and the computer program is executed by a processor to implement the steps of the substation switch state recognition method.

[0036] In a fifth aspect, the present application also provides a computer program product. The computer program product comprises a computer program which, when executed by a processor, implements the steps of the substation switch state recognition method.

[0037] The substation switch state recognition method, device, computer device, storage medium and computer program product described above, after obtaining the inspection image collected by the inspection device, align the inspection image and the standard image based on the inspection key points of the inspection image and the standard key points of the pre-deployed standard image, obtain the aligned inspection image, and recognize the knob switch angle of the knob switch image in the region of interest to obtain the knob switch state of the substation. The deviation between the inspection image collected by the inspection device and the standard image is effectively avoided, so that the situation that the corresponding target cannot be correctly detected is avoided, and the substation switch state recognition accuracy is improved. BRIEF DESCRIPTION OF DRAWINGS

[0038] Figure 1 It is an application environment diagram of the substation switch state recognition method in an embodiment;

[0039] Figure 2 It is a flowchart of the substation switch state recognition method in an embodiment;

[0040] Figure 3 It is a flowchart of the step of performing alignment operation on the inspection image and the standard image based on the inspection key points of the inspection image and the standard key points of the pre-deployed standard image to obtain the aligned inspection image in an embodiment;

[0041] Figure 4 It is a flowchart of the step of obtaining the homography matrix of the inspection image and the standard image based on the inspection key points of the inspection image and the standard key points of the pre-deployed standard image in an embodiment;

[0042] Figure 5 It is a flowchart of the step of obtaining the homography matrix of the inspection image and the standard image based on the inspection key points of the inspection image and the standard key points of the pre-deployed standard image in another embodiment;

[0043] Figure 6 It is a flowchart of the step of recognizing the knob switch angle of the knob switch image in the region of interest to obtain the knob switch state of the substation in an embodiment;

[0044] Figure 7 It is a flowchart of the substation switch state recognition method in another embodiment;

[0045] Figure 8A structural block diagram of a substation switch state recognition device in an embodiment;

[0046] Figure 9 An internal structural diagram of a computer device in an embodiment. DETAILED DESCRIPTION

[0047] In order to make the purposes, technical solutions and advantages of the present application clearer, the present application is further described in detail below with reference to the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and not to limit the present application.

[0048] The substation switch state recognition method provided by the embodiments of the present application can be applied to an application environment as shown in Figure 1 . The inspection device 102 communicates with the recognition platform 104 through a network. The data storage system can store data required to be processed by the recognition platform 104. The data storage system can be integrated on the recognition platform 104, or placed on a cloud or other network server. The recognition platform 104 obtains an inspection image collected by the inspection device 102 in a preset collection area, performs an image alignment operation on the inspection image and a standard image based on inspection key points of the inspection image and standard key points of the standard image, obtains an aligned inspection image, obtains a region of interest of the aligned inspection image, performs knob switch angle recognition on a knob switch image in the region of interest, and obtains a knob switch state of the substation. The inspection device 102 can be, but is not limited to, various devices capable of image collection, such as various inspection robots. The recognition platform 104 can be implemented by an independent server or a server cluster composed of multiple servers.

[0049] In an embodiment, as shown in Figure 2 , a substation switch state recognition method is provided. Taking the recognition platform in Figure 1 as an example, the method comprises the following steps:

[0050] Step 202: Obtain an inspection image collected by an inspection device in a preset collection area.

[0051] The inspection device is a device for inspecting and collecting images in the preset collection area of the substation. The inspection device can be equipped with common navigation functions, such as laser navigation, radar navigation or visual navigation functions, and can move flexibly and autonomously in the preset collection area of the substation. The inspection device is also configured with corresponding image collection components, such as color cameras, infrared cameras, etc., which can collect corresponding inspection images while inspecting the preset collection area of the substation for subsequent analysis by the staff.

[0052] In one of the embodiments, the image acquisition component of the inspection device can also be used in combination with a stabilizer such as a gimbal.

[0053] The preset acquisition area is an image acquisition area deployed by the staff in advance according to the inspection requirements, which can be regarded as a preset fixed-point acquisition area. In order to maximize the guarantee that the inspection images collected by the inspection device can contain the to-be-recognized area, the staff has previously deployed a fixed image acquisition area at a fixed position in the inspection area. When there is a need to identify the status of the substation switch, the inspection device is controlled to move to the preset acquisition area for image acquisition, which can improve the possibility of containing the identification area in the inspection image collected by the inspection device.

[0054] Specifically, when there is a need to identify the status of the substation switch, the staff controls the inspection device to move to the preset acquisition area, controls the inspection device to adjust to the image acquisition posture, and performs image acquisition on the target device of the substation to obtain an inspection image. The identification platform acquires the inspection image collected by the inspection device.

[0055] In one of the embodiments, after acquiring the inspection image, the identification platform can first perform noise reduction processing on the inspection image.

[0056] In step 204, based on the inspection key points of the inspection image and the standard key points of the standard image, the image alignment operation is performed on the inspection image and the standard image to obtain the aligned inspection image.

[0057] The standard image is an optimal image collected by the staff in advance through the inspection device in the preset acquisition area. The optimal image can reflect the best angle, distance, and shooting height for shooting the target identification device of the substation.

[0058] The inspection key points are feature points that can reflect the significant features of the inspection image, and the standard key points of the standard image are feature points that can reflect the significant features of the standard image. It can be understood that the key points can be sift, surf, fastfeature, etc.

[0059] In one of the embodiments, the inspection key points of the inspection image are acquired by the identification platform in real time from the inspection image.

[0060] In one of the embodiments, the standard key points of the standard image can be acquired by the identification platform in real time from the standard image, or can be acquired directly after the staff deploys the standard image, and then stored in the identification platform for subsequent direct acquisition and use.

[0061] In one of the embodiments, when the identification platform acquires the key points of the image, the image can be first down-sampled to facilitate the speed of subsequent key point extraction.

[0062] Specifically, the recognition platform acquires the inspection key points of the inspection image, acquires the standard key points of the pre-deployed standard image, performs image alignment operation on the inspection image and the standard image based on the inspection key points of the inspection image and the standard key points of the standard image, and obtains the aligned inspection image.

[0063] In step 206, the region of interest of the aligned inspection image is acquired, the knob switch image in the region of interest is subjected to knob switch angle recognition, and the knob switch state of the substation is obtained.

[0064] The region of interest (ROI) is a region that needs to be processed in machine vision and image processing, which is outlined by a box, a circle, an ellipse or an irregular polygon, and the image in this region is the focus of image analysis. By circling the target that needs to be focused on, the subsequent processing time can be effectively reduced and the processing accuracy can be improved.

[0065] Specifically, the staff labels the region of interest in the standard image when deploying the standard image. After aligning the inspection image and the standard image, the region of interest of the inspection image can be determined according to the region of interest of the standard image. The recognition platform acquires the region of interest of the aligned inspection image, subjects the knob switch image contained in the region of interest to knob switch angle recognition, obtains the angle of the knob switch of the substation, and thus determines the state of the knob switch of the substation.

[0066] In the above substation switch state recognition method, after the inspection image collected by the inspection equipment is acquired, the inspection image and the standard image are subjected to alignment operation based on the inspection key points of the inspection image and the standard key points of the pre-deployed standard image, the aligned inspection image is obtained, the knob switch angle recognition is performed based on the aligned inspection image, and the state of the knob switch of the substation is obtained. This effectively avoids the situation that the inspection image collected by the inspection equipment and the standard image have deviations, which leads to incorrect detection of the corresponding target, and improves the recognition accuracy of the substation switch state.

[0067] Image alignment operation is an effective operation to solve the possible deviation of the image collected by the inspection equipment. In an embodiment, as shown in Figure 3 Based on the inspection key points of the inspection image and the standard key points of the pre-deployed standard image, the aligned inspection image is obtained by performing alignment operation on the inspection image and the standard image, including:

[0068] In step 302, based on the inspection key points of the inspection image and the standard key points of the pre-deployed standard image, a homography matrix of the inspection image and the standard image is obtained, and the homography matrix is used to represent the corresponding relationship between the inspection key points and the standard key points.

[0069] Wherein, the homography matrix refers to a matrix expression of corresponding points on images of two different perspectives for projection transformation, and the homography matrix can represent the corresponding relationship between the inspection key points and the standard key points.

[0070] Specifically, after obtaining the inspection key points of the inspection image and the standard key points of the standard image, since the inspection image and the standard image are both images collected by the inspection device in the preset image collection area, they can be considered as two images of different perspectives. Therefore, the recognition platform can obtain the homography matrix according to the inspection key points and the standard key points, so as to represent the corresponding relationship between the inspection key points and the standard key points.

[0071] Step 304: Obtain the region of interest of the standard image, and the label frame of the knob switch is pre-deployed on the region of interest of the standard image.

[0072] Specifically, when deploying the standard image, the staff labels the region of interest of the labeled image by using the label frame on the knob switch image part in the standard image. The recognition platform obtains the region of interest of the labeled image.

[0073] Step 306: Perform perspective transformation on the inspection image according to the homography matrix and the label frame of the knob switch, map the inspection image onto the standard image, and obtain the aligned inspection image.

[0074] Specifically, the recognition platform performs perspective transformation on the inspection image according to the obtained homography matrix, maps the inspection image with the transformed perspective onto the standard image, and obtains the aligned inspection image. It can be understood that the aligned inspection image has a corresponding region of interest.

[0075] In this embodiment, the homography matrix of the inspection image and the standard image is obtained through the inspection key points and the standard key points. The inspection image is transformed in perspective based on the homography matrix and projected onto the standard image. The corresponding region of interest of the inspection image is obtained according to the standard frame of the region of interest of the standard image, which improves the accuracy of determining the region of interest of the inspection image and provides a data basis for subsequent substation switch state recognition based on the region of interest of the inspection image.

[0076] Further, in one embodiment, as shown in Figure 4 the homography matrix of the inspection image and the standard image is obtained based on the inspection key points of the inspection image and the pre-deployed standard key points of the standard image, including:

[0077] Step 402: Obtain the initial inspection key points of the inspection image and the pre-deployed standard key points of the standard image.

[0078] Among them, the initial inspection key points are key points extracted directly from the inspection images.

[0079] Specifically, the recognition platform extracts key points from the inspection images to obtain the initial inspection key points of the inspection images, and at the same time obtains the standard key points of the pre-deployed standard images.

[0080] In one embodiment, the recognition platform uses FastFeature to extract key feature points from the inspection image to obtain the initial inspection key points of the inspection image.

[0081] Step 404: Calculate the similarity between the initial inspection key points and the standard key points, and determine the key points with similarity greater than the preset similarity threshold as inspection key points.

[0082] The preset similarity threshold is used to determine whether the initial inspection key points and the standard key points are mismatched. The preset similarity threshold is set by the designer in advance based on experience or actual situation.

[0083] Specifically, the identification platform uses a similarity algorithm to match the initial inspection key points with standard key points, calculates the similarity between the initial inspection key points and standard key points, compares the similarity with a preset similarity threshold, and identifies key points with a similarity greater than the preset similarity threshold as inspection key points.

[0084] In one embodiment, the identification platform can perform similarity matching based on the similarity of the feature vectors of the initial inspection key points and the standard key points.

[0085] Step 406: Based on the inspection key points of the inspection image and the standard key points of the pre-deployed standard image, obtain the homography matrix of the inspection image and the standard image.

[0086] Specifically, the recognition platform can obtain the homography matrix of the inspection image and the standard image based on the inspection key points and the standard key points, so as to represent the correspondence between the inspection key points and the standard key points.

[0087] In the above embodiments, after the recognition platform extracts the initial inspection key points of the inspection image, it performs similarity matching between the initial inspection key points and standard key points, removes key points with similarity less than a preset similarity threshold, and determines key points with similarity greater than the preset similarity threshold as inspection key points. Based on the selected inspection key points and standard key points, a homography matrix is ​​obtained. By performing similarity screening on the initial inspection key points, inaccurate key point extraction and key point matching errors can be effectively avoided, thus improving the accuracy of key point extraction.

[0088] To obtain a more accurate homography matrix, in one embodiment, such asFigure 5 As shown, based on the inspection key points of the inspection image and the standard key points of the pre-deployed standard image, a homography matrix of the inspection image and the standard image is obtained, including:

[0089] In step 502, a specific image extraction operation is performed on each inspection key point and the standard key point of the pre-deployed standard image to obtain an inspection key image corresponding to each inspection key point and a standard key image corresponding to each standard key point.

[0090] The specific image extraction operation refers to image extraction centered on each key point according to a preset image size. It can be understood that, in order to better perform image matching operations subsequently, the preset image size is different according to different key points, and the inspection key point corresponds to a preset size of the inspection key image, and the standard key point corresponds to a preset size of the standard key image. For example, the preset size of the inspection key image is width w = 64 and height h = 64, and the preset size of the standard key image is width w = 32 and height h = 32.

[0091] Specifically, the recognition platform performs image extraction operations centered on each inspection key point according to the preset size of the inspection key image to obtain an inspection key image corresponding to each inspection key point, and performs image extraction operations centered on each standard key point according to the preset size of the standard key image to obtain a standard key image corresponding to each standard key point. It can be understood that the inspection key image and the standard key image can be regarded as the region of interest of the inspection key point and the standard key point.

[0092] In step 504, image matching operations are performed according to the inspection key image and the standard key image to obtain a matching score of each matching key group.

[0093] Specifically, image matching operations are performed with the standard key image as a template image and the inspection key image as a search image, the recognition platform searches for the standard key image in the inspection key image to obtain a matching score of each matching key group after the inspection key image and the standard key image are matched, and the matching score is used to measure the similarity between the inspection key image and the standard key image.

[0094] In step 506, a matching key group with a matching score greater than a preset score threshold is determined as a target matching key group.

[0095] The preset score threshold is a preset threshold parameter used to evaluate whether the inspection key image and the standard key image match. If the matching score is less than the preset score threshold, it means that the inspection key image and the standard key image corresponding to the current matching key group are not similar and cannot be matched. If the matching score is greater than the preset score threshold, it means that the inspection key image and the standard key image corresponding to the current matching key group are similar enough and can be matched.

[0096] Specifically, the recognition platform compares the matching scores of the matched key groups with a preset score threshold, and retains the matched key groups with the matching scores greater than the preset score threshold as target matched key groups.

[0097] At step 508, the homography matrix of the inspection image and the standard image is obtained based on the target matched key groups.

[0098] Specifically, the recognition platform determines the homography matrix of the inspection image and the standard image based on the target matched key groups.

[0099] In this embodiment, the recognition platform can further filter the extracted inspection key points by obtaining the inspection key images of the inspection key points and the standard key images of the standard key points, and respectively matching the inspection key images and the standard key images, and retaining the target matched key groups with the matching scores greater than the preset score threshold, so as to improve the accuracy of key point extraction. Furthermore, the homography matrix of the inspection image and the standard image obtained based on the target matched key groups provides a data basis for subsequent acquisition of an accurate inspection image region of interest.

[0100] After the region of interest of the inspection image is determined, the knob switch state of the substation can be identified. In one embodiment, as shown in FIG. 6A, the knob switch angle of the knob switch image in the region of interest is identified to obtain the knob switch state of the substation, including: Figure 6

[0101] At step 602, the region of interest is subjected to fuzzy processing and binarization processing to obtain a processed region of interest image.

[0102] The fuzzy processing is a processing step for reducing noise interference of the image of the region of interest.

[0103] The binarization processing is a simplest method of image segmentation. The binarization can convert a gray-scale image into a binary image. The pixel gray scale greater than a certain critical gray scale value is set as a gray scale maximum value, and the pixel gray scale less than the value is set as a gray scale minimum value, so as to realize the binarization.

[0104] Specifically, the recognition platform subjects the image of the region of interest to fuzzy processing to reduce noise interference, and then subjects the image of the region of interest to binarization processing according to the obtained binarization threshold to obtain a processed region of interest image.

[0105] In one embodiment, the recognition platform obtains the binarization threshold by using the maximum inter-class variance of mask.

[0106] ​In one of the embodiments, the recognition platform removes burrs of the image of the region of interest by using morphological processing after binarizing the image of the region of interest, to obtain the processed image of the region of interest.

[0107] In step 604, the recognition platform searches for the switch-connected region in the processed image of the region of interest.

[0108] Specifically, the recognition platform searches for the switch-connected region in the middle of the image of the region of interest from the four directions of up, down, left and right, and screens out the switch region according to the area.

[0109] In step 606, the recognition platform calculates the angle of the switch-connected region by using the geometric second moment, to obtain the angle of the rotary switch.

[0110] The geometric moment of the image is an operator used to describe the features of the image, wherein the first moment and the zero moment can be used to calculate the center of gravity of a shape, and the second moment can be used to calculate the direction of the shape.

[0111] Specifically, the recognition platform calculates the angle of the switch-connected region in the image of the region of interest by using the geometric second moment of the image of the region of interest, to obtain the angle of the rotary switch.

[0112] In step 608, the recognition platform compares the angle of the rotary switch with a preset angle range, and obtains the state of the rotary switch of the substation according to the comparison result.

[0113] The preset angle range is a rotary angle range set by the designer according to the state of the rotary switch of the substation in advance, and can be used to determine whether the state of the rotary switch of the substation is normal.

[0114] Specifically, the recognition platform compares the angle of the rotary switch with the preset angle range, and obtains the state of the rotary switch of the substation according to the comparison result.

[0115] In one of the embodiments, if the angle of the rotary switch exceeds the preset angle range, it is considered that the state of the rotary switch of the substation is an abnormal state, and the recognition platform generates an abnormal alarm information to report the abnormal result of the rotary switch of the substation to the staff.

[0116] In one of the embodiments, if the angle of the rotary switch does not exceed the preset angle range, it is considered that the state of the rotary switch of the substation is a normal state.

[0117] In the above embodiment, the recognition platform obtains the processed region of interest image by blurring and binarizing the region of interest of the inspection image, and can accurately obtain the knob switch angle of the substation by calculating the angle of the switch connected region in the region of interest image. By comparing the knob switch angle with the preset angle range, the knob switch state of the substation can be accurately recognized. The whole recognition process is efficient and accurate, and the substation switch state recognition accuracy is improved.

[0118] During the inspection image collection of the inspection recognition, there will inevitably be a large deviation. Based on this, in one embodiment, after the step of obtaining the region of interest of the aligned inspection image, the method further includes: inputting the region of interest into a preset neural network to identify whether there is a knob switch image in the region of interest; if not, generating a prompt information to prompt that the image alignment fails, and ending the switch state recognition.

[0119] The preset neural network is a pre-trained image recognition network for identifying whether there is a knob switch image in the input image.

[0120] Specifically, after obtaining the region of interest of the aligned inspection image, the recognition platform inputs the region of interest image into the preset neural network for image recognition to identify whether there is a knob switch image in the region of interest of the aligned inspection image. If not, it means that the inspection device has a large deviation during the inspection image collection, or the alignment fails during the image alignment operation. At this time, the recognition platform generates a prompt information to prompt the staff that the image alignment fails, and ends the switch state recognition operation.

[0121] If there is, it means that the region of interest of the inspection image contains a knob switch image at this time, and the subsequent switch state recognition operation can be performed.

[0122] In this embodiment, by identifying the knob switch image in the region of interest of the aligned inspection image, the staff can be prompted in time when the inspection device has a large deviation or the image alignment operation fails, so as to avoid the situation that the recognition error is caused by continuing the switch state recognition. At the same time, based on the generality of the model framework and the automatic learning feature of deep learning, the switch part features can be accurately classified even if they are blocked or incomplete, which can meet the requirements of classification accuracy in actual production, and will play an important role in the field of target device detection.

[0123] In one embodiment, as shown in Figure 7 , a substation switch state recognition method is provided. Taking the inspection robot as an example, the method specifically includes the following steps:

[0124] Firstly, the control inspection robot moves to a specified location, adjusts a gimbal carrying a camera to a specified height, and takes a picture of the target device to obtain an inspection image of the target device.

[0125] The recognition platform obtains the inspection image of the target device, and performs noise reduction processing on the inspection image to obtain a processed inspection image.

[0126] Key points are extracted from the inspection image and the pre-deployed standard image respectively. First, the inspection image and the standard image are down-sampled for acceleration. Then, FastFeature is used to extract feature key points in the images to obtain initial inspection key points of the inspection image and standard key points of the pre-deployed standard image. The similarity of the initial inspection key points and the standard key points is calculated, and the key points with a similarity greater than a preset similarity threshold are retained and determined as inspection key points.

[0127] The recognition platform obtains a specific size image for each retained key point to obtain an inspection key image corresponding to the inspection key point and a standard key image corresponding to the standard key point. Then, the inspection key image is taken as a search image, and the standard key image is taken as a template image. In the search image, a template matching method is used for searching to obtain a matching score of a matching key group composed of the inspection key image and the standard key image. The matching score is compared with a preset score threshold, and the matching key group with a matching score greater than the preset score threshold is retained. The matching key group with a matching score greater than the preset score threshold is determined as a target matching key group. Based on each target matching key group, a homography matrix of the inspection image and the standard image is obtained.

[0128] The recognition platform obtains a region of interest of the standard image labeled by a user in advance, performs perspective transformation on the inspection image according to the obtained homography matrix, maps the inspection image after the perspective transformation to the standard image with the labeled region of interest, and thus obtains a region of interest of the inspection image.

[0129] The region of interest of the inspection image is input into a pre-set neural network to identify whether there is a knob switch image in the region of interest of the inspection image. If not, it means that the inspection robot has a large position deviation or the image alignment fails. The recognition platform generates a prompt information to prompt the staff that the image alignment has failed, and ends the processing.

[0130] If there is a knob switch image in the region of interest of the inspection image, the region of interest of the inspection image is subjected to a blur processing to reduce noise interference. Then, a maximum inter-class variance with a mask is used to obtain a binary threshold value, the region of interest of the inspection image is subjected to a binary processing, and a morphological processing is used to remove burrs.

[0131] For the region of interest of the processed inspection image, the recognition platform searches for the switch connected region from the four directions of up, down, left and right to the middle, and finally screens out the switch region according to the area and calculates the angle of the connected region by the geometric second moment to obtain the knob switch angle.

[0132] The knob switch angle is compared with the preset angle range. If the knob switch angle does not exceed the preset angle range, it is determined that the switch state of the transformer substation is normal. If the knob switch angle exceeds the preset angle range, it is determined that the switch state of the transformer substation is abnormal, and the recognition platform generates an abnormal prompt information to inform the staff of the abnormal result.

[0133] The transformer substation switch state recognition method in the embodiment uses a key point matching strategy when aligning the image, has higher accuracy compared with other matching technologies, improves the accuracy of determining the target region of interest, and the deep learning-based technology in the method can use the same model, which can be easily extended to other target devices, can be efficiently reused, and can shorten the development cycle. When measuring the knob switch angle, the maximum inter-class variance with mask is used to obtain the binary threshold, which can better obtain the knob switch connected region, and the geometric second moment is used to calculate the connected domain angle, which has better robustness. The recognition accuracy of the robot for the knob switch meets the requirement of more than 98%, has better automation degree, and has wide application prospect.

[0134] It should be understood that, although each step in the flowchart involved in each embodiment as described above is displayed in sequence according to the arrow, these steps are not necessarily executed in sequence according to the arrow. Unless otherwise specified herein, the execution of these steps is not strictly limited in sequence, and these steps can be executed in other sequences. Moreover, at least part of the steps in the flowchart involved in each embodiment as described above can include multiple steps or stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution sequence of these steps or stages is not necessarily sequential, but can be executed in rotation or alternation with at least part of other steps or steps or stages in other steps.

[0135] Based on the same inventive concept, the embodiments of the present application also provide a transformer substation switch state recognition device for implementing the transformer substation switch state recognition method described above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme described in the above method, and therefore the specific limitations in one or more transformer substation switch state recognition device embodiments provided below can refer to the limitations of the transformer substation switch state recognition method described above, which will not be repeated here.

[0136] In one embodiment, as Figure 8As shown, a substation switch state recognition device 800 is provided, comprising: an image acquisition module 801, an image alignment module 802, and a switch state recognition module 803, wherein:

[0137] The image acquisition module 801 is configured to acquire an inspection image collected by the inspection device in a preset collection area.

[0138] The image alignment module 802 is configured to perform an image alignment operation on the inspection image and the standard image based on the inspection key points of the inspection image and the standard key points of the pre-deployed standard image, to obtain an aligned inspection image.

[0139] The switch state recognition module 803 is configured to acquire a region of interest of the aligned inspection image, perform knob switch angle recognition on a knob switch image in the region of interest, and obtain a knob switch state of the substation.

[0140] The above-mentioned substation switch state recognition device, after acquiring the inspection image collected by the inspection device, performs an alignment operation on the inspection image and the standard image based on the inspection key points of the inspection image and the standard key points of the pre-deployed standard image, to obtain an aligned inspection image. By performing knob switch angle recognition based on the aligned inspection image, the knob switch state of the substation is obtained, effectively avoiding the situation that the inspection image collected by the inspection device and the standard image have deviations, resulting in incorrect detection of the corresponding target, and improving the substation switch state recognition accuracy.

[0141] In one embodiment, the image alignment module is further configured to: based on the inspection key points of the inspection image and the standard key points of the pre-deployed standard image, obtain a homography matrix of the inspection image and the standard image, the homography matrix being used to represent the correspondence between the inspection key points and the standard key points; acquire a region of interest of the standard image, the region of interest of the standard image having a pre-deployed annotation box of the knob switch; and perform perspective transformation on the inspection image according to the homography matrix and the annotation box of the knob switch, to map the inspection image onto the standard image, to obtain the aligned inspection image.

[0142] In one embodiment, the image alignment module is further configured to: acquire initial inspection key points of the inspection image and standard key points of the pre-deployed standard image; calculate the similarity of the initial inspection key points and the standard key points, and determine the key points with a similarity greater than a preset similarity threshold as the inspection key points; and based on the inspection key points of the inspection image and the standard key points of the pre-deployed standard image, obtain a homography matrix of the inspection image and the standard image.

[0143] In one embodiment, the image alignment module is further configured to: perform specific image extraction operations on each inspection key point and the standard key points of a pre-deployed standard image to obtain inspection key images corresponding to each inspection key point and standard key images corresponding to each standard key point; perform image matching operations on the inspection key images and the standard key images to obtain matching scores for each matching key group; determine the matching key groups with matching scores greater than a preset score threshold as target matching key groups; and obtain the homography matrix between the inspection image and the standard image based on each target matching key group.

[0144] In one embodiment, the switch state recognition module is further configured to: perform fuzzing and binarization processing on the region of interest to obtain a processed region of interest image; search for the switch connected region in the processed region of interest image; calculate the angle of the switch connected region using the second geometric moment to obtain the knob switch angle; compare the knob switch angle with a preset angle range, and obtain the knob switch state of the substation based on the comparison result.

[0145] In one embodiment, the substation switch status recognition device further includes: a region of interest recognition module, used to input the region of interest into a preset neural network to identify whether a rotary switch image exists in the region of interest; if not, a prompt message is generated indicating that the image alignment failed and the switch status recognition ends.

[0146] Each module in the aforementioned substation switch status identification device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of a computer device in hardware form or independent of it, or stored in the memory of a computer device in software form, so that the processor can call and execute the corresponding operations of each module.

[0147] In one embodiment, a computer device is provided, which may be a server integrated with an identification platform, and its internal structure diagram may be as follows: Figure 9 As shown, the computer device includes a processor, memory, and network interface connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores data such as inspection images, standard images, inspection key points, standard key points, and regions of interest. The network interface is used for communication with external terminals via a network connection. When executed by the processor, the computer program implements a substation switch status identification method.

[0148] Those skilled in the art will understand that Figure 9The structure shown in the figure is only a block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the computer device to which the scheme of the present application is applied. The specific computer device can include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0149] In one embodiment, a computer device is provided, comprising a memory and a processor, the memory storing a computer program, and the processor implements the following steps when executing the computer program:

[0150] Obtaining an inspection image collected by the inspection device in a preset collection area;

[0151] Performing an image alignment operation on the inspection image and the standard image based on the inspection key points of the inspection image and the standard key points of the standard image to obtain an aligned inspection image;

[0152] Obtaining a region of interest of the aligned inspection image, and performing knob switch angle recognition on a knob switch image in the region of interest to obtain a knob switch state of the transformer substation.

[0153] In one embodiment, the processor further implements the following steps when executing the computer program:

[0154] Obtaining a homography matrix of the inspection image and the standard image based on the inspection key points of the inspection image and the standard key points of the standard image, the homography matrix being used to represent a corresponding relationship between the inspection key points and the standard key points;

[0155] Obtaining a region of interest of the standard image, and the region of interest of the standard image having a label frame of the knob switch pre-deployed thereon;

[0156] Performing perspective transformation on the inspection image according to the homography matrix and the label frame of the knob switch to map the inspection image onto the standard image to obtain an aligned inspection image.

[0157] In one embodiment, the processor further implements the following steps when executing the computer program:

[0158] Obtaining initial inspection key points of the inspection image and standard key points of the standard image pre-deployed;

[0159] Calculating a similarity between the initial inspection key points and the standard key points, and determining a key point with a similarity greater than a preset similarity threshold as an inspection key point;

[0160] Obtaining a homography matrix of the inspection image and the standard image based on the inspection key points of the inspection image and the standard key points of the standard image pre-deployed.

[0161] In one embodiment, the processor, when executing the computer program, also implements the following steps:

[0162] performing a specific image extraction operation on each inspection key point and the standard key point of the pre-deployed standard image to obtain an inspection key image corresponding to each inspection key point and a standard key image corresponding to each standard key point;

[0163] performing an image matching operation on the inspection key image and the standard key image to obtain a matching score of each matching key group;

[0164] determining a target matching key group from the matching key group whose matching score is greater than a preset score threshold;

[0165] obtaining a homography matrix of the inspection image and the standard image based on each target matching key group.

[0166] In one embodiment, the processor, when executing the computer program, also implements the following steps:

[0167] performing blur and binarization processing on the region of interest to obtain a processed region of interest image;

[0168] searching for a switch-connected region of the processed region of interest image;

[0169] calculating a knob switch angle by a geometric second moment to obtain the knob switch angle;

[0170] comparing the knob switch angle with a preset angle range to obtain a knob switch state of the substation according to a comparison result.

[0171] In one embodiment, the processor, when executing the computer program, also implements the following steps:

[0172] inputting the region of interest into a preset neural network to identify whether a knob switch image exists in the region of interest;

[0173] if not, generating a prompt information to prompt that image alignment fails, and ending the switch state identification.

[0174] In one embodiment, a computer readable storage medium is provided, and the computer readable storage medium stores a computer program. The computer program is executed by a processor to implement the following steps:

[0175] obtaining an inspection image collected by an inspection device in a preset collection area;

[0176] performing an image alignment operation on the inspection image and a standard image based on an inspection key point of the inspection image and a standard key point of the standard image to obtain an aligned inspection image;

[0177] An interest region of the aligned inspection image is obtained, knob switch angle recognition is performed on a knob switch image in the interest region, and a knob switch state of the transformer substation is obtained.

[0178] In one embodiment, the computer program, when executed by the processor, further implements the following steps:

[0179] Based on the inspection key points of the inspection image and the standard key points of the pre-deployed standard image, a homography matrix of the inspection image and the standard image is obtained, and the homography matrix is used to represent the corresponding relationship between the inspection key points and the standard key points.

[0180] An interest region of the standard image is obtained, and a label box of a knob switch is pre-deployed on the interest region of the standard image.

[0181] According to the homography matrix and the label box of the knob switch, perspective transformation is performed on the inspection image, the inspection image is mapped onto the standard image, and an aligned inspection image is obtained.

[0182] In one embodiment, the computer program, when executed by the processor, further implements the following steps:

[0183] Obtaining initial inspection key points of the inspection image and standard key points of the pre-deployed standard image.

[0184] The similarity between the initial inspection key points and the standard key points is calculated, and the key points with a similarity greater than a preset similarity threshold are determined as the inspection key points.

[0185] Based on the inspection key points of the inspection image and the standard key points of the pre-deployed standard image, a homography matrix of the inspection image and the standard image is obtained.

[0186] In one embodiment, the computer program, when executed by the processor, further implements the following steps:

[0187] Performing a specific image extraction operation on each inspection key point and the standard key point of the pre-deployed standard image to obtain an inspection key image corresponding to each inspection key point and a standard key image corresponding to each standard key point.

[0188] According to the inspection key image and the standard key image, an image matching operation is performed to obtain a matching score of each matching key group.

[0189] The matching key groups with a matching score greater than a preset score threshold are determined as target matching key groups.

[0190] Based on each target matching key group, a homography matrix of the inspection image and the standard image is obtained.

[0191] In one embodiment, the computer program, when executed by the processor, further implements the following steps:

[0192] Blur and binarize the region of interest to obtain a processed region of interest image;

[0193] Search for a switch-connected region of the processed region of interest image;

[0194] Calculate the switch-connected region angle through the geometric second moment to obtain the knob switch angle;

[0195] Compare the knob switch angle with a preset angle range, and obtain the knob switch state of the substation according to the comparison result.

[0196] In one embodiment, the computer program, when executed by the processor, further implements the following steps:

[0197] Input the region of interest into a preset neural network to identify whether there is a knob switch image in the region of interest;

[0198] If not, generate a prompt information to prompt that the image alignment fails, and end the switch state identification.

[0199] In one embodiment, a computer program product is provided, comprising a computer program which, when executed by the processor, implements the following steps:

[0200] Obtain an inspection image collected by the inspection device in a preset collection area;

[0201] Perform an image alignment operation on the inspection image and the standard image based on the inspection key points of the inspection image and the standard key points of the standard image to obtain an aligned inspection image;

[0202] Obtain a region of interest of the aligned inspection image, and perform knob switch angle identification on a knob switch image in the region of interest to obtain a knob switch state of the substation.

[0203] In one embodiment, the computer program, when executed by the processor, further implements the following steps:

[0204] Based on the inspection key points of the inspection image and the standard key points of the standard image, obtain a homography matrix of the inspection image and the standard image, the homography matrix being used to represent the corresponding relationship between the inspection key points and the standard key points;

[0205] Obtain a region of interest of the standard image, and the region of interest of the standard image has a label frame of the knob switch pre-deployed thereon;

[0206] According to the homography matrix and the label frame of the knob switch, perform a perspective transformation on the inspection image to map the inspection image onto the standard image to obtain an aligned inspection image.

[0207] In one embodiment, the computer program, when executed by the processor, further implements the following steps:

[0208] Obtaining initial inspection key points of the inspection image and standard key points of the pre-deployed standard image;

[0209] Calculating the similarity between the initial inspection key points and the standard key points, and determining the key points with a similarity greater than a preset similarity threshold as the inspection key points;

[0210] Based on the inspection key points of the inspection image and the standard key points of the pre-deployed standard image, obtaining a homography matrix of the inspection image and the standard image.

[0211] In one embodiment, the computer program, when executed by the processor, further implements the following steps:

[0212] Performing a specific image extraction operation on each inspection key point and the standard key point of the pre-deployed standard image to obtain an inspection key image corresponding to each inspection key point and a standard key image corresponding to each standard key point;

[0213] According to the inspection key image and the standard key image, performing an image matching operation to obtain a matching score of each matching key group;

[0214] Determining a matching key group with a matching score greater than a preset score threshold as a target matching key group;

[0215] Based on each target matching key group, obtaining a homography matrix of the inspection image and the standard image.

[0216] In one embodiment, the computer program, when executed by the processor, further implements the following steps:

[0217] Performing blur and binarization processing on the region of interest to obtain a processed region of interest image;

[0218] Searching for a switch-connected region of the processed region of interest image;

[0219] Calculating the angle of the switch-connected region by the geometric second moment to obtain a knob switch angle;

[0220] Comparing the knob switch angle with a preset angle range to obtain the state of the knob switch of the substation according to the comparison result.

[0221] In one embodiment, the computer program, when executed by the processor, further implements the following steps:

[0222] Inputting the region of interest into a preset neural network to identify whether there is a knob switch image in the region of interest;

[0223] If not, generating a prompt information to prompt that the image alignment fails, and ending the switch state identification.

[0224] It should be noted that the user information (including but not limited to user equipment information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data authorized by the user or authorized by all parties.

[0225] It can be understood by those skilled in the art that all or part of the processes in the above-mentioned embodiment methods can be completed by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer readable storage medium. When the computer program is executed, it can include the processes of the above-mentioned embodiments. Any reference to memory, database or other medium used in the embodiments provided by the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (Read-Only Memory, ROM), magnetic tape, floppy disk, flash memory, optical storage, high-density embedded non-volatile memory, resistive memory (ReRAM), magnetoresistive memory (Magnetoresistive Random Access Memory, MRAM), ferroelectric memory (Ferroelectric Random Access Memory, FRAM), phase change memory (Phase Change Memory, PCM), graphene memory, etc. Volatile memory can include random access memory (Random Access Memory, RAM) or external cache memory, etc. As an illustration but not limitation, RAM can be in various forms, such as static random access memory (Static Random Access Memory, SRAM) or dynamic random access memory (Dynamic Random Access Memory, DRAM), etc. The database involved in the embodiments provided by the present application can include at least one of a relational database and a non-relational database. The non-relational database can include a distributed database based on a block chain, etc., without being limited thereto. The processor involved in the embodiments provided by the present application can be a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., without being limited thereto.

[0226] The technical features of the above embodiments can be combined in any way. In order to make the description concise, all possible combinations of the technical features in the above embodiments are not described, but as long as the combination of the technical features does not exist contradictory, it should be considered as the scope of the present application.

[0227] The above-described embodiments are merely illustrative of several embodiments of the present application, and the description is relatively specific and detailed, but should not be understood as a limitation on the scope of the patent. It should be noted that for those skilled in the art, without departing from the concept of the present application, a number of modifications and improvements can be made, which are all within the scope of the present application. Therefore, the scope of protection of the present application should be subject to the appended claims.

Claims

1. A method for identifying the status of a substation switch, characterized in that, The method includes: Acquire inspection images captured by the inspection equipment in the preset acquisition area; Obtain the initial key points of the inspection image and the standard key points of the pre-deployed standard image; Calculate the similarity between the initial inspection key points and the standard key points, and determine the key points with similarity greater than a preset similarity threshold as inspection key points; A specific image extraction operation is performed on each of the inspection key points and the standard key points of the pre-deployed standard image to obtain the inspection key image corresponding to each of the inspection key points and the standard key image corresponding to each of the standard key points; the specific image extraction operation refers to the operation of extracting images according to a preset image size with each key point as the center. Image matching is performed between the inspection key images and the standard key images to obtain the matching scores for each matching key group. The matching key groups whose matching scores are greater than a preset score threshold are identified as target matching key groups; Based on each of the target matching key groups, a homography matrix between the inspection image and the standard image is obtained. The homography matrix is ​​used to characterize the correspondence between the inspection key points and the standard key points. Obtain the region of interest of the standard image, on which a label box for a rotary switch is pre-deployed; Based on the homography matrix and the label box of the rotary switch, the inspection image is subjected to perspective transformation, and the inspection image is mapped onto the standard image to obtain an aligned inspection image. The region of interest (ROI) of the aligned inspection image is obtained, and the rotary switch image in the ROI is used to identify the rotary switch angle to obtain the rotary switch status of the substation.

2. The method according to claim 1, characterized in that, The step of identifying the rotary switch angle in the rotary switch image within the region of interest to obtain the rotary switch status of the substation includes: The region of interest is blurred and binarized to obtain the processed region of interest image; Search for the switched connected regions in the processed region of interest image; The angle of the rotary switch is obtained by calculating the angle of the connected region of the switch using the second-order geometric moment. The rotary switch angle is compared with a preset angle range, and the rotary switch status of the substation is obtained based on the comparison result.

3. The method according to claim 1, characterized in that, Following the step of acquiring the region of interest in the aligned inspection image, the method further includes: The region of interest is input into a preset neural network to identify whether a knob switch image exists in the region of interest; If it does not exist, a prompt message will be generated indicating that image alignment failed, and the switch status recognition will end.

4. The method according to claim 2, characterized in that, The step of comparing the rotary switch angle with a preset angle range and obtaining the rotary switch state of the substation based on the comparison result includes: If the rotary switch angle does not exceed the preset angle range, the rotary switch status of the substation is determined to be normal. If the rotary switch angle exceeds the preset angle range, the rotary switch state of the substation is determined to be abnormal, and an abnormal alarm message is generated.

5. A substation switch status identification device, characterized in that, The device includes: The image acquisition module is used to acquire inspection images collected by the inspection equipment in the preset acquisition area; An image alignment module is used to acquire initial inspection key points of the inspection image and standard key points of a pre-deployed standard image; calculate the similarity between the initial inspection key points and the standard key points, and determine key points with similarity greater than a preset similarity threshold as inspection key points; perform a specific image extraction operation on each inspection key point and the standard key points of the pre-deployed standard image to obtain inspection key images corresponding to each inspection key point and standard key images corresponding to each standard key point; the specific image extraction operation refers to the operation of extracting images centered on each key point according to a preset image size; and based on the inspection key images and the standard key images... The process involves performing image matching to obtain matching scores for each key matching group; identifying key matching groups with matching scores greater than a preset threshold as target key matching groups; obtaining a homography matrix between the inspection image and the standard image based on each target key matching group, where the homography matrix characterizes the correspondence between the inspection key points and the standard key points; acquiring the region of interest (ROI) of the standard image, where a pre-deployed bounding box for a rotary switch is located; and performing a perspective transformation on the inspection image based on the homography matrix and the bounding box of the rotary switch to map the inspection image onto the standard image, resulting in an aligned inspection image. The switch status recognition module is used to acquire the region of interest of the aligned inspection image, and to identify the rotary switch angle of the rotary switch image in the region of interest to obtain the rotary switch status of the substation.

6. The apparatus according to claim 5, characterized in that, The switch state recognition module is used to: perform blurring and binarization processing on the region of interest to obtain a processed region of interest image; Search for the switched connected regions in the processed region of interest image; The angle of the rotary switch is obtained by calculating the angle of the connected region of the switch using the second-order geometric moment. The rotary switch angle is compared with a preset angle range, and the rotary switch status of the substation is obtained based on the comparison result.

7. The apparatus according to claim 5, characterized in that, The device further includes: The region of interest (ROI) identification module is used to input the ROI into a preset neural network to identify whether a knob switch image exists in the ROI; if not, a prompt message is generated indicating that the image alignment failed and the switch status identification ends.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 4.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 4.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 4.

Images