Image recognition method and device, electronic device and storage medium
By intercepting the target image from the first image, performing key point detection and feature extraction, the problem of special hardware equipment for detecting string defects in contact networks in the prior art is solved, and low-cost string fracture detection without equipment is realized.
Patent Information
- Application Number
- CN202010983194.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-09-17
- Publication Date
- 2025-02-21
- Estimated Expiration
- 2040-09-17
AI Technical Summary
The prior art requires special hardware equipment to detect contact network hanging string defects, which is relatively expensive, and the number of hanging string defect data is small, and the characteristics are inconsistent, making it difficult to effectively model.
By intercepting the target image from the first image, key point detection and feature extraction are performed, determining whether the hanging string is broken, reducing the detection cost.
The need to use special equipment to determine whether the hanging string is broken, reducing the cost of defect detection and improving the accuracy of detection.
Smart Images
Figure CN112102298B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of image recognition technology, and in particular to an image recognition method and device, an electronic device and a storage medium. Background Art
[0002] Contact network suspension string defect detection technology has important applications in many fields of industry, especially in the high-speed rail field, where it can effectively protect the travel safety of passengers.
[0003] However, it is very difficult to model the defect detection of contact network suspension strings, mainly because the amount of data on suspension string defects is relatively small, and the same suspension string presents different characteristics in different scenarios. At present, the defect detection of contact network suspension strings is mainly carried out with the help of special hardware detection equipment. For example, the light of the line laser at both ends is irradiated on the suspension string of the anchor section to be detected to form a bright light area, and the industrial camera receives the reflected light of the suspension string of the anchor section to be detected and forms an image, and transmits the image to the system controller, and the system controller analyzes the image to realize the defect detection of the contact network suspension string.
[0004] Therefore, currently, defect detection of contact network suspension strings requires the use of special hardware equipment, which is relatively costly. Summary of the invention
[0005] The embodiments of the present application provide an image recognition method and device, an electronic device and a storage medium. By intercepting a target image from a first image to detect the breakage of a suspension string, the cost of detecting the defects of the suspension string is reduced.
[0006] In a first aspect, an embodiment of the present application provides an image recognition method, comprising:
[0007] Acquire a first image, wherein the first image includes the hanging string;
[0008] According to the first image, a target image is obtained, wherein the target image includes a catenary cable sling clamp and a portion of the sling connected to the catenary cable sling clamp;
[0009] It is determined whether the suspension string is broken according to the target image.
[0010] In some possible implementations, the target image includes a second image, the catenary string clamp is a first catenary string clamp, and obtaining the target image according to the first image includes:
[0011] Performing key point detection on the first image to obtain a first pixel point, where the first pixel point is a pixel point corresponding to the first catenary cable string clamp in the first image, and the first catenary cable string clamp is a catenary cable string clamp connected to the upper end of the suspension string;
[0012] The second image is extracted from the first image according to the first pixel point.
[0013] In some possible implementations, determining whether the suspension string is broken according to the target image includes:
[0014] Performing feature extraction on the second image to obtain a first feature map;
[0015] Obtaining a first probability of suspension string breakage according to the first characteristic graph;
[0016] It is determined whether the suspension string is broken according to the first probability.
[0017] In some possible implementations, the target image includes a third image, the catenary cable string clamp is a second catenary cable string clamp, and obtaining the target image according to the first image includes:
[0018] Performing key point detection on the first image to obtain a second pixel point, where the second pixel point is a pixel point corresponding to the second catenary cable string clamp in the first image, and the second catenary cable string clamp is a catenary cable string clamp connected to the lower end of the suspension string;
[0019] The third image is extracted from the first image according to the second pixel point.
[0020] In some possible implementations, determining whether the suspension string is broken according to the target image includes:
[0021] Performing feature extraction on the third image to obtain a second feature map;
[0022] According to the second characteristic graph, a second probability of the suspension string breaking is obtained;
[0023] According to the second probability, it is determined whether the suspension string is broken.
[0024] In some possible implementations, the target image includes a second image and a third image, the catenary cable string clamp includes a first catenary cable string clamp and a second catenary cable clamp, and obtaining the target image according to the first image includes:
[0025] Performing key point detection on the first image to obtain a first pixel point and a second pixel point, wherein the first pixel point is a pixel point corresponding to the first catenary cable string clamp in the first image, and the first catenary cable string clamp is a catenary cable string clamp connected to the upper end of the suspension string, and the second pixel point is a pixel point corresponding to the second catenary cable string clamp in the first image, and the second catenary cable string clamp is a catenary cable string clamp connected to the lower end of the suspension string;
[0026] According to the first pixel point, extracting the second image from the first image;
[0027] The third image is extracted from the first image according to the second pixel point.
[0028] In some possible implementations, determining whether the suspension string is broken according to the target image includes:
[0029] Performing feature extraction on the second image to obtain a first feature map;
[0030] Obtaining a first probability of suspension string breakage according to the first characteristic graph;
[0031] Performing feature extraction on the third image to obtain a second feature map;
[0032] According to the second characteristic graph, a second probability of the suspension string breaking is obtained;
[0033] It is determined whether the suspension string is broken according to the first probability and the second probability.
[0034] In some possible implementations, determining whether the suspension string is broken according to the first probability and the second probability includes:
[0035] determining a maximum probability of the first probability and the second probability;
[0036] According to the maximum probability, it is determined whether the suspension string is broken.
[0037] In some possible implementations, extracting the second image from the first image according to the first pixel point includes:
[0038] Starting from the first pixel point, a first number of pixel points are intercepted to the left, a second number of pixel points are intercepted to the right, and a third number of pixel points are intercepted downward to obtain the second image.
[0039] In some possible implementations, extracting the third image from the first image according to the second pixel point includes:
[0040] Starting from the first pixel point, a fourth number of pixel points are intercepted to the left, a fifth number of pixel points are intercepted to the right, and a sixth number of pixel points are intercepted downward to obtain the third image.
[0041] In some possible implementations, the first image is an original image or is obtained by performing target detection on the original image, wherein the original image is obtained by photographing a high-speed railway line, and when the first image is obtained by performing target detection on the original image, before acquiring the first image, the method further includes:
[0042] Performing object detection on the original image to obtain at least one candidate frame;
[0043] Identify the object corresponding to each candidate frame to obtain the candidate frame where the hanging string is located;
[0044] The image in the candidate frame where the hanging string is located is intercepted to obtain the first image.
[0045] In a second aspect, an embodiment of the present application provides an image recognition device, including:
[0046] A transceiver unit, configured to acquire a first image, wherein the first image includes the hanging string;
[0047] a processing unit, configured to obtain a target image according to the first image, wherein the target image includes a catenary cable string clamp and a portion of the string connected to the catenary cable string clamp;
[0048] The processing unit is further used to determine whether the suspension string is broken according to the target image.
[0049] In some possible implementations, the target image includes a second image, the catenary string clamp is a first catenary string clamp, and in obtaining the target image according to the first image, the processing unit is specifically configured to:
[0050] Performing key point detection on the first image to obtain a first pixel point, where the first pixel point is a pixel point corresponding to the first catenary cable string clamp in the first image, and the first catenary cable string clamp is a catenary cable string clamp connected to the upper end of the suspension string;
[0051] The second image is extracted from the first image according to the first pixel point.
[0052] In some possible implementations, in determining whether the suspension string is broken according to the target image, the processing unit is specifically configured to:
[0053] Performing feature extraction on the second image to obtain a first feature map;
[0054] Obtaining a first probability of suspension string breakage according to the first characteristic graph;
[0055] It is determined whether the suspension string is broken according to the first probability.
[0056] In some possible implementations, the target image includes a third image, the catenary cable string clamp is a second catenary cable string clamp, and in obtaining the target image according to the first image, the processing unit is specifically configured to:
[0057] Performing key point detection on the first image to obtain a second pixel point, where the second pixel point is a pixel point corresponding to the second catenary cable string clamp in the first image, and the second catenary cable string clamp is a catenary cable string clamp connected to the lower end of the suspension string;
[0058] The third image is extracted from the first image according to the second pixel point.
[0059] In some possible implementations, in determining whether the suspension string is broken according to the target image, the processing unit is specifically configured to:
[0060] Performing feature extraction on the third image to obtain a second feature map;
[0061] According to the second characteristic graph, a second probability of the suspension string breaking is obtained;
[0062] According to the second probability, it is determined whether the suspension string is broken.
[0063] In some possible implementations, the target image includes a second image and a third image, the catenary cable string clamp includes a first catenary cable string clamp and a second catenary cable clamp, and in obtaining the target image according to the first image, the processing unit is specifically configured to:
[0064] Performing key point detection on the first image to obtain a first pixel point and a second pixel point, wherein the first pixel point is a pixel point corresponding to the first catenary cable string clamp in the first image, and the first catenary cable string clamp is a catenary cable string clamp connected to the upper end of the suspension string, and the second pixel point is a pixel point corresponding to the second catenary cable string clamp in the first image, and the second catenary cable string clamp is a catenary cable string clamp connected to the lower end of the suspension string;
[0065] According to the first pixel point, extracting the second image from the first image;
[0066] The third image is extracted from the first image according to the second pixel point.
[0067] In some possible implementations, in determining whether the suspension string is broken according to the target image, the processing unit is specifically configured to:
[0068] Performing feature extraction on the second image to obtain a first feature map;
[0069] Obtaining a first probability of suspension string breakage according to the first characteristic graph;
[0070] Performing feature extraction on the third image to obtain a second feature map;
[0071] According to the second characteristic graph, a second probability of the suspension string breaking is obtained;
[0072] It is determined whether the suspension string is broken according to the first probability and the second probability.
[0073] In some possible implementations, in determining whether the suspension string is broken according to the first probability and the second probability, the processing unit is specifically configured to:
[0074] determining a maximum probability of the first probability and the second probability;
[0075] According to the maximum probability, it is determined whether the suspension string is broken.
[0076] In some possible implementations, in terms of extracting the second image from the first image according to the first pixel point, the processing unit is specifically configured to:
[0077] Starting from the first pixel point, a first number of pixel points are intercepted to the left, a second number of pixel points are intercepted to the right, and a third number of pixel points are intercepted downward to obtain the second image.
[0078] In some possible implementations, in terms of extracting the third image from the first image according to the second pixel point, the processing unit is specifically configured to:
[0079] Starting from the first pixel point, a fourth number of pixel points are intercepted to the left, a fifth number of pixel points are intercepted to the right, and a sixth number of pixel points are intercepted downward to obtain the third image.
[0080] In some possible embodiments, the first image is an original image or is obtained by performing target detection on the original image, wherein the original image is obtained by photographing a high-speed railway line, and in the case where the first image is obtained by performing target detection on the original image, before acquiring the first image, the processing unit is further used to perform target detection on the original image to obtain at least one candidate frame; identify the object corresponding to each candidate frame to obtain the candidate frame where the suspension string is located; and capture the image in the candidate frame where the suspension string is located to obtain the first image.
[0081] In a third aspect, an embodiment of the present application provides an electronic device, comprising: a processor, the processor is connected to a memory, the memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory, so that the electronic device performs the method described in the first aspect.
[0082] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and the computer program enables a computer to execute the method described in the first aspect.
[0083] In a fifth aspect, an embodiment of the present application provides a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and the computer is operable to cause the computer to execute the method described in the first aspect.
[0084] Implementing the embodiments of the present application has the following beneficial effects:
[0085] It can be seen that in the embodiment of the present application, by extracting the target image from the first image and identifying the target image, it can be determined whether the suspension string is broken without the aid of special equipment, thereby reducing the cost of detecting suspension string defects. BRIEF DESCRIPTION OF THE DRAWINGS
[0086] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.
[0087] Figure 1 A flowchart of an image recognition method provided in an embodiment of the present application;
[0088] Figure 2 A schematic diagram of a suspension string provided in an embodiment of the present application;
[0089] Figure 3A schematic diagram of a neural network provided in an embodiment of the present application;
[0090] Figure 4 A schematic diagram of the structure of an image recognition device provided in an embodiment of the present application;
[0091] Figure 5 A block diagram of the functional units of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0092] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0093] The terms "first", "second", "third" and "fourth" etc. in the specification and claims of the present application and the drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally includes steps or units that are not listed, or optionally includes other steps or units inherent to these processes, methods, products or devices.
[0094] Reference to "embodiments" herein means that a particular feature, result, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.
[0095] See also Figure 1 , Figure 1 A flowchart of an image recognition method provided in an embodiment of the present application. The method uses an image recognition device. The method of this embodiment includes the following steps:
[0096] 101: Acquire a first image, wherein the first image includes a suspension string.
[0097] The suspension string is a complete suspension string in the first image.
[0098] Exemplarily, the first image is an original image or is obtained by performing target detection on the first image. For example, target detection is performed on the original image to obtain at least one candidate frame; then, the objects corresponding to each candidate frame are classified to obtain the candidate frame where the hanging string is located, and the image in the candidate frame where the hanging string is located is intercepted to obtain the first image.
[0099] Among them, target detection on the original image can be achieved through existing models, for example, target detection on the original image can be performed through the FasterR-CNN model, the SSD model or the YOLO model; object classification and recognition can be achieved through existing network models, for example, objects can be classified through a convolutional network.
[0100] The original image is obtained by photographing the high-speed railway line, and includes the suspension string, and of course, the original image may also include other objects.
[0101] 102: Obtain a target image according to the first image, wherein the target image includes a portion of the suspension string, wherein the portion is a portion of the suspension string connected to a catenary suspension string clamp.
[0102] Exemplarily, the target image can be obtained by intercepting the first image. Specifically, the first image is intercepted mainly by intercepting the vicinity of the upper end of the suspension string and / or the vicinity of the lower end of the suspension string with the catenary cable string clamp as the center to obtain the target image. Therefore, the target image includes a part of the suspension string, that is, includes a part of the suspension string connected to the catenary cable string clamp.
[0103] Exemplarily, when the target image includes the second image, the catenary string clamp is the first catenary string clamp, and key point detection can be performed on the first image to obtain a first pixel point, which is the pixel point corresponding to the first catenary string clamp in the first image, and the first catenary string clamp is the catenary string clamp connected to the upper end of the string. Exemplarily, the first pixel point can be the pixel point corresponding to the center position of the first catenary string clamp, or the pixel point corresponding to the leftmost end or the left and right ends of the first catenary string clamp. Then, the second image is cut out from the first image based on the first pixel point.
[0104] Exemplarily, when the target image includes a third image, the catenary string clamp is a second catenary string clamp, and key point detection can be performed on the first image to obtain a second pixel point, which is a pixel point corresponding to the second catenary string clamp in the first image, and the second catenary string clamp is a catenary string clamp connected to the lower end of the string. Exemplarily, the second pixel point can be a pixel point corresponding to the center position of the second catenary string clamp, or a pixel point corresponding to the leftmost end or left and right ends of the second catenary string clamp. Then, the third image is cut out from the first image based on the second pixel point.
[0105] Exemplarily, in the case where the target image includes a second image and a third image, the catenary cable string includes a first catenary cable string clamp and a second catenary cable string clamp, and key point detection can be performed on the first image to obtain the first pixel point and the second pixel point; then, based on the second pixel point and the third pixel point, the second image and the third image can be respectively extracted from the first image.
[0106] Exemplarily, starting from the first pixel point, a first number of pixel points can be intercepted to the left, a second number of pixel points can be intercepted to the right, and a third number of pixel points can be intercepted downward to obtain the second image. The first number, the second number, and the third number can be the same or different, and the present application does not limit this. In addition, starting from the second pixel point, a fourth number of pixel points can be intercepted to the left, a fifth number of pixel points can be intercepted to the right, and a sixth number of pixel points can be intercepted downward to obtain the third image. The fourth number, the fifth number, and the sixth number can be the same or different, and the present application does not limit this.
[0107] like Figure 2 As shown, through key point detection, the first pixel point and the second pixel point are obtained, and the first pixel point and the second pixel point are pixel points corresponding to the center positions of the first load-bearing cable string clamp and the second load-bearing cable string clamp respectively; then, according to the above-mentioned interception method, the second image and the third image are respectively intercepted from the first image according to the first pixel point and the second pixel point.
[0108] The following takes the first pixel point as an example to illustrate the implementation process of key point detection for the first image. The detection of the second pixel point is similar to the detection of the first pixel point and will not be described again.
[0109] Exemplarily, feature extraction is performed on the first image to obtain a fourth feature map; then, deconvolution is performed on the fourth feature map to obtain a fifth feature map, wherein the dimension of the fifth feature map is the same as that of the first image; key point prediction is performed based on the fifth feature map to obtain first pixel coordinates; and the first pixel point is determined based on the first pixel coordinates.
[0110] 103: Determine whether the suspension string is broken according to the target image.
[0111] Exemplarily, feature extraction is performed on the target image to obtain a target feature map; based on the target feature map, a probability of the suspension string being broken is obtained; and based on the probability, whether the suspension string is broken is determined. For example, if the probability is greater than a threshold value (e.g., 0.5), it is determined that the suspension string is broken.
[0112] When the target image is the second image, feature extraction may be performed on the second image to obtain a first feature map; based on the first feature map, a first probability of the suspension string being broken may be obtained; based on the first probability, it may be determined whether the suspension string is broken.
[0113] When the target image is the third image, feature extraction may be performed on the third image to obtain a second feature map; based on the second feature map, a second probability of the suspension string being broken is obtained; and based on the second probability, it is determined whether the suspension string is broken.
[0114] In the case where the target image includes the second image and the third image, the first probability and the second probability are determined respectively; and whether the suspension string is broken is determined based on the first probability and the second probability. Exemplarily, the maximum probability of the first probability and the second probability can be determined, and whether the suspension string is broken is determined based on the maximum probability.
[0115] In practical applications, the first probability and the second probability may be subjected to weighted processing or linear processing or nonlinear processing to obtain a target probability, and whether the suspension string is broken may be determined based on the target probability.
[0116] It can be seen that in the embodiment of the present application, by intercepting the target image and identifying the target image, it can be determined whether the suspension string is broken, thereby reducing the cost of detecting the defects of the suspension string without the help of special equipment; in addition, when the suspension string is broken, due to the gravity factor, the load-bearing rope of the suspension string will fall down, causing the image of the area near the second pixel point to be different from the image of the normal suspension string (unbroken suspension string). In addition, if the suspension string breaks at the current-carrying ring of the suspension string, for example, the current-carrying ring at the upper end of the suspension string breaks, the rope at the current-carrying ring will fall down, which will cause the image of the area near the first pixel point to be different from the image of the normal suspension string. Therefore, in the present application, by acquiring the target image, in the case of a broken suspension string, a local image of the abnormal area can be accurately acquired. Compared with using the entire suspension string image for fracture detection, some useless information in the image is actively removed through prior knowledge, which can improve the accuracy of determining whether the suspension string is broken.
[0117] In some possible implementations, identifying whether the suspension string is broken can be achieved by a neural network. Figure 3As shown, the neural network includes a first network, a second network and a third network. The first network is used to detect the target of the original image, determine the area where the hanging string is located, and obtain the first image; the second network is used to detect key points of the first image to obtain the target image; the third network is used to classify the target image and determine whether the hanging string is broken.
[0118] Since the objectives to be achieved by the first network, the second network and the third network are quite different, the first network, the second network and the third network can be trained separately, and after the training is completed, the first network, the second network and the third network are jointly debugged to form the neural network.
[0119] The training process of these three networks is described in detail below.
[0120] Exemplarily, a hanging string image sample is obtained, wherein the hanging string in the hanging string image sample can be a normal hanging string or a broken hanging string, which is not limited in the present application; the hanging string image sample is input into a first network to obtain predicted hanging string area coordinates; according to the predicted hanging string area coordinates and the actual hanging string area coordinates, the parameter values of the network parameters of the first network are adjusted to obtain the first network.
[0121] In addition, before training the first network, the samples can be expanded using data augmentation methods such as brightness adjustment, flipping, and multi-scale to obtain a larger number of hanging string image samples. During the training process, the first network can be trained using online difficult example mining technology to improve the recognition accuracy of the first network.
[0122] Exemplarily, a second suspension string image sample is obtained, wherein the suspension string in the second suspension string image sample can be a normal suspension string or a broken suspension string, which is not limited in the present application; the second suspension string image sample is input into the second network to obtain the predicted pixel coordinates of the catenary cable suspension string clamp; according to the predicted pixel coordinates of the catenary cable suspension string clamp and the actual pixel coordinates of the catenary cable suspension string clamp, the parameter values of the network parameters of the second network are adjusted to obtain the second network.
[0123] Exemplarily, a third image sample is obtained, and the third image sample can be a positive sample or a negative sample. The third image sample is a local image sample cut out from the original image, wherein the specific cutting method is similar to the cutting method of the second image or the third image and is not described again; the third image sample is input into the third network to obtain the probability of the suspension string in the third image sample being broken; according to the probability and the actual situation of the suspension string in the third image sample, the parameter values of the network parameters of the third network are adjusted.
[0124] It should be noted that in practice, there are very few images of broken suspension strings. Therefore, before training the third network, the number of third image samples needs to be expanded. For example, normal image samples can be modified (for example, PS) to obtain samples of broken suspension strings. Because the third image samples are only local images, it is more conducive to image modification, and a larger number of third image samples are obtained, which solves the problem of lack of image samples of broken suspension strings in the prior art.
[0125] See also Figure 4 , Figure 4 The electronic device 400 includes a transceiver unit 401 and a processing unit 402, wherein:
[0126] The transceiver unit 401 is configured to acquire a first image, wherein the first image includes the hanging string;
[0127] A processing unit 402 is configured to obtain a target image according to the first image, wherein the target image includes a catenary cable string clamp and a portion of the string connected to the catenary cable string clamp;
[0128] The processing unit 402 is further configured to determine whether the suspension string is broken according to the target image.
[0129] In some possible implementations, the target image includes a second image, the catenary string clamp is a first catenary string clamp, and in obtaining the target image according to the first image, the processing unit 401 is specifically configured to:
[0130] Performing key point detection on the first image to obtain a first pixel point, where the first pixel point is a pixel point corresponding to the first catenary cable string clamp in the first image, and the first catenary cable string clamp is a catenary cable string clamp connected to the upper end of the suspension string;
[0131] The second image is extracted from the first image according to the first pixel point.
[0132] In some possible implementations, in determining whether the suspension string is broken according to the target image, the processing unit 402 is specifically configured to:
[0133] Performing feature extraction on the second image to obtain a first feature map;
[0134] Obtaining a first probability of suspension string breakage according to the first characteristic graph;
[0135] It is determined whether the suspension string is broken according to the first probability.
[0136] In some possible implementations, the target image includes a third image, the catenary string clamp is a second catenary string clamp, and in obtaining the target image according to the first image, the processing unit 402 is specifically configured to:
[0137] Performing key point detection on the first image to obtain a second pixel point, where the second pixel point is a pixel point corresponding to the second catenary cable string clamp in the first image, and the second catenary cable string clamp is a catenary cable string clamp connected to the lower end of the suspension string;
[0138] The third image is extracted from the first image according to the second pixel point.
[0139] In some possible implementations, in determining whether the suspension string is broken according to the target image, the processing unit 402 is specifically configured to:
[0140] Performing feature extraction on the third image to obtain a second feature map;
[0141] According to the second characteristic graph, a second probability of the suspension string breaking is obtained;
[0142] According to the second probability, it is determined whether the suspension string is broken.
[0143] In some possible implementations, the target image includes a second image and a third image, and the catenary cable string clamp includes a first catenary cable string clamp and a second catenary cable clamp. In obtaining the target image according to the first image, the processing unit 402 is specifically configured to:
[0144] Performing key point detection on the first image to obtain a first pixel point and a second pixel point, wherein the first pixel point is a pixel point corresponding to the first catenary cable string clamp in the first image, and the first catenary cable string clamp is a catenary cable string clamp connected to the upper end of the suspension string, and the second pixel point is a pixel point corresponding to the second catenary cable string clamp in the first image, and the second catenary cable string clamp is a catenary cable string clamp connected to the lower end of the suspension string;
[0145] According to the first pixel point, extracting the second image from the first image;
[0146] The third image is extracted from the first image according to the second pixel point.
[0147] In some possible implementations, in determining whether the suspension string is broken according to the target image, the processing unit 402 is specifically configured to:
[0148] Performing feature extraction on the second image to obtain a first feature map;
[0149] Obtaining a first probability of suspension string breakage according to the first characteristic graph;
[0150] Performing feature extraction on the third image to obtain a second feature map;
[0151] According to the second characteristic graph, a second probability of the suspension string breaking is obtained;
[0152] It is determined whether the suspension string is broken according to the first probability and the second probability.
[0153] In some possible implementations, in determining whether the suspension string is broken according to the first probability and the second probability, the processing unit 402 is specifically configured to:
[0154] determining a maximum probability of the first probability and the second probability;
[0155] According to the maximum probability, it is determined whether the suspension string is broken.
[0156] In some possible implementations, in terms of extracting the second image from the first image according to the first pixel point, the processing unit 402 is specifically configured to:
[0157] Starting from the first pixel point, a first number of pixel points are intercepted to the left, a second number of pixel points are intercepted to the right, and a third number of pixel points are intercepted downward to obtain the second image.
[0158] In some possible implementations, in terms of extracting the third image from the first image according to the second pixel point, the processing unit 402 is specifically configured to:
[0159] Starting from the first pixel point, a fourth number of pixel points are intercepted to the left, a fifth number of pixel points are intercepted to the right, and a sixth number of pixel points are intercepted downward to obtain the third image.
[0160] In some possible embodiments, the first image is an original image or is obtained by performing target detection on the original image, wherein the original image is obtained by photographing a high-speed railway line, and in the case where the first image is obtained by performing target detection on the original image, before acquiring the first image, the processing unit 402 is further used to perform target detection on the original image to obtain at least one candidate frame; identify the object corresponding to each candidate frame to obtain the candidate frame where the suspension string is located; and capture the image in the candidate frame where the suspension string is located to obtain the first image.
[0161] See also Figure 5 , Figure 5This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 5 As shown, the electronic device 500 includes a transceiver 501, a processor 502 and a memory 503. They are connected via a bus 504. The memory 503 is used to store computer programs and data, and can transmit the data stored in the memory 503 to the processor 502.
[0162] The processor 502 is used to read the computer program in the memory 503 and perform the following operations:
[0163] Acquire a first image, wherein the first image includes the hanging string;
[0164] According to the first image, a target image is obtained, wherein the target image includes a catenary cable sling clamp and a portion of the sling connected to the catenary cable sling clamp;
[0165] It is determined whether the suspension string is broken according to the target image.
[0166] In some possible implementations, the target image includes a second image, the catenary string clamp is a first catenary string clamp, and in obtaining the target image according to the first image, the processor 502 is specifically configured to perform the following operations:
[0167] Performing key point detection on the first image to obtain a first pixel point, where the first pixel point is a pixel point corresponding to the first catenary cable string clamp in the first image, and the first catenary cable string clamp is a catenary cable string clamp connected to the upper end of the suspension string;
[0168] The second image is extracted from the first image according to the first pixel point.
[0169] In some possible implementations, in determining whether the suspension string is broken according to the target image, the processor 502 is specifically configured to perform the following operations:
[0170] Performing feature extraction on the second image to obtain a first feature map;
[0171] Obtaining a first probability of suspension string breakage according to the first characteristic graph;
[0172] It is determined whether the suspension string is broken according to the first probability.
[0173] In some possible implementations, the target image includes a third image, the catenary string clamp is a second catenary string clamp, and in obtaining the target image according to the first image, the processor 502 is specifically configured to perform the following operations:
[0174] Performing key point detection on the first image to obtain a second pixel point, where the second pixel point is a pixel point corresponding to the second catenary cable string clamp in the first image, and the second catenary cable string clamp is a catenary cable string clamp connected to the lower end of the suspension string;
[0175] The third image is extracted from the first image according to the second pixel point.
[0176] In some possible implementations, in determining whether the suspension string is broken according to the target image, the processor 502 is specifically configured to perform the following operations:
[0177] Performing feature extraction on the third image to obtain a second feature map;
[0178] According to the second characteristic graph, a second probability of the suspension string breaking is obtained;
[0179] According to the second probability, it is determined whether the suspension string is broken.
[0180] In some possible implementations, the target image includes a second image and a third image, and the catenary wire clamp includes a first catenary wire clamp and a second catenary wire clamp. In obtaining the target image according to the first image, the processor 502 is specifically configured to perform the following operations:
[0181] Performing key point detection on the first image to obtain a first pixel point and a second pixel point, wherein the first pixel point is a pixel point corresponding to the first catenary cable string clamp in the first image, and the first catenary cable string clamp is a catenary cable string clamp connected to the upper end of the suspension string, and the second pixel point is a pixel point corresponding to the second catenary cable string clamp in the first image, and the second catenary cable string clamp is a catenary cable string clamp connected to the lower end of the suspension string;
[0182] According to the first pixel point, extracting the second image from the first image;
[0183] The third image is extracted from the first image according to the second pixel point.
[0184] In some possible implementations, in determining whether the suspension string is broken according to the target image, the processor 502 is specifically configured to perform the following operations:
[0185] Performing feature extraction on the second image to obtain a first feature map;
[0186] Obtaining a first probability of suspension string breakage according to the first characteristic graph;
[0187] Performing feature extraction on the third image to obtain a second feature map;
[0188] According to the second characteristic graph, a second probability of the suspension string breaking is obtained;
[0189] It is determined whether the suspension string is broken according to the first probability and the second probability.
[0190] In some possible implementations, in determining whether the suspension string is broken according to the first probability and the second probability, the processor 502 is specifically configured to perform the following operations:
[0191] determining a maximum probability of the first probability and the second probability;
[0192] According to the maximum probability, it is determined whether the suspension string is broken.
[0193] In some possible implementations, in terms of extracting the second image from the first image according to the first pixel point, the processor 502 is specifically configured to perform the following operations:
[0194] Starting from the first pixel point, a first number of pixel points are intercepted to the left, a second number of pixel points are intercepted to the right, and a third number of pixel points are intercepted downward to obtain the second image.
[0195] In some possible implementations, in terms of extracting the third image from the first image according to the second pixel point, the processor 502 is specifically configured to perform the following operations:
[0196] Starting from the first pixel point, a fourth number of pixel points are intercepted to the left, a fifth number of pixel points are intercepted to the right, and a sixth number of pixel points are intercepted downward to obtain the third image.
[0197] In some possible implementations, the first image is an original image or is obtained by performing target detection on the original image, wherein the original image is obtained by photographing a high-speed railway line, and when the first image is obtained by performing target detection on the original image, before acquiring the first image, the processor 502 is further configured to read a computer program in the memory 503 to perform the following operations:
[0198] Performing object detection on the original image to obtain at least one candidate frame;
[0199] Identify the object corresponding to each candidate frame to obtain the candidate frame where the hanging string is located;
[0200] The image in the candidate frame where the hanging string is located is intercepted to obtain the first image.
[0201] Specifically, the transceiver 501 may be Figure 4 The transceiver unit 401 of the image recognition device 400 of the embodiment described above, the processor 502 may be Figure 4 The processing unit 402 of the image recognition device 400 of the embodiment described.
[0202] An embodiment of the present application also provides a computer storage medium, wherein the computer-readable storage medium stores a computer program, and the computer program is executed by a processor to implement part or all of the steps of any image recognition method recorded in the above method embodiments.
[0203] An embodiment of the present application also provides a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to cause a computer to execute part or all of the steps of any image recognition method recorded in the above method embodiments.
[0204] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily required by the present application.
[0205] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.
[0206] In the several embodiments provided in the present application, it should be understood that the disclosed device can be implemented in other ways. For example, the device embodiments described above are only schematic, such as the division of the units, which is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the device or unit can be electrical or other forms.
[0207] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0208] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above integrated unit may be implemented in the form of hardware or in the form of a software program module.
[0209] If the integrated unit is implemented in the form of a software program module and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a memory, including a number of instructions to enable a computer device (which can be a personal computer, server or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned memory includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, disk or CD-ROM and other media that can store program codes.
[0210] A person skilled in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable memory, and the memory can include: a flash drive, a read-only memory (English: Read-Only Memory, abbreviated as: ROM), a random access memory (English: Random Access Memory, abbreviated as: RAM), a magnetic disk or an optical disk, etc.
[0211] The embodiments of the present application are introduced in detail above. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method and core idea of the present application. At the same time, for general technical personnel in this field, according to the idea of the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.
Claims
1. An image recognition method, characterized in that: include: Acquire a first image, wherein the first image includes the hanging string; According to the first image, a target image is obtained, wherein the target image includes a catenary cable sling clamp and a portion of the sling connected to the catenary cable sling clamp; including: The target image includes a second image, key point detection is performed on the first image to obtain a first pixel point, the first pixel point is a pixel point corresponding to a first catenary cable string clamp in the first image, and the first catenary cable string clamp is a catenary cable string clamp connected to the upper end of the suspension string; according to the first pixel point, the second image is intercepted from the first image; and / or, The target image includes a third image, and key point detection is performed on the first image to obtain a second pixel point, where the second pixel point is a pixel point corresponding to a second catenary cable string clamp in the first image, and the second catenary cable string clamp is a catenary cable string clamp connected to the lower end of the suspension string; and the third image is intercepted from the first image according to the second pixel point; It is determined whether the suspension string is broken according to the target image.
2. The method according to claim 1, characterized in that If the target image includes a second image, determining whether the suspension string is broken according to the target image includes: Performing feature extraction on the second image to obtain a first feature map; Obtaining a first probability of suspension string breakage according to the first characteristic graph; It is determined whether the suspension string is broken according to the first probability.
3. The method according to claim 1, characterized in that If the target image includes a third image, determining whether the suspension string is broken according to the target image includes: Performing feature extraction on the third image to obtain a second feature map; According to the second characteristic graph, a second probability of the suspension string breaking is obtained; According to the second probability, it is determined whether the suspension string is broken.
4. The method according to claim 1, characterized in that: If the target image includes the second image and the third image, determining whether the suspension string is broken according to the target image includes: Performing feature extraction on the second image to obtain a first feature map; Obtaining a first probability of suspension string breakage according to the first characteristic graph; Performing feature extraction on the third image to obtain a second feature map; According to the second characteristic graph, a second probability of the suspension string breaking is obtained; It is determined whether the suspension string is broken according to the first probability and the second probability.
5. The method according to claim 4, characterized in that The determining, according to the first probability and the second probability, whether the suspension string is broken comprises: determining a maximum probability of the first probability and the second probability; According to the maximum probability, it is determined whether the suspension string is broken.
6. The method according to claim 1, characterized in that The extracting the second image from the first image according to the first pixel point includes: Starting from the first pixel point, a first number of pixel points are intercepted to the left, a second number of pixel points are intercepted to the right, and a third number of pixel points are intercepted downward to obtain the second image.
7. The method according to claim 1, characterized in that The extracting the third image from the first image according to the second pixel point includes: Starting from the second pixel point, a fourth number of pixel points are intercepted to the left, a fifth number of pixel points are intercepted to the right, and a sixth number of pixel points are intercepted downward to obtain the third image.
8. The method according to any one of claims 1 to 7, characterized in that The first image is an original image or is obtained by performing target detection on the original image, wherein the original image is obtained by photographing a high-speed railway line, and in the case where the first image is obtained by performing target detection on the original image, before acquiring the first image, the method further includes: Performing object detection on the original image to obtain at least one candidate frame; Identify the object corresponding to each candidate frame to obtain the candidate frame where the hanging string is located; The image in the candidate frame where the hanging string is located is intercepted to obtain the first image.
9. An image recognition device, characterized in that: The device is used to implement the method according to any one of claims 1 to 8, and the device comprises: A transceiver unit, configured to acquire a first image, wherein the first image includes the hanging string; a processing unit, configured to obtain a target image according to the first image, wherein the target image includes a catenary cable string clamp and a portion of the string connected to the catenary cable string clamp; The processing unit is further used to determine whether the suspension string is broken according to the target image.
10. An electronic device, characterized in that: include: A processor, wherein the processor is connected to a memory, the memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory, so that the electronic device executes the method as described in any one of claims 1 to 8.
11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the computer program is executed by a processor to implement the method according to any one of claims 1 to 8.