Image detection method, device, electronic device and storage medium

Abstract

The embodiment of the present application provides an image detection method, device, electronic device and storage medium, wherein the image detection method comprises: obtaining an image to be detected of a high-speed railway contact network; performing pixel filling on the image to be detected based on a preset filling method to obtain at least one first mask image; and detecting at least one first mask image to determine whether there is a target component missing from the high-speed railway contact network. The embodiment of the present application is conducive to reducing the difficulty of detecting missing nuts, bolts and cotter pins of the contact network.

Description

Technical Field

[0001] The present application relates to the field of image detection technology, and in particular to an image detection method, device, electronic device and storage medium. Background Art

[0002] The contact network is an important part of the railway electrification project. Regular inspection of the contact network is a strong guarantee for the safety of passengers. In the industrial field, especially in the high-speed rail field, the contact network nut, bolt, and cotter pin missing detection technology has been widely used. The overall idea is to use the missing detection model to roughly locate the nuts, bolts, and cotter pins, and use the texture features of nuts, bolts, and cotter pins for missing detection. However, compared with the normal nut, bolt, and cotter pin data, the number of missing data of nuts, bolts, and cotter pins is very small, which makes the modeling of missing detection relatively difficult; on the other hand, the scenes where nuts, bolts, and cotter pins exist are different, and the features they present are also different, which increases the difficulty of feature extraction in missing detection. It can be seen that the current detection of missing nuts, bolts, and cotter pins in the contact network is relatively complicated and has the problem of high detection difficulty. Summary of the invention

[0003] The embodiments of the present application provide an image detection method, device, electronic device and storage medium, which are conducive to reducing the difficulty of detecting missing nuts, bolts and cotter pins in the contact network.

[0004] A first aspect of an embodiment of the present application provides an image detection method, the method comprising:

[0005] Obtain the image to be inspected of the high-speed railway contact network;

[0006] Filling pixels of the image to be detected based on a preset filling method to obtain at least one first mask image;

[0007] At least one first mask image is detected to determine whether there are any target components missing in the high-speed railway contact network.

[0008] It can be seen that the embodiment of the present application obtains the image to be detected of the high-speed rail contact network; fills the image to be detected with pixels based on a preset filling method to obtain at least one first mask image; and detects at least one first mask image to determine whether there is a target component missing in the high-speed rail contact network. In this way, by means of pixel filling, it is helpful to reduce the dependence on the texture features of target components such as nuts, bolts and cotter pins in the missing detection, so as to perform missing detection through context information, thereby helping to reduce the difficulty of missing detection of nuts, bolts and cotter pins in the contact network.

[0009] In conjunction with the first aspect, in a possible implementation, detecting at least one first mask image to determine whether a target component is missing in the high-speed railway contact network includes:

[0010] Performing target detection on at least one first mask image to obtain a reference prediction box corresponding to each first mask image in the at least one first mask image;

[0011] Fuse the reference prediction frame to obtain the first target prediction frame;

[0012] According to the first target prediction frame, determine whether there are any target components missing in the high-speed railway contact network.

[0013] In this example, since the first target prediction box is a prediction box obtained by fusing the reference prediction box, it means that the area framed by the first target prediction box is most likely to contain target components such as nuts, bolts, and cotter pins. Using this area for missing part detection is beneficial to improving the accuracy of missing part detection.

[0014] In combination with the first aspect, in a possible implementation manner, each first mask image includes a pixel-filled area obtained by performing pixel filling, performing target detection on at least one first mask image, and obtaining a reference prediction box corresponding to each first mask image in at least one first mask image, including:

[0015] Performing target detection on each first mask image to obtain a first prediction box corresponding to each first mask image;

[0016] The first prediction box in each first mask image that falls in the pixel filling area is determined as a reference prediction box.

[0017] In this example, the first prediction box that entirely falls within the pixel-filled area in each first mask image is used as a reference prediction box, which helps avoid using object texture information in non-pixel-filled areas in subsequent predictions, thereby reducing dependence on object texture information.

[0018] In combination with the first aspect, in a possible implementation, determining whether there is a target component missing in the high-speed railway contact network according to the first target prediction frame includes:

[0019] Determine the position information of the first target prediction box in the image to be detected;

[0020] Cut out the image of the area to be classified from the image to be detected according to the position information;

[0021] The image of the area to be classified is inspected to determine whether there are any target components missing in the high-speed rail contact network.

[0022] In this example, target detection is used to first obtain an initial detection result. For example, it is believed that the area framed by the first target prediction box should contain the target component. Then, the image of the area to be classified is cut out from the image to be detected according to the first target prediction box. The texture information in the image of the area to be classified is used to classify whether there is a target component in the image of the area to be classified, so as to further determine whether there is a target component missing in the high-speed rail contact network. This dual detection method can make the final output missing detection result more accurate.

[0023] In combination with the first aspect, in a possible implementation, obtaining an image to be detected of a high-speed railway contact network includes:

[0024] Performing target detection on the original image of the high-speed railway contact network to obtain a first target detection result;

[0025] If the first target detection result includes at least one second prediction box, determining a second target prediction box from the at least one second prediction box, and obtaining an image to be detected according to the second target prediction box and a preset grid;

[0026] If the first target detection result does not include the second prediction box, the image to be detected is obtained according to the preset rectangular box and grid.

[0027] In this example, the first target detection result refers to the detection result after target detection on the original image without subsequent processing (such as non-maximum suppression). The first target detection result includes at least one second prediction frame, indicating that target components such as nuts, bolts, and cotter pins are detected in the original image. The second prediction frame with the largest width or height is determined as the second target prediction frame, and the original image is adjusted according to the second target prediction frame and the grid to obtain the image to be detected; the first target detection result does not include the second prediction frame, indicating that target components such as nuts, bolts, and cotter pins are not detected in the original image. The original image is adjusted according to the preset rectangular frame and grid to obtain the image to be detected. In this way, regardless of whether target components such as nuts, bolts, and cotter pins are detected in the original image, the original image will be adjusted to ensure that the perception range of each layer of the neural network model for the image to be detected is as consistent as possible.

[0028] In combination with the first aspect, in a possible implementation, obtaining an image to be detected according to the second target prediction box and a preset grid includes:

[0029] Calculate a first scaling factor according to the size of the grid and the width or height of the second target prediction box;

[0030] The original image is scaled according to the first scaling factor to obtain an image to be detected.

[0031] In this example, when target components such as nuts, bolts, and cotter pins are detected in the original image, a first scaling factor is obtained based on the size of the grid and the width or height of the second target prediction box, and the original image is scaled according to the first scaling factor, which helps to ensure that the perception range of each layer of the neural network model for the image to be detected remains consistent.

[0032] In combination with the first aspect, in a possible implementation, obtaining an image to be detected according to a preset rectangular frame and a grid includes:

[0033] Calculate a second scaling factor according to the size of the grid and the width or height of the rectangular frame;

[0034] The original image is scaled according to the second scaling factor to obtain an image to be detected.

[0035] In this example, when target components such as nuts, bolts, and cotter pins are not detected in the original image, a second scaling factor is obtained based on the size of the grid and the width or height of the rectangular frame, and the original image is scaled according to the second scaling factor, which helps ensure that the perception range of each layer of the neural network model for the image to be detected remains consistent.

[0036] In combination with the first aspect, in a possible implementation, performing target detection on each first mask image to obtain a first prediction box corresponding to each first mask image includes:

[0037] Perform at least one convolution, gated convolution, and weight coefficient multiplication operation, and use the resulting fused feature map as the target feature map;

[0038] Based on the target feature map, each first mask image is classified and predicted to obtain a first prediction box;

[0039] Among them, at least one convolution, gated convolution and weight coefficient multiplication operation includes:

[0040] Convolve each first mask map or feature map obtained in the last operation to obtain a first feature map;

[0041] Perform gated convolution on each first mask map or feature map obtained by the last operation to obtain a second feature map, the second feature map has the same size as the first feature map, and the value of each position in the second feature map is used to represent the weight coefficient of the corresponding position in the first feature map;

[0042] Multiply the value of each position in the first feature map by the corresponding weight coefficient in the second feature map to obtain a fused feature map.

[0043] In this example, a two-stage neural network model is used to perform target detection on each first mask image, and gated convolution is added to the detection in the first stage. Based on the weight coefficients extracted by the gated convolution, the neural network model can learn a dynamic feature selection mechanism for each channel and each spatial position of the first mask image, that is, it pays more attention to the areas in the first mask image that are not pixel-filled. These areas usually contain key contextual information. That is, the target feature map obtained in the first stage is a feature map with more contextual information, so that the target feature map with more contextual information can be used for classification prediction in the second stage, thereby reducing the dependence on object texture features.

[0044] In combination with the first aspect, in a possible implementation manner, pixel filling is performed on the image to be detected based on a preset filling method to obtain at least one first mask image, including:

[0045] Constructing at least one matrix according to a preset filling method, wherein the size of each matrix in the at least one matrix is ​​the same as the size of the image to be detected, and each matrix includes an element whose value is 0;

[0046] Each matrix is ​​dot-multiplied with the pixels of the image to be detected to fill the pixels corresponding to the elements with a value of 0 in the image to be detected with black pixels, thereby obtaining at least one first mask image, wherein the pixel filling area is the area filled with black pixels in each first mask image.

[0047] In this example, a matrix is ​​constructed based on a preset filling method, and the matrix is ​​dot-multiplied with the image to be detected to fill pixels in the image to be detected, thereby ensuring that target parts such as nuts, bolts, and cotter pins in the image to be detected can be covered by black pixels to remove the texture features of the object, thereby facilitating classification prediction based on context information in subsequent target detection.

[0048] In combination with the first aspect, in a possible implementation manner, target detection on at least one first mask image is performed by a neural network model, and the neural network model is trained by the following steps:

[0049] Acquire a sample image and annotation data of a target component in the sample image;

[0050] Inputting the sample image into a neural network for target detection to obtain at least one third prediction frame, and determining a third target prediction frame from the at least one third prediction frame;

[0051] Scaling the sample image according to the third target prediction box and the grid to obtain a scaled sample image;

[0052] Perform pixel filling on the scaled sample image to obtain at least one second mask image;

[0053] The neural network is trained using at least one second mask image, the labeled data, and the loss function to obtain a neural network model.

[0054] In this example, for sample images of target parts such as nuts, bolts, and cotter pins, the above target parts are covered by pixel filling to construct a batch of training data with missing target parts, namely the second mask map, thereby solving the problem of less image data with missing nuts, bolts, and cotter pins in the high-speed rail contact network. In addition, using the second mask map with covered target parts to train the neural network is conducive to making the neural network model pay more attention to the contextual information in the image, thereby reducing the dependence on the texture features of target parts such as nuts, bolts, and cotter pins.

[0055] A second aspect of an embodiment of the present application provides an image detection device, the device comprising an acquisition unit and a processing unit;

[0056] An acquisition unit, used for acquiring an image to be detected of the high-speed railway contact network;

[0057] A processing unit, configured to perform pixel filling on the image to be detected based on a preset filling method to obtain at least one first mask image;

[0058] The processing unit is also used to detect the at least one first mask image to determine whether there are any target components missing in the high-speed railway contact network.

[0059] A third aspect of an embodiment of the present application provides an electronic device, comprising a processor, an input device, an output device and a memory, wherein the processor, the input device, the output device and the memory are interconnected, wherein the memory is used to store a computer program, the computer program comprises program instructions, and the processor is configured to call the program instructions to execute some or all of the steps described in the first aspect of the embodiment of the present application.

[0060] The fourth aspect of the embodiments of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program for electronic data exchange, wherein the computer program enables a computer to execute part or all of the steps described in the first aspect of the embodiments of the present application.

[0061] A fifth aspect of the embodiments of the present application provides a computer program product, wherein the computer program product includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to cause a computer to execute some or all of the steps described in the first aspect of the embodiments of the present application. The computer program product may be a software installation package.

[0062] These and other aspects of the present application will become more clearly understood in the description of the following embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0064] Figure 1 A schematic diagram of an application environment provided for an embodiment of the present application;

[0065] Figure 2 A schematic diagram of a flow chart of an image detection method provided in an embodiment of the present application;

[0066] Figure 3 A schematic diagram of a pixel filling method and a corresponding first mask image provided in an embodiment of the present application;

[0067] Figure 4 A schematic diagram of a pixel filling method and a corresponding matrix provided in an embodiment of the present application;

[0068] Figure 5 A schematic diagram for explaining a principle provided by an embodiment of the present application;

[0069] Figure 6 A schematic diagram of feature extraction provided in an embodiment of the present application;

[0070] Figure 7 A schematic diagram of a flow chart of another image detection method provided in an embodiment of the present application;

[0071] Figure 8 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application;

[0072] Fig. 9 A schematic diagram of the structure of an image detection device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0073] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0074] The terms "first", "second", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally includes steps or units that are not listed, or optionally includes other steps or units inherent to these processes, methods, products or devices.

[0075] Reference to "embodiments" in this application means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described in this application may be combined with other embodiments.

[0076] In order to better understand the image detection method provided in the embodiment of the present application, the application environment of the image detection method is briefly introduced below.

[0077] See also Figure 1 , Figure 1 A schematic diagram of an application environment provided in an embodiment of the present application, such as Figure 1 As shown, it includes a terminal device 110, an electronic device 120, a high-speed rail contact network 130 and an inspection vehicle 140, wherein the terminal device 110 and the electronic device 120 are connected through a network. The terminal device 110 is used to provide an original image obtained by shooting the high-speed rail contact network. For example, the terminal device 110 can be an image acquisition device on a pillar or an inspection vehicle 140. The electronic device 120 is used to obtain an image to be detected based on the original image and a preset grid, and then use a preset N filling methods to fill pixels of the image to be detected to cover the target components such as nuts, bolts, and cotter pins that may exist in the image to be detected, to obtain N mask images, and then perform two-stage target detection on the N mask images to predict whether there are target components such as nuts, bolts, and cotter pins in the pixel filling area using the context information of the pixel-filled area in the mask image. If there are, the electronic device 120 cuts out a regional image from the image to be detected based on the position information of the predicted frame of the pixel filling area in the image to be detected, and finally classifies and predicts the regional image to determine whether there is a target component missing in the high-speed rail contact network. Wherein, N is an integer greater than or equal to 1.

[0078] Compared with the existing solutions, the embodiments of the present application cover the texture features of target components such as nuts, bolts and cotter pins by means of pixel filling, and mainly use context information for prediction, which is beneficial to reduce the dependence on the texture features of target components such as nuts, bolts and cotter pins in missing detection, thereby helping to reduce the difficulty of missing detection of contact network nuts, bolts and cotter pins (for example, when the target component is completely missing, the texture features of the target component do not exist on the image, and it is more difficult to use texture features for accurate prediction).

[0079] It should be understood that in some embodiments, the terminal device 110 may also be a computer, a smart phone or other device. For example, the terminal device 110 may obtain the original image of the high-speed rail contact network from the database and provide the original image to the electronic device 120. The electronic device 120 may be an independent physical server, a server cluster or a distributed system, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, and big data and artificial intelligence platforms, or may be a device that can operate independently, such as an embedded device.

[0080] See also Figure 2 , Figure 2 The following is a flow chart of an image detection method provided in an embodiment of the present application. The image detection method is applied to electronic devices, such as Figure 2 As shown, the method includes:

[0081] 201. Obtain the image to be inspected of the high-speed railway contact network.

[0082] In a specific embodiment of the present application, the image to be detected can be obtained by scaling the original image of the high-speed rail contact network. The original image of the high-speed rail contact network is sent from a terminal device to an electronic device. The terminal device can be an image acquisition device on a patrol vehicle. For example, in a transmission line inspection, the image acquisition device can send the captured original image to the electronic device; the terminal device can also be an image acquisition device installed at a fixed position in the transmission line, such as a camera on a pillar. By setting a shooting time interval, the camera can send the captured original image to the electronic device at regular intervals; the terminal device can also be a user device. For example, in algorithm development or model training, the user usually needs to send the original image set of the high-speed rail contact network to the electronic device through the user device. It should be understood that the present application does not limit the method for obtaining the original image of the high-speed rail contact network.

[0083] Exemplarily, obtaining an image to be detected of a high-speed railway contact network includes:

[0084] A1: Perform target detection on the original image to obtain the first target detection result;

[0085] A2: If the first target detection result includes at least one second prediction box, determine a second target prediction box from the at least one second prediction box, and obtain an image to be detected according to the second target prediction box and the grid;

[0086] A3: If the first target detection result does not include the second prediction box, the image to be detected is obtained according to the preset rectangular box and grid.

[0087] In a specific embodiment of the present application, the first target detection result refers to the detection result after target detection is performed on the original image without subsequent processing (such as non-maximum suppression). It should be understood that if there are target components such as nuts, bolts, and cotter pins in the original image, the first target detection result will include at least one candidate box, that is, the second prediction box. If there are no target components such as nuts, bolts, and cotter pins in the original image, the first target detection result may not include the second prediction box, that is, no candidate box is generated.

[0088] Exemplarily, the present application may use a trained neural network model for target detection, such as the FasterR-CNN framework. Since the receptive field (receptive range) of the backbone network of the neural network model is fixed, it is necessary to ensure that the receptive fields of each layer of the neural network for the input image remain as consistent as possible. In other words, the number of grids seen is as consistent as possible, which requires scaling and adjusting the original image of the high-speed rail contact network.

[0089] In the case where the first target detection result includes at least one second prediction box, the width and height of each second prediction box in the at least one second prediction box are obtained, and the second prediction box with the largest width or the largest height is determined as the second target prediction box, and then the image to be detected is obtained based on the second target prediction box and the grid. It should be noted that the maximum width means that the width value is greater than all other width values ​​and height values, and the maximum height means that the height value is greater than all other width values ​​and height values.

[0090] Exemplarily, obtaining the image to be detected according to the second target prediction box and the grid includes:

[0091] B1: Calculate the first scaling factor according to the size of the grid and the width or height of the second target prediction box;

[0092] B2: Scale the original image according to the first scaling factor to obtain the image to be detected.

[0093] Among them, assuming that the size of the grid is 2*2, the maximum value of the width or height of the second target prediction box is reduced to half of the grid size. For example, the maximum value of the width or height of the second target prediction box is 1.5, and the first scaling factor 1 / 1.5 is calculated using the size of the grid and the maximum value 1.5. The original image is scaled according to the first scaling factor to obtain the image to be detected.

[0094] In this implementation, when target components such as nuts, bolts, and cotter pins are detected in the original image, a first scaling factor is obtained based on the size of the grid and the width or height of the second target prediction box, and the original image is scaled according to the first scaling factor, which helps ensure that the perception range of each layer of the neural network model for the image to be detected remains consistent.

[0095] In the case where the first target detection result does not include the second prediction box, the image to be detected is obtained according to the preset rectangular box and the grid. The size of the rectangular box can be set based on empirical values, for example, in general, the circumscribed rectangular box of the largest target component in the collected images is set as the rectangular box.

[0096] Exemplarily, obtaining an image to be detected according to a preset rectangular frame and a grid includes:

[0097] C1: Calculate the second scaling factor based on the size of the grid and the width or height of the rectangular box;

[0098] C2: Scale the original image according to the second scaling factor to obtain the image to be detected.

[0099] Among them, assuming that the size of the grid is 2*2, the maximum value of the preset rectangular frame width or height is reduced to half of the grid size. For example, if the maximum value of the rectangular frame width or height is 2, the second scaling factor 1 / 2 is calculated using the grid size and the maximum value 2, and the original image is scaled according to the second scaling factor to obtain the image to be detected.

[0100] In this embodiment, when target components such as nuts, bolts, and cotter pins are not detected in the original image, a second scaling factor is obtained based on the size of the grid and the width or height of the rectangular frame, and the original image is scaled according to the second scaling factor, which helps to ensure that the perception range of each layer of the neural network model for the image to be detected remains consistent.

[0101] 202. Fill pixels of the image to be detected based on a preset filling method to obtain at least one first mask image.

[0102] Exemplarily, pixel filling is performed on the image to be detected based on a preset filling method to obtain at least one first mask image, including:

[0103] Constructing at least one matrix according to a preset filling method, wherein the size of each matrix in the at least one matrix is ​​the same as the size of the image to be detected, and each matrix includes an element whose value is 0;

[0104] Each matrix is ​​dot-multiplied with the pixels of the image to be detected to fill the pixels corresponding to the elements with a value of 0 in the image to be detected with black pixels, thereby obtaining at least one first mask image, wherein the pixel filling area is the area filled with black pixels in each of the first mask images.

[0105] In the specific embodiment of the present application, 8 pixel filling modes are preset, such as Figure 3 As shown in the grid on the left, the pixels of the image to be detected are filled according to these 8 filling methods to obtain the corresponding 8 first mask images, as shown in Figure 3 Specifically, for each filling method, a matrix is ​​constructed according to the filling method and the size of the image to be detected. Figure 3 Taking the first filling method in the upper left corner as an example, assuming that the size of the image to be detected is 10*10 (pixels), then according to the first filling method ( Figure 4 The distribution of black pixels and white pixels in the image can be constructed as follows: Figure 4 The matrix shown on the right side of the figure, where the elements with a value of 0 in the matrix represent black pixels, and the elements with a value of 1 represent white pixels. Since the size of the matrix is ​​the same as the size of the image to be detected, the constructed matrix is ​​multiplied by the image to be detected to obtain Figure 3 The first mask images shown on the right side of the figure, each of which includes a pixel filling area covered by black pixels.

[0106] It should be understood that by filling pixels in the image to be detected according to the above 8 filling methods, it can ensure that the target component in the image to be detected is covered by black pixels in at least one of the first mask images, thereby achieving the purpose of removing the texture features of the target component in the image to be detected. The principle can be found in Figure 5 , assuming Figure 5 The middle line segment AB=BC=CD, its length is s. If there is a line segment with the same length s, and its starting point is within the interval of line segment AB, then the line segment can be Figure 5 The curly brackets on the upper middle side cover it. If its starting point is within the line segment BC interval, then the line segment can definitely be Figure 5 The curly brackets on the lower middle side cover. This is the proof of one-dimensional space. By analogy to two-dimensional space, according to the above 8 filling methods, it can be guaranteed that the target part in the image to be detected is completely covered by one of the filling methods.

[0107] In this embodiment, a matrix is ​​constructed based on a preset filling method, and the matrix is ​​dot-multiplied with the image to be detected to fill pixels in the image to be detected, thereby ensuring that target components such as nuts, bolts, and cotter pins in the image to be detected can be covered by black pixels to remove the texture features of the object, thereby facilitating classification prediction based on context information during subsequent target detection.

[0108] 203. Detect at least one first mask image to determine whether there are any target components missing in the high-speed railway contact network.

[0109] Exemplarily, detecting at least one first mask image to determine whether there is a target component missing in the high-speed railway contact network includes:

[0110] D1: performing target detection on at least one first mask image to obtain a reference prediction box corresponding to each first mask image in at least one first mask image;

[0111] D2: Fuse the reference prediction frame to obtain the first target prediction frame;

[0112] D3: According to the first target prediction frame, determine whether there are any target components missing in the high-speed rail contact network.

[0113] In a specific embodiment of the present application, each first mask image is input into a neural network model for target detection, and the candidate box generated in each first mask image is determined as the first prediction box corresponding to each first mask image, and then the first prediction box is screened, and the first prediction box that completely falls in the pixel-filled area is retained and determined as the reference prediction box. In this embodiment, when the first prediction box does not completely fall in the pixel-filled area, the texture information of the object corresponding to the first prediction box cannot be effectively blocked by the pixel-filled area, and the texture information of the target object may be used in subsequent predictions. The first prediction box that completely falls in the pixel-filled area in each first mask image is used as a reference prediction box, which is beneficial to avoid using the texture information of the object in the non-pixel-filled area in the subsequent prediction, thereby reducing the dependence on the texture information of the object.

[0114] After the reference prediction frame is determined, the reference prediction frame is fused. Exemplarily, the reference prediction frame can be subjected to non-maximum suppression, and the reference prediction frame with a score greater than a threshold is used as the first target prediction frame. Since the first target prediction frame is a prediction frame obtained by performing non-maximum suppression on the reference prediction frame, it means that the area defined by the first target prediction frame should contain target components such as nuts, bolts, and cotter pins. Using this area for missing detection is beneficial to improving the accuracy of missing detection.

[0115] For the determined first target prediction frame, since the first mask image has the same size as the image to be detected, the position information of the first target prediction frame in the first mask image can be used as the position information of the first target prediction frame in the image to be detected. According to the position information, the image of the area to be classified can be cut out from the image to be detected, and the image of the area to be classified can be input into the classification model for detection, so as to use the texture information in the image of the area to be classified to determine whether there are target components in the area framed by the first target prediction frame, so as to determine whether there are target components missing in the high-speed rail contact network. For example, if the target components such as nuts, bolts, and cotter pins are not predicted in the image of the area to be classified, it means that there are target components missing in the high-speed rail contact network.

[0116] In this implementation, target detection is used to first obtain an initial detection result. For example, it is believed that the area framed by the first target prediction frame should have a target component. Then, the image of the area to be classified is cut out from the image to be detected based on the first target prediction frame. The texture information in the image of the area to be classified is used to classify whether there is a target component in the image of the area to be classified, so as to further determine whether there is a target component missing in the high-speed rail contact network. This dual detection method can make the final output missing detection result more accurate.

[0117] Exemplarily, performing target detection on each first mask image to obtain a first prediction box corresponding to each first mask image includes:

[0118] Perform at least one convolution, gated convolution, and weight coefficient multiplication operation, and use the resulting fused feature map as the target feature map;

[0119] Based on the target feature map, each first mask image is classified and predicted to obtain a first prediction box;

[0120] Among them, at least one convolution, gated convolution and weight coefficient multiplication operation includes:

[0121] E1: Convolve each first mask map or feature map obtained in the last operation to obtain the first feature map;

[0122] E2: Perform gated convolution on each first mask map or feature map obtained in the last operation to obtain a second feature map. The second feature map has the same size as the first feature map, and the value of each position in the second feature map is used to represent the weight coefficient of the corresponding position in the first feature map.

[0123] E3: Multiply the value of each position in the first feature map by the corresponding weight coefficient in the second feature map to obtain a fused feature map.

[0124] In the specific embodiment of the present application, a two-stage neural network model is used to perform target detection on each first mask image. In the first stage of detection, in addition to the ordinary convolution operation, a gated convolution is also added. Figure 6 , Figure 6 A schematic diagram of feature extraction provided in an embodiment of the present application, such as Figure 6 As shown, for the first input mask map, ordinary convolution is performed on it to obtain the first feature map, and gated convolution is performed on it to obtain the second feature map. Since the value of each position in the second feature map is used to represent the weight coefficient of the corresponding position in the first feature map, the two can be multiplied to obtain the third feature map (i.e., the feature map obtained by the last operation) to focus on the unfilled area with context information. The third feature map is then subjected to ordinary convolution to obtain the fourth feature map, and the third feature map is then subjected to gated convolution to obtain the fifth feature map. Since the value of each position in the fifth feature map is used to represent the weight coefficient of the corresponding position in the fourth feature map, the two can be multiplied to obtain the sixth feature map. It should be understood that Figure 6 The operations of performing two convolutions, gated convolutions and multiplication of weight coefficients are given in the figure. The sixth feature map is the fused feature map. The sixth feature map can be used as the target feature map. The embodiment of the present application can obtain the fused feature map through at least one convolution, gated convolution and multiplication of weight coefficients. For example, when it is performed once, the third feature map is the fused feature map.

[0125] The target feature map can be used as the output of the first stage detection, and the neural network model can perform classification prediction based on the target feature map in the second stage, and the generated candidate box is the first prediction box. Exemplarily, the target feature map can also be subjected to ordinary convolution to obtain the output seventh feature map of the first stage detection, and the neural network model can also perform classification prediction based on the seventh feature map in the second stage, and the generated candidate box is the first prediction box.

[0126] In this implementation, a two-stage neural network model is used to perform target detection on each first mask image, and gated convolution is added to the detection in the first stage. Based on the weight coefficients extracted by the gated convolution, the neural network model can learn a dynamic feature selection mechanism for each channel and each spatial position of the first mask image, that is, paying more attention to the areas in the first mask image that are not pixel-filled, which usually contain key contextual information, that is, the target feature map obtained in the first stage is a feature map with more contextual information, so that the target feature map with more contextual information can be used for classification prediction in the second stage, thereby reducing the dependence on object texture features.

[0127] It can be seen that the embodiment of the present application obtains the image to be detected of the high-speed rail contact network; fills the image to be detected with pixels based on a preset filling method to obtain at least one first mask image; and detects at least one first mask image to determine whether there is a target component missing in the high-speed rail contact network corresponding to the image to be detected. In this way, by means of pixel filling, it is helpful to reduce the dependence on the texture features of target components such as nuts, bolts and cotter pins in the missing detection, so as to perform missing detection through context information, thereby helping to reduce the difficulty of missing detection of nuts, bolts and cotter pins in the contact network.

[0128] See also Figure 7 , Figure 7 A schematic diagram of another image detection method provided in an embodiment of the present application. Figure 7 As shown, the method includes:

[0129] 701. Obtain an image to be detected of a high-speed railway contact network;

[0130] 702. Fill pixels of the image to be detected based on a preset filling method to obtain at least one first mask image;

[0131] 703. Perform object detection on at least one first mask image to obtain a reference prediction box corresponding to each first mask image in the at least one first mask image;

[0132] 704: Fusing the reference prediction frame to obtain a first target prediction frame;

[0133] 705: Determine whether there are any target components missing in the high-speed rail contact network according to the first target prediction frame.

[0134] The specific implementation of the above steps 701-705 is as follows: Figure 2 The embodiments shown have been described in detail and can achieve the same or similar beneficial effects, so they will not be described again here.

[0135] In step 703, target detection on at least one first mask image is performed by a neural network model, and the neural network model is trained by the following steps:

[0136] F1: Obtain sample images and annotation data of target parts in the sample images;

[0137] F2: Input the sample image into the neural network for target detection, obtain at least one third prediction frame, and determine a third target prediction frame from the at least one third prediction frame;

[0138] F3: Scale the sample image according to the third target prediction box and the grid to obtain a scaled sample image;

[0139] F4: Filling pixels of the scaled sample image to obtain at least one second mask image;

[0140] F5: Use at least one second mask image, labeled data and loss function to train the neural network to obtain a neural network model.

[0141] In the specific embodiment of the present application, since the image data of missing nuts, bolts and cotter pins in the high-speed rail contact network are relatively scarce, and a large amount of training data is required to train the neural network model in deep learning, sample images of target components such as nuts, bolts and cotter pins are used to construct a batch of training data of missing target components.

[0142] Specifically, the target components in the sample image can be marked with a box to obtain the annotation data, and then the sample image can be input into the neural network for target detection. The generated candidate box is the third prediction box. The width and height of each third prediction box in at least one third prediction box are obtained, and the third prediction box with the largest width or the largest height is determined as the third target prediction box. The scaling factor is calculated according to the preset grid size and the width and height of the third target prediction box, and the sample image is scaled according to the scaling factor to obtain the scaled sample image. Figure 3 The scaled sample image is pixel-filled using the filling method shown in the figure to obtain at least one second mask image, the second mask image is input into the neural network for reasoning, and the parameters of the neural network are updated using the labeled data, the loss function and the reasoning results. After multiple iterations, a trained neural network model is obtained.

[0143] In this embodiment, for sample images of target parts such as nuts, bolts, and cotter pins, the above target parts are covered by pixel filling to construct a batch of training data with missing target parts, namely, the second mask image, thereby solving the problem of less image data with missing nuts, bolts, and cotter pins in the high-speed rail contact network. In addition, using the second mask image with the target parts covered to train the neural network is conducive to making the neural network model pay more attention to the contextual information in the image, thereby reducing the dependence on the texture features of the target parts such as nuts, bolts, and cotter pins.

[0144] For the above embodiments, please refer to Figure 8 , Figure 8 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application is shown in FIG. Figure 8 As shown, it includes a processor, an input device, an output device and a memory, which are connected to each other, wherein the memory is used to store a computer program, the computer program includes program instructions, the processor is configured to call the program instructions, and the program includes instructions for executing the following steps;

[0145] Obtain the image to be inspected of the high-speed railway contact network;

[0146] Filling pixels of the image to be detected based on a preset filling method to obtain at least one first mask image;

[0147] At least one first mask image is detected to determine whether there are any target components missing in the high-speed railway contact network.

[0148] It can be seen that in Figure 8 In the electronic device shown, an image to be detected of a high-speed railway contact network is obtained; pixels of the image to be detected are filled based on a preset filling method to obtain at least one first mask image; at least one first mask image is detected to determine whether there is a target component missing in the high-speed railway contact network corresponding to the image to be detected. Thus, by means of pixel filling, the reliance on texture features of target components such as nuts, bolts and cotter pins in missing detection is reduced, so that missing detection is performed through context information, which is conducive to reducing the difficulty of missing detection of nuts, bolts and cotter pins in the contact network.

[0149] In a possible implementation, the processor performs detection on at least one first mask image to determine whether a target component is missing in the high-speed railway contact network, including:

[0150] Performing target detection on at least one first mask image to obtain a reference prediction box corresponding to each first mask image in the at least one first mask image;

[0151] Fuse the reference prediction frame to obtain the first target prediction frame;

[0152] According to the first target prediction frame, determine whether there are any target components missing in the high-speed railway contact network.

[0153] In a possible implementation, each first mask image includes a pixel-filled area obtained by performing pixel filling, and the processor performs target detection on at least one first mask image to obtain a reference prediction box corresponding to each first mask image in at least one first mask image, including:

[0154] Performing target detection on each first mask image to obtain a first prediction box corresponding to each first mask image;

[0155] The first prediction box in each first mask image that falls in the pixel filling area is determined as a reference prediction box.

[0156] In a possible implementation, the processor determines whether there is a target component missing in the high-speed railway contact network according to the first target prediction frame, including:

[0157] Determine the position information of the first target prediction box in the image to be detected;

[0158] Cut out the image of the area to be classified from the image to be detected according to the position information;

[0159] The image of the area to be classified is inspected to determine whether there are any target components missing in the high-speed rail contact network.

[0160] In a possible implementation, the processor executes the step of acquiring the image to be detected of the high-speed railway contact network, including:

[0161] Performing target detection on the original image of the high-speed railway contact network to obtain a first target detection result;

[0162] If the first target detection result includes at least one second prediction box, determining a second target prediction box from the at least one second prediction box, and obtaining an image to be detected according to the second target prediction box and a preset grid;

[0163] If the first target detection result does not include the second prediction box, the image to be detected is obtained according to the preset rectangular box and grid.

[0164] In a possible implementation, the processor executes, according to the second target prediction box and the preset grid, obtaining the image to be detected, including:

[0165] Calculate a first scaling factor according to the size of the grid and the width or height of the second target prediction box;

[0166] The original image is scaled according to the first scaling factor to obtain an image to be detected.

[0167] In a possible implementation, the processor executes the steps of obtaining the image to be detected according to the preset rectangular frame and grid, including:

[0168] Calculate a second scaling factor according to the size of the grid and the width or height of the rectangular frame;

[0169] The original image is scaled according to the second scaling factor to obtain an image to be detected.

[0170] In a possible implementation, the processor performs target detection on each first mask image to obtain a first prediction box corresponding to each first mask image, including:

[0171] Perform at least one convolution, gated convolution, and weight coefficient multiplication operation, and use the resulting fused feature map as the target feature map;

[0172] Based on the target feature map, each first mask image is classified and predicted to obtain a first prediction box;

[0173] Among them, at least one convolution, gated convolution and weight coefficient multiplication operation includes:

[0174] Convolve each first mask map or feature map obtained in the last operation to obtain a first feature map;

[0175] Perform gated convolution on each first mask map or feature map obtained by the last operation to obtain a second feature map, the second feature map has the same size as the first feature map, and the value of each position in the second feature map is used to represent the weight coefficient of the corresponding position in the first feature map;

[0176] Multiply the value of each position in the first feature map by the corresponding weight coefficient in the second feature map to obtain a fused feature map.

[0177] In a possible implementation, the processor performs pixel filling on the image to be detected based on a preset filling method to obtain at least one first mask image, including:

[0178] Constructing at least one matrix according to a preset filling method, wherein the size of each matrix in the at least one matrix is ​​the same as the size of the image to be detected, and each matrix includes an element whose value is 0;

[0179] Each matrix is ​​dot-multiplied with the pixels of the image to be detected to fill the pixels corresponding to the elements with a value of 0 in the image to be detected with black pixels, thereby obtaining at least one first mask image, wherein the pixel filling area is the area filled with black pixels in each first mask image.

[0180] In one possible implementation, the object detection on the at least one first mask image is performed by a neural network model, and the processor performs a training step of the neural network model, including:

[0181] Acquire a sample image and annotation data of a target component in the sample image;

[0182] Inputting the sample image into a neural network for target detection to obtain at least one third prediction frame, and determining a third target prediction frame from the at least one third prediction frame;

[0183] Scaling the sample image according to the third target prediction box and the grid to obtain a scaled sample image;

[0184] Perform pixel filling on the scaled sample image to obtain at least one second mask image;

[0185] The neural network is trained using at least one second mask image, the labeled data, and the loss function to obtain a neural network model.

[0186] The above mainly introduces the scheme of the embodiment of the present application from the perspective of the execution process on the method side. It is understandable that, in order to realize the above functions, the electronic device includes a hardware structure and / or software module corresponding to the execution of each function. It should be easily appreciated by those skilled in the art that, in combination with the units and algorithm steps of each example described in the embodiments provided herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present application.

[0187] The embodiment of the present application can divide the electronic device into functional units according to the above method example. For example, each functional unit can be divided according to each function, or two or more functions can be integrated into one processing unit. The above integrated unit can be implemented in the form of hardware or in the form of software functional units. It should be noted that the division of units in the embodiment of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation.

[0188] In line with the above, see Fig. 9 , Fig. 9 This is a schematic diagram of the structure of an image detection device provided in an embodiment of the present application. Fig. 9 As shown, the device includes an acquisition unit 901 and a processing unit 902;

[0189] An acquisition unit 901 is used to acquire an image to be detected of the high-speed railway contact network;

[0190] The processing unit 902 is used to fill pixels of the image to be detected based on a preset filling method to obtain at least one first mask image;

[0191] The processing unit 902 is further used to detect the at least one first mask image to determine whether there is any missing target component in the high-speed railway contact network.

[0192] It can be seen that in Fig. 9 In the image detection device shown, the image to be detected of the high-speed railway contact network is obtained; the image to be detected is pixel-filled based on a preset filling method to obtain at least one first mask image; at least one first mask image is detected to determine whether there is a target component missing in the high-speed railway contact network corresponding to the image to be detected. Thus, by pixel filling, the reliance on the texture features of target components such as nuts, bolts and cotter pins in the missing detection is reduced, so that missing detection is performed through context information, which is conducive to reducing the difficulty of missing detection of nuts, bolts and cotter pins in the contact network.

[0193] In a possible implementation, in terms of detecting at least one first mask image to determine whether there is a target component missing in the high-speed railway contact network, the processing unit 902 is specifically configured to:

[0194] Performing target detection on at least one first mask image to obtain a reference prediction box corresponding to each first mask image in the at least one first mask image;

[0195] Fuse the reference prediction frame to obtain the first target prediction frame;

[0196] According to the first target prediction frame, determine whether there are any target components missing in the high-speed railway contact network.

[0197] In a possible implementation, each first mask image includes a pixel-filled area obtained by performing pixel filling, and in performing target detection on at least one first mask image to obtain a reference prediction box corresponding to each first mask image in at least one first mask image, the processing unit 902 is specifically configured to:

[0198] Performing target detection on each first mask image to obtain a first prediction box corresponding to each first mask image;

[0199] The first prediction box in each first mask image that falls in the pixel filling area is determined as a reference prediction box.

[0200] In a possible implementation, in determining whether there is a target component missing in the high-speed railway contact network according to the first target prediction frame, the processing unit 902 is specifically configured to:

[0201] Determine the position information of the first target prediction box in the image to be detected;

[0202] Cut out the image of the area to be classified from the image to be detected according to the position information;

[0203] The image of the area to be classified is inspected to determine whether there are any target components missing in the high-speed rail contact network.

[0204] In a possible implementation, in acquiring the image to be detected of the high-speed railway contact network, the processing unit 902 is specifically used to:

[0205] Performing target detection on the original image of the high-speed railway contact network to obtain a first target detection result;

[0206] If the first target detection result includes at least one second prediction box, determining a second target prediction box from the at least one second prediction box, and obtaining an image to be detected according to the second target prediction box and a preset grid;

[0207] If the first target detection result does not include the second prediction box, the image to be detected is obtained according to the preset rectangular box and grid.

[0208] In a possible implementation, in terms of obtaining the image to be detected according to the second target prediction box and the preset grid, the processing unit 902 is specifically configured to:

[0209] Calculate a first scaling factor according to the size of the grid and the width or height of the second target prediction box;

[0210] The original image is scaled according to the first scaling factor to obtain an image to be detected.

[0211] In a possible implementation, in terms of obtaining the image to be detected according to the preset rectangular frame and grid, the processing unit 902 is specifically configured to:

[0212] Calculate a second scaling factor according to the size of the grid and the width or height of the rectangular frame;

[0213] The original image is scaled according to the second scaling factor to obtain an image to be detected.

[0214] In a possible implementation, in performing target detection on each first mask image to obtain a first prediction box corresponding to each first mask image, the processing unit 902 is specifically configured to:

[0215] Perform at least one convolution, gated convolution, and weight coefficient multiplication operation, and use the resulting fused feature map as the target feature map;

[0216] Based on the target feature map, each first mask image is classified and predicted to obtain a first prediction box;

[0217] Among them, at least one convolution, gated convolution and weight coefficient multiplication operation includes:

[0218] Convolve each first mask map or feature map obtained in the last operation to obtain a first feature map;

[0219] Perform gated convolution on each first mask map or feature map obtained by the last operation to obtain a second feature map, the second feature map has the same size as the first feature map, and the value of each position in the second feature map is used to represent the weight coefficient of the corresponding position in the first feature map;

[0220] Multiply the value of each position in the first feature map by the corresponding weight coefficient in the second feature map to obtain a fused feature map.

[0221] In a possible implementation, in terms of performing pixel filling on the image to be detected based on a preset filling method to obtain at least one first mask image, the processing unit 902 is specifically configured to:

[0222] Constructing at least one matrix according to a preset filling method, wherein the size of each matrix in the at least one matrix is ​​the same as the size of the image to be detected, and each matrix includes an element whose value is 0;

[0223] Each matrix is ​​dot-multiplied with the pixels of the image to be detected to fill the pixels corresponding to the elements with a value of 0 in the image to be detected with black pixels, thereby obtaining at least one first mask image, wherein the pixel filling area is the area filled with black pixels in each first mask image.

[0224] In a possible implementation, the processing unit 902 is further configured to:

[0225] Acquire a sample image and annotation data of a target component in the sample image;

[0226] Inputting the sample image into a neural network for target detection to obtain at least one third prediction frame, and determining a third target prediction frame from the at least one third prediction frame;

[0227] Scaling the sample image according to the third target prediction box and the grid to obtain a scaled sample image;

[0228] Perform pixel filling on the scaled sample image to obtain at least one second mask image;

[0229] The neural network is trained using at least one second mask image, the labeled data, and the loss function to obtain a neural network model.

[0230] An embodiment of the present application also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program for electronic data exchange, and the computer program enables a computer to execute part or all of the steps of any image detection method recorded in the above method embodiments.

[0231] An embodiment of the present application also provides a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and the computer program enables a computer to execute part or all of the steps of any image detection method recorded in the above method embodiments.

[0232] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0233] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0234] In the several embodiments provided in the present application, it should be understood that the disclosed device can be implemented in other ways. For example, the device embodiments described above are only schematic, such as the division of the units, which is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the device or unit can be electrical or other forms.

[0235] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0236] In addition, the functional units in the various embodiments of the application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated units may be implemented in the form of hardware or in the form of software program modules.

[0237] If the integrated unit is implemented in the form of a software program module and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a memory, including a number of instructions to enable a computer device (which can be a personal computer, server or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned memory includes: U disk, read-only memory (ROM), random access memory (RAM), mobile hard disk, disk or optical disk and other media that can store program codes.

[0238] A person of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable memory, which can include: a flash drive, a read-only memory, a random access memory, a magnetic disk or an optical disk, etc.

[0239] The embodiments of the present application are introduced in detail above. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method and core idea of ​​the present application. At the same time, for general technical personnel in this field, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. An image detection method, characterized in that: The method comprises: Obtain the image to be inspected of the high-speed railway contact network; Filling pixels of the image to be detected based on a preset filling method to obtain at least one first mask image; Detecting the at least one first mask image to determine whether there is a target component missing from the high-speed railway contact network; Each of the at least one first mask image includes a pixel-filled area obtained by performing pixel filling; and performing pixel filling on the image to be detected to obtain at least one first mask image includes: Constructing at least one matrix according to a preset filling method, wherein the size of each matrix in the at least one matrix is ​​the same as the size of the image to be detected, and each matrix includes an element whose value is 0; Perform a dot product of each matrix and the pixels of the image to be detected to fill the pixels corresponding to the elements with a value of 0 in the image to be detected with black pixels, and obtain the at least one first mask image, wherein the pixel filling area is the area filled with black pixels in each first mask image; the target component in the image to be detected is covered by black pixels in at least one first mask image.

2. The method according to claim 1, characterized in that: The detecting the at least one first mask image to determine whether the high-speed railway contact network has a target component missing includes: Performing target detection on the at least one first mask image to obtain a reference prediction box corresponding to each first mask image in the at least one first mask image; Fusing the reference prediction frames to obtain a first target prediction frame; According to the first target prediction frame, it is determined whether there are any target components missing from the high-speed railway contact network.

3. The method according to claim 2, characterized in that The performing target detection on the at least one first mask image to obtain a reference prediction frame corresponding to each first mask image in the at least one first mask image includes: Performing target detection on each of the first mask images to obtain a first prediction box corresponding to each of the first mask images; The first prediction box in each of the first mask images that falls in the pixel filling area is determined as the reference prediction box.

4. The method according to claim 2, characterized in that: The determining, according to the first target prediction frame, whether there is a target component missing in the high-speed railway contact network includes: Determine the position information of the first target prediction box in the image to be detected; Cutting out an image of a region to be classified from the image to be detected according to the position information; The image of the area to be classified is detected to determine whether there are any target components missing from the high-speed railway contact network.

5. The method according to any one of claims 1 to 4, characterized in that: The step of obtaining the image to be detected of the high-speed railway contact network includes: Performing target detection on the original image of the high-speed railway contact network to obtain a first target detection result; If the first target detection result includes at least one second prediction box, determining a second target prediction box from the at least one second prediction box, and obtaining the image to be detected according to the second target prediction box and a preset grid; If the first target detection result does not include the second prediction box, the image to be detected is obtained according to the preset rectangular box and the grid.

6. The method according to claim 5, characterized in that The obtaining the image to be detected according to the second target prediction frame and the preset grid includes: Calculate a first scaling factor according to the size of the grid and the width or height of the second target prediction box; The original image is scaled according to the first scaling factor to obtain the image to be detected.

7. The method according to claim 5, characterized in that The step of obtaining the image to be detected according to the preset rectangular frame and the grid includes: Calculate a second scaling factor according to the size of the grid and the width or height of the rectangular frame; The original image is scaled according to the second scaling factor to obtain the image to be detected.

8. The method according to claim 3, 4, 6 or 7, characterized in that: The performing target detection on each of the first mask images to obtain a first prediction frame corresponding to each of the first mask images includes: Perform at least one convolution, gated convolution, and weight coefficient multiplication operation, and use the resulting fused feature map as the target feature map; Performing classification prediction on each of the first mask images based on the target feature map to obtain the first prediction box; Among them, at least one convolution, gated convolution and weight coefficient multiplication operation includes: Convolving each of the first mask images or the feature images obtained in the last operation to obtain a first feature image; Performing gated convolution on each of the first mask images or the feature images obtained in the last operation to obtain a second feature image, where the second feature image has the same size as the first feature image, and the value of each position in the second feature image is used to represent the weight coefficient of the corresponding position in the first feature image; The value of each position in the first feature map is multiplied by the corresponding weight coefficient in the second feature map to obtain a fused feature map.

9. The method according to claim 6 or 7, characterized in that: The target detection on the at least one first mask image is performed by a neural network model, and the neural network model is trained by the following steps: Acquire a sample image and annotation data of a target component in the sample image; Inputting the sample image into a neural network for target detection to obtain at least one third prediction frame, and determining a third target prediction frame from the at least one third prediction frame; Scaling the sample image according to the third target prediction box and the grid to obtain a scaled sample image; Performing pixel filling on the scaled sample image to obtain at least one second mask image; The neural network is trained using the at least one second mask image, the labeled data and the loss function to obtain the neural network model.

10. An image detection device, characterized in that: The device comprises an acquisition unit and a processing unit; The acquisition unit is used to acquire the image to be detected of the high-speed railway contact network; The processing unit is used to fill pixels of the image to be detected based on a preset filling method to obtain at least one first mask image; The processing unit is further used to detect the at least one first mask image to determine whether there is a target component missing in the high-speed railway contact network; Each of the at least one first mask image includes a pixel-filled area obtained by performing pixel filling. In performing pixel filling on the image to be detected to obtain at least one first mask image, the processing unit is specifically used to: Constructing at least one matrix according to a preset filling method, wherein the size of each matrix in the at least one matrix is ​​the same as the size of the image to be detected, and each matrix includes an element whose value is 0; Perform a dot product of each matrix and the pixels of the image to be detected to fill the pixels corresponding to the elements with a value of 0 in the image to be detected with black pixels, and obtain the at least one first mask image, wherein the pixel filling area is the area filled with black pixels in each first mask image; the target component in the image to be detected is covered by black pixels in at least one first mask image.

11. An electronic device, characterized in that: The method comprises a processor, an input device, an output device and a memory, wherein the processor, the input device, the output device and the memory are interconnected, wherein the memory is used to store a computer program, the computer program comprises program instructions, and the processor is configured to call the program instructions to execute the method according to any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by a processor, the processor is caused to perform the method according to any one of claims 1 to 9.

Images