Method and device for training fitting detection network, and method and device for fitting identification

Abstract

The application discloses a kind of hardware detection network training method, device and hardware identification method, device.Hardware detection network includes area generation subnetwork, N identification subnetwork, the training method of hardware detection network includes: obtaining initial training set;Random number generated using area generation subnetwork, and according to random number and N from initial training set in training image in N preset size Segmentation N target area image, wherein, N is positive integer;N target area image is respectively one-to-one corresponding input N identification subnetwork and carries out hardware identification, and obtains hardware identification result;According to hardware identification result and corresponding real label, construct loss function, and according to loss function, respectively to area generation subnetwork and identification subnetwork are trained.The method can train to obtain hardware detection network, to realize high efficiency accurately by hardware detection network Hardware detection.

Description

Technical Field

[0001] This invention relates to the field of neural network technology, and in particular to a training method and apparatus for a hardware detection network, and a hardware identification method and apparatus. Background Technology

[0002] Ensuring the reliability of power transmission lines is a crucial aspect of the construction of the energy internet and smart grids. Fittings are essential and abundant metal components on power transmission lines. In related technologies, drones are commonly used for power transmission line inspections. Drone pilots, following safe flight protocols, operate the drones to designated locations to photograph and collect image data of the fittings at the ground wire connection points on high-voltage towers. Transmission line inspection personnel then examine each image captured by the drone to check if the fittings at the connection points need replacement.

[0003] However, in related technologies, because drones need to photograph the ground wire connection point data from outside the safe distance of the high-voltage power line tower, the target of the ground wire connection point hardware in the acquired images is very small. Existing target inspection algorithms in the field of image recognition struggle to accurately locate the hardware, and it is also difficult to accurately identify the effective position of the hardware's edges. The only option is to manually observe the image, magnifying it hundreds of times to determine whether the hardware needs replacement, which is unsatisfactory in both efficiency and accuracy. Summary of the Invention

[0004] This invention aims to at least partially solve one of the technical problems in related technologies. Therefore, the first objective of this invention is to propose a training method for a hardware inspection network, so as to obtain a hardware inspection network through training and achieve efficient and accurate hardware inspection.

[0005] The second objective of this invention is to provide a training device for a hardware detection network.

[0006] The third objective of this invention is to provide a method for testing metal fittings.

[0007] The fourth objective of this invention is to provide a hardware testing device.

[0008] To achieve the above objectives, a first aspect of the present invention proposes a training method for a hardware detection network. The hardware detection network includes a region generation subnetwork and N recognition subnetworks. The method includes: obtaining an initial training set; using random numbers generated by the region generation subnetwork, and segmenting N target region images from the training images in the initial training set according to the random numbers and N preset sizes, where N is a positive integer; inputting the N target region images one-to-one into the N recognition subnetworks for hardware recognition, and obtaining hardware recognition results; constructing a loss function based on the hardware recognition results and corresponding real labels, and training the region generation subnetwork and the recognition subnetworks according to the loss function.

[0009] The training method for the hardware detection network in this invention involves obtaining an initial training set, generating random numbers using a region generation sub-network, and segmenting N target region images from the training images in the initial training set based on the random numbers and N preset sizes. These N target region images are then input one-to-one into N recognition sub-networks for hardware identification, yielding hardware identification results. A loss function is constructed based on the hardware identification results and corresponding ground truth labels, and the region generation sub-network and recognition sub-network are trained using this loss function. This allows for the training of the hardware detection network, enabling hardware detection without manual observation, thus achieving more accurate and efficient hardware detection.

[0010] To achieve the above objectives, a second aspect of the present invention provides a training device for a hardware detection network. The hardware detection network includes a region generation subnetwork and N recognition subnetworks. The device includes: a first acquisition module for acquiring an initial training set; a segmentation module for segmenting N target region images from training images in the initial training set using random numbers generated by the region generation subnetwork and based on the random numbers and N preset sizes, where N is a positive integer; a first recognition module for inputting the N target region images one-to-one into the N recognition subnetworks for hardware recognition to obtain hardware recognition results; and a training module for constructing a loss function based on the hardware recognition results and corresponding real labels, and training the region generation subnetwork and the recognition subnetworks based on the loss function.

[0011] The training device for the hardware detection network in this embodiment of the invention acquires an initial training set, generates random numbers using a region generation sub-network, and segments N target region images from the training images in the initial training set based on the random numbers and N preset sizes. These N target region images are then input one-to-one into N recognition sub-networks for hardware identification, yielding hardware identification results. A loss function is constructed based on the hardware identification results and corresponding real labels, and the region generation sub-network and recognition sub-network are trained using this loss function. This allows for the training of the hardware detection network, enabling hardware detection without manual observation, achieving more accurate and efficient hardware detection.

[0012] To achieve the above objectives, a third aspect of the present invention provides a method for detecting metal fittings, the method comprising: acquiring an image to be detected; inputting the image to be detected into a metal fitting detection network trained according to the above-described training method for metal fitting detection network to perform metal fitting recognition, thereby obtaining a metal fitting recognition result.

[0013] The hardware detection method of this invention acquires an image to be detected and inputs the image to be detected into a hardware detection network trained using the above-mentioned hardware detection network training method to obtain hardware recognition results. This enables hardware recognition using a hardware detection network without the need for manual observation, achieving more accurate and efficient hardware detection.

[0014] To achieve the above objectives, a fourth aspect of the present invention provides a hardware detection device, the device comprising: a second acquisition module for acquiring an image to be detected; and a second recognition module for inputting the image to be detected into a hardware detection network trained by the above-mentioned hardware detection network training device to perform hardware recognition and obtain a hardware recognition result.

[0015] The hardware detection device of this invention acquires an image to be detected and inputs the image to be detected into a hardware detection network trained by the above-mentioned hardware detection network training device to obtain hardware identification results, thereby realizing hardware identification using the hardware detection network without manual observation, achieving more accurate and efficient hardware detection.

[0016] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0017] Figure 1 This is a flowchart of a training method for a hardware detection network according to an embodiment of the present invention;

[0018] Figure 2 This is a flowchart of a training method for a hardware detection network according to an example of the present invention;

[0019] Figure 3 This is a schematic diagram of a hardware detection network according to an example of the present invention;

[0020] Figure 4 This is a structural block diagram of the training device for the hardware detection network according to an embodiment of the present invention;

[0021] Figure 5 This is a flowchart of the hardware identification method according to an embodiment of the present invention;

[0022] Figure 6 This is a flowchart of an example of a hardware testing method according to the present invention;

[0023] Figure 7 This is a structural block diagram of the hardware testing device according to an embodiment of the present invention. Detailed Implementation

[0024] The training method and apparatus for the hardware detection network, and the hardware identification method and apparatus according to embodiments of the present invention are described below with reference to the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described with reference to the accompanying drawings are exemplary and should not be construed as limiting the present invention.

[0025] The training method of the hardware detection network according to an embodiment of the present invention is described below with reference to the accompanying drawings.

[0026] Figure 1 This is a flowchart of a training method for a hardware detection network according to an embodiment of the present invention.

[0027] In this embodiment of the invention, the hardware detection network includes a region generation subnetwork and N recognition subnetworks. The recognition subnetworks include a feature extraction module and a prediction / discrimination module, such as a YOLO V5 feature extraction module and a YOLO V5 prediction / discrimination module.

[0028] like Figure 1 As shown, the training method for the hardware detection network includes:

[0029] S11, Obtain the initial training set.

[0030] Specifically, the initial training set is a dataset of mounting point hardware targets designed based on hardware images obtained during actual applications. For example, images of ground wire mounting point hardware collected during UAV inspections can be obtained, and a dataset consisting of two types of data can be designed based on these images: typical front targets of crossarm U-shaped hardware and typical side targets of conductor end U-shaped hardware.

[0031] S12 uses a random number generated by a region generation subnetwork and segments N target region images from the training images in the initial training set according to the random number and N preset sizes, where N is a positive integer.

[0032] Specifically, the training images in the initial training set are obtained, and random numbers generated by the network are obtained to generate regions. Initial points are determined on the training images based on the random numbers, and the target region image is segmented from the training images based on the initial points and preset sizes.

[0033] It should be noted that since the location of the metal fitting in the training image is known, after acquiring the target region image, it is possible to determine whether the target region image contains the complete metal fitting, and this can be used as the true label of the target region image. Obviously, because the target region is initially randomly selected, the true label of the target region image is a random value.

[0034] As an example, the value of N above is 3, and the three preset sizes are 152*152 pixels, 304*304 pixels, and 608*608 pixels.

[0035] S13, input the N target area images one by one into the N recognition sub-networks for hardware recognition, and obtain the hardware recognition result.

[0036] S14. Construct a loss function based on the hardware recognition results and the corresponding real labels, and train the region generation subnetwork and the recognition subnetwork respectively based on the loss function.

[0037] As an example, the process first determines whether a complete piece of metal exists in the target region image based on the recognition result and the corresponding ground truth label. Specifically, since the recognition result is a rectangular region in the target region image, we can compare whether the recognized rectangular region completely overlaps with the region where the metal is located in the training image. If they completely overlap, then the presence of a complete piece of metal is confirmed. If the recognition result and the corresponding ground truth label determine that a complete piece of metal exists in the target region image, then the label corresponding to that target region image is marked as 1, and the ground truth distribution is obtained based on label 1. If the recognition result and the corresponding ground truth label determine that a complete piece of metal does not exist in the target region image, then the label corresponding to that target region image is marked as 0, and the non-ground truth distribution is obtained based on label 0.

[0038] Specifically, after obtaining the labels corresponding to the target region images, the probability distributions of the location information of the labeled target regions can be obtained by combining the positions of the hardware in the training images. These include the probability distributions of the hardware positions in the training images when the label is 1 and when the label is 0. The probability distribution of the hardware positions in the training images when the label is 1 is the true distribution, and the probability distribution of the hardware positions in the training images when the label is 0 is a false distribution. After obtaining the true and false distributions, the cross-entropy can be calculated based on the true and false distributions, and this cross-entropy can be used as the loss function.

[0039] After obtaining the loss function, see [link / reference] Figure 2 The region generation subnetwork and feature extraction module are trained using the loss function obtained from cross-entropy. For the prediction and discrimination module, training can be performed based on the recognition results of the target region image and the corresponding ground truth labels.

[0040] During the training of the region generation subnetwork, when the target region image input to the recognition subnetwork is a positive sample, if the label corresponding to the target region image is 1, a positive stimulus is fed back to the region generation subnetwork; if the label corresponding to the target region image is 0, a negative stimulus is fed back to the region generation subnetwork. Similarly, when the target region image input to the recognition subnetwork is a negative sample, if the label corresponding to the target region image is 1, a negative stimulus is fed back to the region generation subnetwork; if the label corresponding to the target region image is 0, a positive stimulus is fed back to the region generation subnetwork. Specifically, if the target region image includes complete metal fittings, it is a positive sample; if the target region image does not include complete metal fittings, it is a negative sample. When the target region image is a positive sample, the Boolean value input to the region generation subnetwork is 1; when the target region image is a negative sample, the Boolean value input to the region generation subnetwork is 0.

[0041] Thus, through continuous iteration, the aforementioned real and non-real distributions are made to continuously approach the required distribution. Ultimately, when the input of the region generation subnetwork is 1, the output target region is similar to the real distribution, that is, it contains the complete hardware. When the input of the region generation subnetwork is 0, the output target region is similar to the non-real distribution, that is, it does not contain the complete hardware.

[0042] In one embodiment of the present invention, the region generation subnetwork includes an input layer, an intermediate layer, and an output layer. The input signal received by the input layer is a Boolean value, and the output signal output by the output layer is two random numbers y1 and y2, wherein the values ​​of y1 and y2 are both in the range of [0, 1].

[0043] As an example, the aforementioned region generation subnetwork can consist of 3 DNN layers: one neuron in the output layer, 200 neurons in the intermediate layer, and two neurons in the output layer.

[0044] The process of segmenting N target region images from the training images in the initial training set based on random numbers and N preset sizes includes:

[0045] The starting point is determined in the training images according to the following formula:

[0046]

[0047] Where (sx, sy) are the coordinates of the starting point, and W0 and H0 are the width and height of the hardware image, respectively.

[0048] Furthermore, based on the starting point and N preset dimensions, N target region images are segmented from the hardware image.

[0049] Therefore, it is possible to obtain the target region image based on the training image.

[0050] The embodiments of the present invention will be described in detail below with reference to a specific example.

[0051] See this specific example. Figure 3 The multi-scale region generation model is the aforementioned region generation sub-network, and the 152*152 scale attachment point hardware target detection model, the 304*304 scale attachment point hardware target detection model, and the 608*608 scale attachment point hardware target detection model are the aforementioned N recognition sub-networks. In this specific example, the hardware detection network also includes a tower support target detection model, used to obtain the image to be detected from the original image data. The tower support target detection model is trained using a tower support target dataset, which is obtained from the attachment point hardware target dataset. It can be seen that the above three attachment point hardware target detection models constitute the discriminative model of the hardware detection network, and this discriminative model is the adversarial part, together with the multi-scale region generation model, forming a random region generation adversarial model.

[0052] Specifically, during training, the tower support target detection model is trained using data from the tower support target dataset, while data from the mounting point hardware target dataset is input into the multi-scale region generation model. The multi-scale region generation model segments the training images from the mounting point hardware target dataset to obtain target region images, which are then input into the three mounting point hardware target detection models. Since the tower support target dataset is derived from the mounting point hardware target dataset, the data from the tower support target dataset can be input into the multi-scale region generation model for training. Furthermore, data from the mounting point hardware target dataset also needs to be input into the mounting point hardware target detection model to input the positions of the hardware in the training images from the mounting point hardware target dataset, thereby training both the mounting point hardware target detection model and the multi-scale region generation model.

[0053] In summary, the training method for the hardware detection network in this embodiment of the invention involves obtaining an initial training set, generating random numbers using a region generation sub-network, segmenting N target region images from the training images in the initial training set based on the random numbers and N preset sizes, and then inputting each of the N target region images into N recognition sub-networks for hardware identification to obtain hardware identification results. A loss function is then constructed based on the hardware identification results and corresponding real labels, and the region generation sub-network and recognition sub-network are trained using the loss function. This allows for the training of the hardware detection network, enabling hardware detection without manual observation, thus achieving more accurate and efficient hardware detection.

[0054] Furthermore, the present invention proposes a training device for a hardware detection network.

[0055] Figure 4 This is a structural block diagram of the training device for the hardware detection network according to an embodiment of the present invention.

[0056] In this embodiment of the invention, the hardware detection network includes a region generation subnetwork and N identification subnetworks.

[0057] like Figure 4 As shown, the training device 100 for the hardware detection network includes: a first acquisition module 101, a segmentation module 102, a first recognition module 103, and a training module 104.

[0058] Specifically, the first acquisition module 101 is used to acquire an initial training set; the segmentation module 102 is used to segment N target region images from the training images in the initial training set using random numbers generated by the region generation sub-network and N preset sizes, where N is a positive integer; the first recognition module 103 is used to input the N target region images one by one into the N recognition sub-networks for hardware recognition to obtain hardware recognition results; and the training module 104 is used to construct a loss function based on the hardware recognition results and the corresponding real labels, and train the region generation sub-network and the recognition sub-network according to the loss function.

[0059] It should be noted that for other specific embodiments of the training device for the hardware detection network in this invention, please refer to the training method for the hardware detection network in the above embodiments.

[0060] The training device for the hardware detection network in this embodiment of the invention acquires an initial training set, generates random numbers using a region generation sub-network, and segments N target region images from the training images in the initial training set based on the random numbers and N preset sizes. These N target region images are then input one-to-one into N recognition sub-networks for hardware identification, yielding hardware identification results. A loss function is constructed based on the hardware identification results and corresponding real labels, and the region generation sub-network and recognition sub-network are trained using this loss function. This allows for the training of the hardware detection network, enabling hardware detection without manual observation, achieving more accurate and efficient hardware detection.

[0061] Furthermore, this invention proposes a method for testing metal fittings.

[0062] Figure 5 This is a flowchart of a hardware testing method according to an embodiment of the present invention.

[0063] like Figure 5 As shown, the methods for testing hardware include:

[0064] S51, acquire the image to be detected.

[0065] Specifically, the first step is to acquire raw image data, which is the image that needs to be inspected during actual application. For example, a drone pilot operates the drone to a designated location according to safe flight regulations to take pictures of the hardware at the ground wire connection point on the high-voltage tower and collect the image.

[0066] After acquiring the captured images, they are input into a pre-trained tower support target detection network to obtain the network to be detected. To train the tower support target detection network, a tower support target dataset needs to be acquired. This dataset is designed based on hardware images obtained during practical applications. For example, it can be designed using the characteristics of ground wire connection point images collected during UAV inspections. The dataset consists of two types of data: typical support targets marked with ground wire connection point hardware locations and typical support targets without ground wire connection point hardware locations.

[0067] S52, input the image to be detected into the hardware detection network trained according to the above-mentioned hardware detection network training method to perform hardware recognition and obtain hardware recognition results.

[0068] Specifically, after the image to be detected is input into the hardware detection network, the Boolean value of the input region generation subnetwork can act as a switch, so the Boolean value of the input region generation subnetwork needs to be set to 1.

[0069] At this point, using the region generation subnetwork in the hardware detection network, the starting point is determined in the image to be detected according to the following formula, and N target region images are segmented from the image to be detected based on the starting point and N preset sizes:

[0070]

[0071] Where (sx, sy) are the coordinates of the starting point, and W0 and H0 are the lengths of the two adjacent sides of the image to be detected, respectively.

[0072] Furthermore, the N target region images are input one-to-one into the N recognition subnetworks of the hardware detection network for hardware recognition. Specifically, a preset result fusion algorithm can be used to obtain the hardware recognition result based on the recognition results of the recognition subnetworks.

[0073] In one embodiment of the present invention, N is 3, and the N target image sizes include 152*152, 304*304, and 608*608. The N target region images include a first image of size 152*152, a second image of size 304*304, and a third image of size 608*608.

[0074] At this point, the above-mentioned method of inputting the N target region images one-to-one into the N recognition subnetworks of the hardware detection network for hardware recognition is described in detail. Figure 6The process includes: inputting a first image into a corresponding first recognition subnetwork; if the first recognition subnetwork identifies a complete hardware fitting from the first image, then outputting the corresponding hardware fitting recognition result; otherwise, inputting a second image into a corresponding second recognition subnetwork; if the second recognition subnetwork identifies a complete hardware fitting from the second image, then outputting the corresponding hardware fitting recognition result; otherwise, inputting a third image into a corresponding third recognition subnetwork; if the third recognition subnetwork identifies a complete hardware fitting from the third image, then outputting the corresponding hardware fitting recognition result; otherwise, determining that no hardware fitting was detected.

[0075] The following will continue to combine Figure 3 The examples shown illustrate embodiments of the present invention.

[0076] In this specific example, the multi-scale region generation model is the aforementioned region generation sub-network, and the 152*152 scale attachment point hardware target detection model, the 304*304 scale attachment point hardware target detection model, and the 608*608 scale attachment point hardware target detection model are the aforementioned N recognition sub-networks. In this specific example, the hardware detection network also includes a tower support target detection model, used to obtain the image to be detected from the original image data.

[0077] Specifically, during the detection process, raw image data captured by a drone is acquired and input into the tower support target detection model to obtain the image to be detected. This image is then input into a multi-scale region generation model. The multi-scale region generation model segments the target region image from the image to be detected and inputs it into the mounting point hardware target detection model for identification. Furthermore, the mounting point hardware target detection model also acquires the image to be detected output by the tower support target detection model to narrow down the target detection area. Since the image to be detected is a small region within the original image, and this region contains hardware, identifying the hardware within this area improves detection efficiency.

[0078] In summary, the hardware detection method of this invention acquires an image to be detected and inputs the image to be detected into a hardware detection network trained using the above-described hardware detection network training method to obtain hardware recognition results. This enables hardware recognition using a hardware detection network without the need for manual observation, achieving more accurate and efficient hardware detection.

[0079] Furthermore, the present invention proposes a hardware testing device.

[0080] Figure 7 This is a structural block diagram of the hardware testing device according to an embodiment of the present invention.

[0081] like Figure 7As shown, the hardware detection transpose 200 includes: a second acquisition module 201 and a second identification module 202.

[0082] Specifically, the second acquisition module 201 is used to acquire the image to be detected; the second recognition module 202 is used to input the image to be detected into the hardware detection network trained according to the above-mentioned hardware detection network training device for hardware recognition, and obtain the hardware recognition result.

[0083] It should be noted that other specific embodiments of the hardware testing device of the present invention can be found in the hardware testing method of the above embodiments.

[0084] The hardware detection device of this invention acquires an image to be detected and inputs the image to be detected into a hardware detection network trained by the above-mentioned hardware detection network training device to obtain hardware identification results, thereby realizing hardware identification using the hardware detection network without manual observation, achieving more accurate and efficient hardware detection.

[0085] It should be noted that the logic and / or steps represented in the flowchart or otherwise described herein can be considered as a ordered list of executable instructions for implementing logical functions, and can be specifically implemented in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Alternatively, the computer-readable medium may be paper or other suitable media on which the program can be printed, since the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.

[0086] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. If implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0087] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0088] In the description of this specification, the terms "center," "longitudinal," "lateral," "length," "width," "thickness," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," "outer," "clockwise," "counterclockwise," "axial," "radial," and "circumferential" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and should not be construed as limiting the present invention.

[0089] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0090] In this specification, unless otherwise stated, the terms "installation," "connection," "joining," and "fixing," etc., should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components, unless otherwise explicitly defined. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0091] In this invention, unless otherwise explicitly specified and limited, "above" or "below" the second feature can mean that the first feature is in direct contact with the second feature, or that the first feature is in indirect contact with the second feature through an intermediate medium. Furthermore, "above," "over," and "on top" of the second feature can mean that the first feature is directly above or diagonally above the second feature, or simply that the first feature is at a higher horizontal level than the second feature. "Below," "below," and "under" the second feature can mean that the first feature is directly below or diagonally below the second feature, or simply that the first feature is at a lower horizontal level than the second feature.

[0092] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A training method for a hardware detection network, characterized in that, The hardware detection network includes a region generation subnetwork and N recognition subnetworks, and the method includes: Obtain the initial training set; The random number generated by the region generation subnetwork is used to segment N target region images from the training images in the initial training set according to the random number and N preset sizes, where N is a positive integer; Each of the N target region images is input into one of the N recognition sub-networks to perform hardware recognition, and the hardware recognition result is obtained. A loss function is constructed based on the hardware identification results and the corresponding real labels, and the region generation subnetwork and the identification subnetwork are trained based on the loss function respectively. The region generation subnetwork includes an input layer, an intermediate layer, and an output layer. The input signal received by the input layer is a Boolean value, and the output signal output by the output layer is two random numbers y1 and y2, where the values ​​of y1 and y2 are both in the range [0, 1].

2. The training method for the hardware detection network according to claim 1, characterized in that, The step of segmenting N target region images from the training images in the initial training set according to the random number and N preset sizes includes: The starting point in the training image is determined according to the following formula: ， Where (sx, sy) are the coordinates of the starting point. and These are the width and height of the training image, respectively; Based on the starting point and N preset sizes, N target region images are segmented from the training image.

3. The training method for the hardware detection network according to claim 1, characterized in that, The step of constructing a loss function based on the recognition result and the corresponding real label includes: Based on the identification results and corresponding real labels, the true distribution and the non-true distribution are obtained; The cross-entropy is calculated based on the true distribution and the false distribution, and this cross-entropy is used as the loss function.

4. The training method for the hardware detection network according to claim 3, characterized in that, The step of obtaining the true distribution and non-true distribution based on the recognition result and the corresponding true label includes: If, based on the recognition results and the corresponding real labels, it is determined that there are complete metal fittings in the corresponding target area image, then the label corresponding to the target area image is marked as 1, and the real distribution is obtained based on label 1; If, based on the recognition results and the corresponding real labels, it is determined that there are no complete hardware fittings in the corresponding target area image, then the label corresponding to the target area image is marked as 0, and the non-real distribution is obtained based on label 0.

5. The training method for the hardware detection network according to claim 4, characterized in that, During the training of the region generation subnetwork. When the target region image input to the recognition sub-network is a positive sample, if the label corresponding to the target region image is 1, then positive incentives are fed back to the region generation sub-network; if the label corresponding to the target region image is 0, then negative incentives are fed back to the region generation sub-network. When the target region image input to the recognition sub-network is a negative sample, if the label corresponding to the target region image is 1, then negative incentives are fed back to the region generation sub-network; if the label corresponding to the target region image is 0, then positive incentives are fed back to the region generation sub-network.

6. A training device for a hardware detection network, characterized in that, The hardware detection network includes a region generation subnetwork and N identification subnetworks, and the device includes: The first acquisition module is used to acquire the initial training set; The segmentation module is used to generate random numbers using the region generation subnetwork, and to segment N target region images from the training images in the initial training set according to the random numbers and N preset sizes, where N is a positive integer; The first recognition module is used to input the N target area images one-to-one into the N recognition sub-networks to perform hardware recognition and obtain hardware recognition results. The training module is used to construct a loss function based on the hardware recognition results and the corresponding real labels, and to train the region generation subnetwork and the recognition subnetwork respectively based on the loss function; The region generation subnetwork includes an input layer, an intermediate layer, and an output layer. The input signal received by the input layer is a Boolean value, and the output signal output by the output layer is two random numbers y1 and y2, where the values ​​of y1 and y2 are both in the range [0, 1].

7. A method for testing hardware, characterized in that, The method includes: Acquire the image to be detected; The image to be detected is input into the hardware detection network trained according to the training method of the hardware detection network as described in any one of claims 1-5 for hardware identification, and the hardware identification result is obtained.

8. The method for testing hardware according to claim 7, characterized in that, Inputting the image to be detected into the hardware detection network for hardware identification includes: Using the region generation subnetwork in the hardware detection network, a starting point is determined in the image to be detected according to the following formula, and N target region images are segmented from the image to be detected based on the starting point and N target image sizes: ， Where (sx, sy) are the coordinates of the starting point. and These are the lengths of the two adjacent sides of the image to be detected; The images of the N target regions are input one-to-one into the N recognition subnetworks of the hardware detection network for hardware recognition.

9. The hardware testing method according to claim 8, characterized in that, The value of N is 3, and the N target image sizes include: 152 152, 304 304, 608 608, N of the target region images include 152 The first image is 152 pixels, 304 pixels. The second image is 304 in size and 608 in size. The third image is 608 pixels in size.

10. The method for testing metal fittings according to claim 9, characterized in that, The step of inputting the N target region images one-to-one into the N recognition subnetworks of the hardware detection network for hardware recognition includes: The first image is input into the corresponding first recognition sub-network; If the first recognition subnetwork identifies a complete hardware fitting from the first image, it outputs the corresponding hardware fitting recognition result; otherwise, it inputs the second image into the corresponding second recognition subnetwork. If the second recognition subnetwork identifies a complete hardware fitting from the second image, it outputs the corresponding hardware fitting recognition result; otherwise, it inputs the third image into the corresponding third recognition subnetwork. If the third recognition sub-network identifies a complete hardware fitting from the third image, it outputs the corresponding hardware fitting recognition result; otherwise, it determines that no hardware fitting was detected.

11. A hardware testing device, characterized in that, The device includes: The second acquisition module is used to acquire the image to be detected; The second recognition module is used to input the image to be detected into the hardware detection network trained according to the hardware detection network training device as described in claim 6 for hardware recognition, and obtain the hardware recognition result.

Images