Method, apparatus and system for detecting a subject to be detected

Abstract

This disclosure relates to a method, apparatus, and system for detecting an object to be detected. The method includes: acquiring data containing an image of the object to be detected; analyzing the image data to determine the position information and tilt angle of the object to be detected; determining position information and tilt angle for generating a prediction bounding box based on the position information and tilt angle of the object to be detected; generating a prediction bounding box matching the object to be detected based on the position information and tilt angle used to generate the prediction bounding box; and detecting the object to be detected using the prediction bounding box. The technical solution provided by this disclosure effectively reduces the introduction of background information during image detection, making the detection results more accurate and thus improving the detection accuracy.

Description

Technical Field

[0001] This disclosure relates to the field of target detection technology, and in particular to a method, apparatus and system for detecting an object to be detected. Background Technology

[0002] Deep learning is a rapidly growing artificial intelligence technology in recent years. Object detection algorithms implemented using deep learning technology boast advantages such as high speed and high accuracy. The framework of deep learning technology is the convolutional neural network (CNN), which extracts abstract features from images through convolutional kernels and uses these features to complete a specified task. To enable the network to perform a given task, the parameters within the network must first be adjusted using a large number of existing samples—this is the network training operation. After the parameters are adjusted, these parameters are saved to obtain the network model. Finally, the network model can be used to complete the task on unknown data.

[0003] Currently, object detection tasks require prior image calibration to obtain ground truth boxes, and the position and label information of these boxes must be saved. Existing ground truth boxes used for object calibration are only non-skewed (upright) rectangular boxes, such as... Figure 4 As shown. However, when the object to be detected is irregularly shaped, such as... Figures 3a to 3e When encountering irregular damage types (such as cracks and paint peeling) during aircraft skin damage inspection, the conventional calibration frame is used to detect these types of damage. Due to the irregular shape of these damages, there are many tilted damages. In this case, the conventional calibration frame will introduce more background information, which will reduce the accuracy of the inspection. Summary of the Invention

[0004] This disclosure provides a method, apparatus, and system for detecting an object to be tested, which solves the aforementioned technical problems in the prior art, reduces background information in the calibration frame, and improves detection accuracy.

[0005] According to one aspect of the present disclosure, a method for detecting an object to be detected is provided, comprising: acquiring data of an image containing the object to be detected; analyzing the image data to determine position information and tilt angle of the object to be detected; determining position information and tilt angle for generating a prediction box based on the position information and tilt angle of the object to be detected; generating a prediction box matching the object to be detected based on the position information and tilt angle for generating the prediction box; and detecting the object to be detected using the prediction box.

[0006] In an exemplary embodiment according to the present disclosure, generating a prediction box that matches the object to be detected based on position information and tilt angle used to generate the prediction box includes: generating multiple prediction boxes using a preset single-shot multi-frame detector (SSD) neural network based on position information and tilt angle used to generate the prediction box; obtaining a preset calibration box for the object to be detected; and determining the prediction box that matches the calibration box among the multiple prediction boxes.

[0007] In an exemplary embodiment according to the present disclosure, determining a prediction box that matches a calibration box among a plurality of prediction boxes includes: determining whether the tilt angle of the calibration box is 0°; when the tilt angle of the calibration box is 0°, determining a prediction box that matches the calibration box among the non-tilted prediction boxes among the plurality of prediction boxes; or when the tilt angle of the calibration box is not 0°, determining a prediction box that matches the calibration box among the tilted prediction boxes among the plurality of prediction boxes.

[0008] In an exemplary embodiment according to the present disclosure, determining a prediction box that matches a calibration box among a plurality of prediction boxes includes: obtaining at least one of the calibration box's size information, position information, tilt angle, and object category information located within the calibration box; and adjusting the prediction box according to at least one of the calibration box's size information, position information, tilt angle, and object category information to obtain a prediction box that matches the calibration box.

[0009] In an exemplary embodiment according to the present disclosure, adjusting a prediction box based on at least one of the size information, position information, tilt angle, and object category information of the calibration box to obtain a prediction box that matches the calibration box includes: obtaining the overlap area between each prediction box in a plurality of prediction boxes and the calibration box; determining the prediction box corresponding to the largest overlap area among the plurality of prediction boxes as the largest prediction box; and adjusting the largest prediction box based on at least one of the size information, position information, tilt angle, and object category information of the calibration box to obtain a prediction box that matches the calibration box.

[0010] According to another aspect of the present disclosure, an apparatus for detecting an object to be detected is provided, characterized in that it includes: an image acquisition module configured to acquire data of an image containing the object to be detected; an image analysis module configured to analyze the image data to determine position information and tilt angle of the object to be detected; a prediction box information determination module configured to determine position information and tilt angle for generating a prediction box based on the position information and tilt angle of the object to be detected; a prediction box generation module configured to generate a prediction box matching the object to be detected based on the position information and tilt angle used to generate the prediction box; and a detection module configured to detect the object to be detected using the prediction box.

[0011] In an exemplary embodiment according to the present disclosure, the prediction box generation module is further configured to: generate multiple prediction boxes using a preset single-shot multi-frame detector (SSD) neural network based on position information and tilt angle for generating prediction boxes; obtain a preset calibration box for the object to be detected; and determine the prediction box that matches the calibration box among the multiple prediction boxes.

[0012] In an exemplary embodiment according to the present disclosure, the prediction box generation module is further configured to: determine whether the tilt angle of the calibration box is 0°; when the tilt angle of the calibration box is 0°, determine a prediction box that matches the calibration box among the non-tilted prediction boxes of the plurality of prediction boxes; or when the tilt angle of the calibration box is not 0°, determine a prediction box that matches the calibration box among the tilted prediction boxes of the plurality of prediction boxes.

[0013] In an exemplary embodiment according to the present disclosure, the prediction box generation module is further configured to: acquire at least one of the size information, position information, tilt angle, and object category information within the calibration box; and adjust the prediction box according to at least one of the size information, position information, tilt angle, and object category information of the calibration box to obtain a prediction box that matches the calibration box.

[0014] In an exemplary embodiment according to the present disclosure, the prediction box generation module is further configured to: obtain the overlap area between each prediction box and the calibration box in a plurality of prediction boxes; determine the prediction box corresponding to the largest overlap area in the plurality of prediction boxes as the largest prediction box; and adjust the largest prediction box according to at least one of the size information, position information, tilt angle and object category information of the calibration box to obtain a prediction box that matches the calibration box.

[0015] According to another aspect of the present disclosure, an apparatus for detecting an object to be detected is provided, comprising: a memory and a processor, the processor being connected to the memory, the memory storing program instructions to be executed by the processor, and the processor being able to execute the above method when executing the program instructions.

[0016] According to another aspect of the present disclosure, a system for detecting an object to be detected is provided, comprising: an image acquisition device for acquiring data containing an image of the object to be detected; and the device for detecting the object to be detected as described above.

[0017] According to another aspect of the present disclosure, a computer-readable storage medium is provided, wherein a computer program including program instructions is stored on the computer-readable storage medium, the program instructions performing the above method when executed by a processor.

[0018] According to another aspect of the present disclosure, a processor is provided, wherein the processor is configured to run a program, wherein the program executes the above-described method during runtime.

[0019] According to another aspect of the present disclosure, a computer program product is provided, wherein the computer program product includes a computer program stored on a non-transitory computer-readable storage medium, the computer program including program instructions that, when executed by a computer, cause the computer to perform the above-described method.

[0020] The methods, apparatus, and systems for detecting objects according to the embodiments of this disclosure determine the position information and tilt angle for generating prediction boxes, generate prediction boxes that match the objects to be detected based on the position information and tilt angle, and use the prediction boxes to perform detection processing on the objects to be detected. This effectively reduces the introduction of background information during image detection, making the detection results more accurate. Moreover, the prediction boxes in the embodiments of this disclosure can better fit the shape of the objects to be detected, further improving the detection accuracy, thereby effectively improving the stability and reliability of the detection process. Attached Figure Description

[0021] The accompanying drawings, which are included to provide a further understanding of this disclosure and form part of this disclosure, illustrate exemplary embodiments of the disclosure and, together with the description thereof, serve to explain this disclosure and do not constitute an undue limitation thereof. In the drawings:

[0022] Figure 1a This is a schematic diagram of an 8×8 feature mapping according to the present disclosure;

[0023] Figure 1b This is a schematic diagram of a 4×4 feature mapping according to the present disclosure;

[0024] Figure 2 This is a schematic diagram illustrating the relationship between the prediction boxes of a feature mapping unit according to this disclosure;

[0025] Figure 3a This is a schematic diagram of irregular damage caused by a crack according to the present disclosure;

[0026] Figure 3b This is a schematic diagram of irregular damage caused by perforation according to the present disclosure;

[0027] Figure 3c This is a schematic diagram of irregular damage caused by a rivet according to the present disclosure;

[0028] Figure 3d This is a schematic diagram of irregular damage caused by a scratch according to the present disclosure;

[0029] Figure 3e This is a schematic diagram of irregular damage caused by paint chipping, according to the present disclosure;

[0030] Figure 4 This is a diagram illustrating the effect of a damage detection method according to this disclosure;

[0031] Figure 5 This is a flowchart of a method for detecting an object to be detected according to an embodiment of the present disclosure;

[0032] Figure 6 A flowchart illustrating how a prediction box matching the object to be detected is generated based on positional information and tilt angle used to generate the prediction box, according to an exemplary embodiment of the present disclosure.

[0033] Figure 7 A flowchart illustrating the process of determining a prediction box that matches a calibration box among a plurality of prediction boxes according to an exemplary embodiment of the present disclosure;

[0034] Figure 8 A flowchart illustrating the process of determining a prediction box that matches a calibration box among a plurality of prediction boxes according to another exemplary embodiment of the present disclosure;

[0035] Figure 9 A flowchart illustrating, according to an exemplary embodiment of the present disclosure, adjusting a prediction box based on at least one of calibration box size information, position information, tilt angle, and object category information to obtain a prediction box that matches the calibration box;

[0036] Figure 10 This is a rendering of a damage detection method according to an embodiment of the present disclosure;

[0037] Figure 11 This is a block diagram of an apparatus for detecting an object to be detected according to an embodiment of the present disclosure;

[0038] Figure 12 A block diagram of a system for detecting an object to be detected according to an embodiment of the present disclosure; and

[0039] Figure 13 This is a block diagram of a system for detecting an object to be detected according to another embodiment of the present disclosure. Detailed Implementation

[0040] To enable those skilled in the art to better understand the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are merely a part of the embodiments of the present disclosure, and not all of them. The following exemplary embodiments are only used to illustrate the present disclosure and are not intended to limit the scope of the present disclosure. All other embodiments obtained by those skilled in the art based on the embodiments of the present disclosure without creative effort should fall within the protection scope of the present disclosure.

[0041] It should be noted that the terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such terms can be used interchangeably where appropriate so that the embodiments of this disclosure described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or device that comprises a series of steps, modules, or units is not necessarily limited to those explicitly listed, but may include other steps, modules, or units not explicitly listed or inherent to such processes, methods, products, or devices.

[0042] To facilitate understanding of this disclosure, the following is a description of key terms:

[0043] Object / target detection: Locate objects / targets, determine their position and size.

[0044] Feature mapping: In each convolutional layer, the data exists in a three-dimensional form. A convolutional layer can be viewed as multiple two-dimensional images superimposed together, and each two-dimensional image is called a feature mapping. If the input layer contains a grayscale image, there is only one feature mapping; if the input layer contains a color image, there are generally three feature mappings (red, green, and blue).

[0045] Feature map cell: refers to each cell in the feature map, such as... Figure 1a and Figure 1b As shown, there are 64 and 16 units respectively.

[0046] Predicted bounding boxes: A series of fixed-size boxes on each feature map unit, such as... Figure 1a and Figure 1b As shown.

[0047] Single Shot MultiBox Detector (SSD): A target detection method that belongs to the single-stage type.

[0048] Intersection over Union (IOU): Since algorithms cannot perfectly match manually labeled data, there is a positioning accuracy evaluation formula: IOU, which defines the degree of overlap between two bounding boxes.

[0049] Implementing an object detection algorithm requires two main steps: creating a dataset and training a neural network. In the SSD object detection algorithm, both low and high feature maps can be used for detection simultaneously. Before object detection, the image needs to be calibrated to obtain bounding boxes, and the position and label information of these boxes must be saved. The label information indicates the type of the object within the bounding box. Existing object detection calibration tools can calibrate upright (non-skewed) rectangles and save the calibration results in an XML file. The XML file contains the object label within the bounding box and the object's position information.

[0050] Figure 1a This is a schematic diagram of an 8×8 feature mapping according to the present disclosure. Figure 1b This is a schematic diagram of a 4×4 feature mapping according to this disclosure. For example... Figure 1a and Figure 1b As shown, there are two different feature maps: 8×8 and 4×4.

[0051] If a feature map is m×n in size, then there are m×n feature map units. Assuming each feature map unit has k predicted boxes, and for each predicted box, c class scores and 4 offsets need to be predicted, the feature map has a total of (c+4)×k×m×n outputs. The number of these outputs represents the number of convolutional kernels used when convolving the feature map of this layer with 3×3 kernels (in actual code, different numbers of 3×3 kernels are used to convolve the feature map of this layer). It includes two parts: c×k×m×n is the confidence output, representing the confidence of each predicted box, i.e., the probability of the class; 4×k×m×n is the localization output, representing the regressed coordinates of each predicted box. During training, there is also a prior box parameter. The final output prior box is actually the ultimately selected predicted box, because not every predicted box needs to be selected for each feature map unit.

[0052] It's important to note that the predicted bounding box is a concept, while the prior bounding box is the actual selection. During training, a complete image is fed into the network to obtain various feature maps. For positive sample training, the prior bounding boxes need to be matched with the labeled bounding boxes. A successful match indicates that the prior bounding box contains an object, but it is still some distance from the complete object's labeled bounding box. The goal of training is to ensure the classification confidence of the predicted bounding box while regressing the prior bounding box to the labeled bounding box as closely as possible.

[0053] Figure 2 This is a schematic diagram illustrating the relationship between the prediction boxes of a feature mapping unit according to this disclosure. For example...

[0054] Figure 2 As shown, the SSD network framework can generate prediction boxes of different sizes and aspect ratios when generating prediction boxes. It generates a series of concentric prediction boxes centered on the midpoint of each point on the feature map (offset = 0.5). The coordinates of the center point are then multiplied by the step size, effectively mapping the location from the feature map position back to the original image position.

[0055] The SSD network framework generates multiple feature maps. The minimum side length (min_size) and maximum side length (max_size) of the square prediction box differ across these feature maps: For the first feature map, the minimum and maximum side lengths of the prediction box need to be pre-defined. The minimum side length of the prediction box on the second feature map is the maximum side length of the previous feature map, and the maximum side length of the prediction box on the second feature map is obtained using the following formula. The sizes of the prediction boxes on other feature maps follow the same logic. Therefore, the minimum side length (min_size) and maximum side length (max_size) of the prediction box corresponding to each feature map are determined by the following formula:

[0056]

[0057] In the formula, m is the number of feature maps used (m = 6 in SSD 300), min_size of the k-th layer = Sk, and max_size of the k-th layer = Sk + 1.

[0058] like Figure 2 As shown, the minimum side length of the square prediction box is min_size, and the maximum side length is... Setting an aspect ratio in prototxt (a file containing parameters for the SSD object detection algorithm) will generate two rectangles with dimensions of: and In other words, the area of ​​the generated rectangular prediction box is the square of the smallest side length. The area of ​​the smallest square prediction box in the middle is also the square of the smallest side length. Only the area of ​​the largest outer square prediction box is the sum of the smallest side length and the largest side length. For example, if the aspect ratio is 2, two square and two rectangular prediction boxes will be generated.

[0059] Figure 3a This is a schematic diagram of irregular damage caused by a crack according to the present disclosure; Figure 3b This is a schematic diagram of irregular damage caused by perforation according to the present disclosure; Figure 3c This is a schematic diagram of irregular damage caused by a rivet according to the present disclosure; Figure 3d This is a schematic diagram of irregular damage caused by a scratch according to the present disclosure; and Figure 3e This is a schematic diagram of irregular damage caused by paint chipping, according to the present disclosure.

[0060] Figure 4 This is an image illustrating the effect of a damage detection method according to this disclosure. The dataset creation requires image calibration software to outline the damage on the image. This software saves the damage type and location information in an XML file. However, the calibration boxes used to calibrate the objects to be detected are only non-skewed rectangles, such as... Figure 4 As shown. When the object to be detected has an irregular structure, such as... Figures 3a-3e The irregular damage types (e.g., cracks and paint peeling) encountered during aircraft skin damage inspection often exhibit slanted shapes due to their irregular shapes. In such cases, [the following is a more detailed description of the process:] ... Figure 4 When the conventional calibration box shown is used to detect this type of damage, it introduces a lot of background information, which reduces the accuracy of calibration and detection.

[0061] This disclosure enables the detection process to generate a tilted rectangular frame while retaining information about the tilt angle. The use of a tilted frame, especially for irregularly shaped damage such as paint chips and cracks, can better fit the edges of the damage and reduce the introduction of background information.

[0062] Figure 5 This is a flowchart of a method for detecting an object to be detected according to an embodiment of the present disclosure. Figure 5 As shown, this disclosure provides a method for detecting an object to be detected. This method can generate a tilted prediction box, thereby improving the accuracy and efficiency of the calibration and detection of the object. The method includes:

[0063] Step S1: Acquire image data containing the object to be detected. The object to be detected can be various types of damage, but is not limited to these. Specifically, the object to be detected can be... Figures 3a to 3e The diagram illustrates various irregular damage shapes on the aircraft skin, such as cracks or paint chips. Image data containing these damages can be pre-captured and stored in memory, then retrieved from memory during method execution. Alternatively, images of the various damages can be captured using an image sensor, such as a camera, and the captured images can be received directly from the sensor via wired or wireless communication.

[0064] Step S3 involves analyzing the image data to determine the position information and tilt angle of the object to be detected. Specifically, image recognition analysis can be performed on previously acquired images containing information about the object to be detected and the background to determine various information such as the position, shape, size, and tilt angle of the object to be detected. The image analysis method can be a known method.

[0065] Step S5: Determine the position information and tilt angle for generating the prediction box based on the position information and tilt angle of the object to be detected. After determining the position information and tilt angle of the object to be detected, estimate the position information and tilt angle of the prediction box based on the above position information and tilt angle. The position information may include at least the position information of the upper left corner and the position information of the lower right corner, but is not limited to these. Generally, the prediction box is a rectangular or square structure. Therefore, the approximate position and size of the prediction box can be estimated using the determined position information of the upper left corner and the lower right corner. In addition, the tilt angle of the prediction box in this embodiment includes 0°, 180° and other angles, such as 5°, 15°, 25°...175°, etc.

[0066] Step S7: Generate a prediction box that matches the object to be detected based on the position information and tilt angle used to generate the prediction box. After obtaining the top-left corner position information, bottom-right corner position information, and tilt angle, a prediction box that matches the object to be detected is generated based on the above information. In this embodiment, matching can mean that the size and tilt angle of the prediction box are the same as those of the object to be detected; or, the similarity between the size and tilt angle of the prediction box and those of the object to be detected is greater than a preset similarity threshold. For example, when the tilt of the object to be detected is 30°, the tilt angle of the generated prediction box can be 30°, so that the tilt angle of the prediction box and the object to be detected is as consistent as possible, thereby effectively reducing the amount of background information in the prediction box.

[0067] Step S9: Detect the object to be detected using the predicted bounding box. After determining the predicted bounding box, the object to be detected is detected using the predicted bounding box. Other detection processing can also be performed on the object to be detected located within the predicted bounding box. Because the predicted bounding box and the object to be detected have a high matching degree, the introduction of background information during image detection is reduced, making the detection results more accurate.

[0068] The method for detecting an object provided in this disclosure determines position information such as the upper left corner and lower right corner, as well as a tilt angle, for generating a prediction bounding box. Based on the position information and tilt angle, a prediction bounding box matching the object to be detected is generated. This method utilizes the prediction bounding box for object detection, effectively reducing the introduction of background information during image detection and resulting in more accurate detection results. Furthermore, the prediction bounding box in this embodiment can better fit the shape edges of the object to be detected, further improving detection accuracy and thus effectively enhancing the stability and reliability of the method.

[0069] Figure 6 This is a flowchart illustrating, according to an exemplary embodiment of the present disclosure, the generation of a prediction bounding box matching an object to be detected based on position information and tilt angle used to generate the prediction bounding box. This embodiment does not limit the specific implementation of generating the prediction bounding box matching the object to be detected based on the position information and tilt angle used to generate the prediction bounding box; those skilled in the art can set it according to specific design requirements. Preferably, such as... Figure 6 As shown, generating a prediction box that matches the object to be detected, based on the position information and tilt angle used to generate the prediction box, includes:

[0070] Step S71: Using a pre-defined single-shot multi-frame detector (SSD) neural network, multiple prediction boxes are generated based on the position information and tilt angle used to generate the prediction boxes. In order to predict prediction boxes that are as close as possible to the calibration boxes, the SSD neural network can set a large number of coarse prediction boxes on the image based on the position information and tilt angle. These prediction boxes do not match the calibration boxes very well, so the prediction boxes still need to be adjusted.

[0071] Step S73: Obtain the preset calibration box for the object to be detected. The calibration box can serve as a reference standard for adjusting the prediction box. The calibration box can be the outer contour of the object to be detected, or it can refer to the calibration area generated based on the outer contour of the object to be detected. The calibration box can be a square or rectangular box preset by a person based on the outer contour or shape of the image to be calibrated.

[0072] Step S75: Determine the prediction box that matches the calibration box from among the multiple prediction boxes. After obtaining the multiple prediction boxes generated by the SSD neural network and the preset calibration box, the calibration box can be compared with each prediction box in the multiple prediction boxes in one or more aspects, or the overlap area between each prediction box and the calibration box can be measured, etc., and the prediction box that matches the calibration box can be selected from the multiple prediction boxes based on the comparison or measurement results.

[0073] Figure 7 This is a flowchart illustrating the process of determining a prediction box that matches a calibration box among a plurality of prediction boxes according to an exemplary embodiment of the present disclosure. This embodiment does not limit the specific implementation of determining the prediction box that matches the calibration box among a plurality of prediction boxes; those skilled in the art can configure it according to specific design requirements. Preferably, such as... Figure 7 As shown, in this embodiment, determining the prediction box that matches the calibration box among multiple prediction boxes may include:

[0074] S751: Determine whether the tilt angle of the calibration frame is 0°;

[0075] S753: When the tilt angle of the calibration box is 0°, determine the prediction box that matches the calibration box among the non-tilted prediction boxes in a plurality of prediction boxes; or

[0076] S755: When the tilt angle of the calibration box is not 0°, determine the prediction box that matches the calibration box among the tilted prediction boxes among multiple prediction boxes.

[0077] By determining whether the tilt angle of the calibration box is 0°, a prediction box matching the calibration box is determined from either non-tilted or tilted prediction boxes based on the determination result. If the tilt angle of the calibration box is 0°, the prediction box matching the calibration box is determined from the non-tilted prediction boxes (positive prediction boxes) among multiple prediction boxes. Since the number of prediction boxes to be considered is reduced, the workload of subsequent steps can be saved.

[0078] Figure 8 This is a flowchart illustrating the process of determining a prediction box that matches a calibration box among multiple prediction boxes according to another exemplary embodiment of the present disclosure. This embodiment does not limit the specific implementation of determining the prediction box that matches the calibration box among multiple prediction boxes; those skilled in the art can configure it according to specific design requirements. Preferably, such as... Figure 8 As shown, determining the prediction box that matches the calibration box among multiple prediction boxes may include:

[0079] Step S757: Obtain at least one of the following: size information, position information, tilt angle, and object category information within the calibration frame; wherein, the object category information may include people, animals, scenery, or buildings, etc.

[0080] Step S759 involves adjusting the prediction box based on at least one of the calibration box's size information, position information, tilt angle, and object category information to obtain a prediction box that matches the calibration box. Specifically, after obtaining the size information, position information, tilt angle, and object category information, the position and size information of the prediction box can be adjusted based on these information. Furthermore, the tilt angle of the prediction box can be adjusted using the calibration box's tilt angle. For example, when the calibration box is large, the prediction box can be enlarged based on its size and position information; when the calibration box is small, it can be reduced. When the calibration box is tilted, its tilt angle can be determined. Based on this tilt angle, the tilt angle used to adjust the prediction box can be determined, and the prediction box can be rotated and adjusted using this tilt angle to match the calibration box. Of course, those skilled in the art can also use other implementation methods according to design requirements, which will not be elaborated here.

[0081] It should be noted that in this embodiment... Figure 8 The technical solution shown is the same as the one described above. Figure 7 The steps in the illustrated embodiments are independent of each other; they can be implemented individually or in combination. For example, the steps in this embodiment can be targeted at... Figure 7 The implementation of multiple tilted or non-tilted prediction boxes obtained in the manner shown effectively improves execution efficiency while further enhancing the accuracy of the method.

[0082] Figure 9 This is a flowchart illustrating, according to an exemplary embodiment of the present disclosure, the adjustment of a prediction box based on at least one of calibration box size information, position information, tilt angle, and object category information to obtain a prediction box that matches the calibration box. Based on the above embodiments, further reference is made to the appendix... Figure 8 It is understood that the specific implementation method for adjusting the prediction box based on at least one of the calibration box's size information, position information, tilt angle, and object category information to obtain a prediction box that matches the calibration box is not limited. Those skilled in the art can set it according to specific design requirements. Preferably, such as Figure 9 As shown, in this embodiment, adjusting the prediction box based on at least one of the calibration box's size information, position information, tilt angle, and object category information to obtain a prediction box that matches the calibration box may include:

[0083] Step S7591: Obtain the overlap area between each prediction box and the calibration box in the multiple prediction boxes. The overlap area is the area of ​​the overlapping region between the prediction box and the calibration box. This can be achieved by first obtaining region A occupied by the prediction box, and then region B occupied by the calibration box; the overlap area is the area of ​​A∩B. Alternatively, the overlap area ratio between each prediction box and the calibration box can be obtained. The prediction box with the largest overlap area ratio is determined as the largest prediction box. Then, the position and size information of the largest prediction box are adjusted to obtain a prediction box that matches the calibration box. Overlap area ratio = Overlap area / (Area of ​​prediction box + Area of ​​calibration box).

[0084] Step S7593: Determine the prediction box corresponding to the largest overlap area among the multiple prediction boxes, and designate it as the largest prediction box. Since there are multiple prediction boxes, multiple overlap areas can be obtained. It should be noted that when a prediction box and the calibration box have no overlap, the overlap area is 0. When obtaining multiple overlap areas, their sizes can be compared to obtain the largest overlap area. The corresponding largest prediction box is then determined based on the largest overlap area; this largest prediction box is the one with the highest similarity to the calibration box.

[0085] Step S7595: Adjust the maximum prediction box based on at least one of the calibration box's size information, position information, tilt angle, and object category information to obtain a prediction box that matches the calibration box. After determining the maximum prediction box, adjust the position and size information of the maximum prediction box using at least one of the calibration box's size information, position information, tilt angle, and object category information to make the maximum prediction box better match the calibration box, thereby obtaining a prediction box that matches the calibration box.

[0086] Obtaining a prediction box that matches the calibration box using the above method not only ensures the accuracy of the prediction box determination, but also allows the prediction box to be tilted, which can better fit the shape edge of the object to be detected, further improving the detection accuracy and expanding the applicability of the method.

[0087] Figure 10 This is a diagram illustrating the effect of a damage detection method according to an embodiment of this disclosure. Figure 10 As shown, by applying the above method, the tilted prediction box is used to better fit the shape edge of the object to be detected, effectively reducing the introduction of background information during the image detection process, making the detection results more accurate, and thus improving the detection accuracy.

[0088] The application example of this disclosure applies the above-described object detection method to the detection of irregular damage types (such as rivet damage, perforation, scratches, cracks, and paint peeling) encountered during aircraft skin damage detection. Although these damages are irregular in shape and often exhibit tilted shapes, when using the method described in the embodiments of this disclosure to detect such damage, the tilted prediction box better fits the shape edge of the object to be detected, effectively reducing the introduction of background information during image detection, making the detection results more accurate, and thus improving the detection accuracy.

[0089] Compared to conventional non-tilted prediction boxes, the embodiments of this disclosure require modifications to the calibration software and the object detection network so that both regular rectangular boxes and rectangular boxes rotated by a certain angle can be calibrated during the object detection process, and SSD can identify and match any rectangular box.

[0090] 1. The software used for manual image calibration in target detection can be modified to not only calibrate common rectangular boxes but also save tilted rectangular boxes rotated at a certain angle, especially useful in cases of damage such as cracks and paint peeling on aircraft skin. In this application example, in addition to obtaining the top-left and bottom-right corner position information (coordinates) used to generate the prediction box, it is also necessary to save the tilt angle information of the calibration rectangle, such as 0° or 45°. This minimizes the introduction of other background information when calibrating damage types such as cracks and paint peeling with a certain tilt angle.

[0091] 2. In order to predict bounding boxes that are as close as possible to the calibration boxes, the SSD neural network can set a large number of coarsely set bounding boxes on the image according to the pre-set aspect ratio and size. These bounding boxes cannot match the calibration boxes very well. Training the SSD neural network allows the network to learn how to adjust the position and size of these bounding boxes in order to match the calibration boxes, and one calibration box corresponds to only one bounding box.

[0092] For a given bounding box, the network will select the predicted box that is closest to the bounding box for matching. Because the form of the bounding box has changed, in order for the network to also recognize and match rectangles rotated by a certain angle, the network structure of the traditional object detection algorithm—SSD—will be modified. SSD will then be able to generate predicted boxes rotated by a certain angle. When matching with the bounding box, the program first determines the tilt angle parameter of the bounding box. If the tilt angle is 0°, a non-tilted rectangle is selected for matching; if the parameter is θ°, a rectangle rotated by θ° is selected for matching. Figure 10 As shown; then, the positions of the upper left and lower right corners are matched by calculating the overlap area with the calibration box.

[0093] Specifically, during training, the labeled boxes and predicted boxes (i.e., prior boxes) are paired in the following manner (1) or (2):

[0094] (1) Obtain the prediction box with the highest similarity to each calibration box.

[0095] Find the prediction box that has the largest Jaccard overlap with each calibration box. This ensures that each calibration box corresponds to a unique prediction box (the overlap area is called IOU). IOU = SI / (SA+SB-SI), where SI is the overlapping area between the calibration box and the prediction box, SA can be the area where the calibration box is located, and SB can be the area where the prediction box is located. The larger the IOU, the higher the correlation between the calibration box and the prediction box.

[0096] (2) Matching the prediction boxes

[0097] SSD can also attempt to pair all predicted bounding boxes with any calibration bounding box. The specific matching process can be achieved using the following formula: as long as the overlap area between the two is greater than the threshold (preset by the user or the system default), it is considered a match (the threshold for SSD 300 is 0.5). This can effectively reduce the occurrence of missed detections.

[0098]

[0099] Where x is the Jaccard coefficient, ranging from (0, 1); c is the confidence score; l is the predicted bounding box; g is the calibration box; and L... conf It is confidence loss, L loc This is the localization loss, and α is a preset parameter. It should be noted that both the confidence loss and localization loss formulas can be obtained using existing technologies.

[0100] (3) Identification of background information and the object to be detected

[0101] Obviously, the predicted bounding box that matches the calibration box is positive, and the predicted bounding box that does not match the calibration box is negative. The matching box set to positive is used to help the network recognize which locations in the image are the objects to be detected, while the matching box set to negative is used to help the network recognize which locations are the background.

[0102] The method provided in this embodiment can effectively reduce the introduction of background information during image detection, making the detection results more accurate. In addition, the prediction box with a tilt angle can better fit the shape edge of the object within the calibration box, improving the detection accuracy and thus effectively ensuring the accuracy and reliability of the method implementation.

[0103] Figure 11 This is a block diagram of an apparatus for detecting an object to be detected according to an embodiment of the present disclosure. Figure 11 As shown, this embodiment provides an apparatus 10 for detecting an object to be detected. The apparatus performs the detection method described above. The apparatus 10 includes: an image acquisition module 101 configured to acquire data of an image containing the object to be detected; an image analysis module 103 configured to analyze the image data to determine the position information and tilt angle of the object to be detected; a prediction box information determination module 105 configured to determine the position information and tilt angle for generating a prediction box based on the position information and tilt angle of the object to be detected; a prediction box generation module 107 configured to generate a prediction box matching the object to be detected based on the position information and tilt angle used to generate the prediction box; and a detection module 109 configured to detect the object to be detected using the prediction box.

[0104] This embodiment does not limit the specific configuration of the image acquisition module 101, image analysis module 103, prediction box information determination module 105, prediction box generation module 107, and detection module 109. Those skilled in the art can arbitrarily configure them according to their functions and effects, and will not be elaborated here. In addition, the specific implementation process and effect of the operation steps implemented by the image acquisition module 101, image analysis module 103, prediction box information determination module 105, prediction box generation module 107, and detection module 109 in this embodiment are the same as the specific implementation process and effect of steps S1 to S9 in the above embodiment. For details, please refer to the above description, and will not be elaborated here.

[0105] Based on the above embodiments, please continue to refer to the appendix. Figure 11 It is understood that this embodiment does not limit the specific implementation of the prediction box generation module 107 in generating a prediction box that matches the object to be detected based on the position information and tilt angle used to generate the prediction box. Those skilled in the art can set it according to specific design requirements. Preferably, when the prediction box generation module 107 generates a prediction box that matches the object to be detected based on the position information and tilt angle used to generate the prediction box, the prediction box generation module 107 can be configured to: use a preset single-shot multi-frame detector (SSD) neural network to generate multiple prediction boxes based on the position information and tilt angle used to generate the prediction box; obtain a preset calibration box for the object to be detected; and determine the prediction box that matches the calibration box among the multiple prediction boxes.

[0106] When the prediction box generation module 107 determines the prediction box that matches the calibration box among multiple prediction boxes, the prediction box generation module 107 can be configured to: determine whether the tilt angle of the object to be detected is 0°; when the tilt angle of the object to be detected is 0°, determine the prediction box that matches the calibration box among the non-tilted prediction boxes among multiple prediction boxes; or when the tilt angle of the object to be detected is not 0°, determine the prediction box that matches the calibration box among the tilted prediction boxes among multiple prediction boxes.

[0107] Furthermore, this embodiment does not limit the specific implementation method of the prediction box generation module 107 determining the prediction box that matches the calibration box among multiple prediction boxes. Those skilled in the art can set it according to specific design requirements. Preferably, another implementation method in this embodiment is: when the prediction box generation module 107 determines the prediction box that matches the calibration box among multiple prediction boxes, the prediction box generation module 107 can be configured to: obtain at least one of the size information, position information, tilt angle and object category information of the calibration box; and adjust the prediction box according to at least one of the size information, position information, tilt angle and object category information of the calibration box to obtain the prediction box that matches the calibration box.

[0108] When the prediction box generation module 107 adjusts the prediction box according to at least one of the size information, position information, tilt angle and object category information of the calibration box to obtain a prediction box that matches the calibration box, the prediction box generation module 107 can be configured to: obtain the overlap area between each prediction box and the calibration box in a plurality of prediction boxes; determine the prediction box corresponding to the largest overlap area in the plurality of prediction boxes as the largest prediction box; and adjust the largest prediction box according to at least one of the size information, position information, tilt angle and object category information of the calibration box to obtain a prediction box that matches the calibration box.

[0109] The apparatus for detecting the object to be detected provided in this embodiment can be used to perform... Figures 5 to 9 The method shown is similar in its specific implementation and beneficial effects to the description of the method, and will not be repeated here.

[0110] An apparatus for detecting an object to be detected according to an embodiment of the present disclosure includes: a memory and a processor, the processor being connected to the memory, the memory storing program instructions to be executed by the processor, and the processor being able to execute the above method when executing the program instructions.

[0111] Figure 12 This is a block diagram of a system for detecting an object to be detected according to an embodiment of the present disclosure. Figure 12As shown, a system 1 for detecting an object to be detected according to an embodiment of this disclosure includes: an image acquisition device 20 for acquiring data containing an image of the object to be detected; and an object detection device 40. The image acquisition device 20 may include sensor components, such as a camera. The object detection device 40 may be the device 10 described above, or alternatively, a processor, etc.

[0112] According to an embodiment of the present disclosure, a computer-readable storage medium stores a computer program including program instructions that, when executed by a processor, perform the above-described method.

[0113] According to an embodiment of the present disclosure, a processor is used to run a program, wherein the program executes the above-described method during runtime.

[0114] A computer program product according to an embodiment of the present disclosure includes a computer program stored on a non-transitory computer-readable storage medium, the computer program including program instructions that, when executed by a computer, cause the computer to perform the above-described method.

[0115] Figure 13 This is a block diagram of a system for detecting an object to be detected according to another embodiment of the present disclosure. Figure 13 As shown, system 800 may include one or more of the following components: processor component 802, memory 804, power supply component 806, multimedia component 808, audio component 810, input / output (I / O) interface 812, sensor component 814, and communication component 816.

[0116] Processor component 802 typically controls the overall operation of system 800, such as operations associated with display, telephone calls, data communication, camera operation, and recording operations. Processor component 802 may include one or more processors 820 to execute instructions to perform all or part of the steps of the methods described above. Furthermore, processor component 802 may include one or more modules to facilitate interaction between processor component 802 and other components. For example, processor component 802 may include a multimedia module to facilitate interaction between multimedia component 808 and processor component 802.

[0117] Memory 804 is configured to store various types of data to support the operation of system 800. Examples of this data include instructions for any application or method operating on system 800, contact data, phonebook data, messages, images, videos, etc. Memory 804 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0118] Power supply component 806 provides power to various components of system 800. Power supply component 806 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to system 800.

[0119] Multimedia component 808 includes a screen that provides an output interface between system 800 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touchscreen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors may sense not only the boundaries of touch or swipe actions but also the duration and pressure associated with the touch or swipe operation.

[0120] Audio component 810 is configured to output and / or input audio signals. For example, audio component 810 includes a microphone (MIC) configured to receive external audio signals when system 800 is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 804 or transmitted via communication component 816. In some embodiments, audio component 810 also includes a speaker for outputting audio signals.

[0121] I / O interface 812 provides an interface between processor component 802 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to, home buttons, volume buttons, power buttons, and lock buttons.

[0122] Sensor assembly 814 includes one or more sensors for providing status assessment and data acquisition for various aspects of system 800. For example, sensor assembly 814 can detect the on / off state of system 800, the relative positioning of components such as the display and keypad of system 800, changes in the position of system 800 or a component of system 800, the presence or absence of user contact with system 800, the orientation or acceleration / deceleration of system 800, and temperature changes of system 800. Sensor assembly 814 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 814 may also include a camera assembly, which may employ an image sensor such as a CMOS or CCD for use in imaging applications. In some embodiments, sensor assembly 814 may also include an accelerometer, a gyroscope, a magnetometer, a pressure sensor, or a temperature sensor.

[0123] Communication component 816 is configured to facilitate wired or wireless communication between system 800 and other devices. System 800 can access wireless networks based on communication standards, such as WiFi, 2G, or 3G, or combinations thereof. In one exemplary embodiment, communication component 816 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, communication component 816 also includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0124] In an exemplary embodiment, system 800 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the methods described above.

[0125] In the above embodiments of this disclosure, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0126] The functional units or components in the embodiments of this disclosure can be integrated into a processing module, or each unit can exist as a separate physical entity, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. The storage medium mentioned above can be a read-only memory, a disk, or an optical disk, etc. The device embodiments described above are merely illustrative. For example, the division of units or modules is only a logical functional division; in actual implementation, there may be other division methods. For example, multiple units, modules, or components can be combined or integrated into another system, or some features can be ignored or not executed. Furthermore, the displayed or discussed mutual coupling or direct coupling or communication connection can be through some interfaces; the indirect coupling or communication connection of modules or units can be electrical or other forms.

[0127] The units or modules described as separate components may or may not be physically separate. The components shown as units or modules may or may not be physical units or modules; that is, they may be located in one place or distributed across multiple network units or modules. Some or all of the units or modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0128] Furthermore, the functional units or modules in the various embodiments of this disclosure can be integrated into one processing unit or module, or each unit or module can exist physically separately, or two or more units or modules can be integrated into one unit or module. The integrated units or modules described above can be implemented in hardware or in the form of software functional units or modules.

[0129] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this disclosure, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this disclosure. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard disk, magnetic disk, or optical disk.

[0130] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware (e.g., circuits) related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media that can store program code, such as ROM, RAM, disk, or optical disk.

[0131] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the parts that contribute to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., including several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods of various embodiments or some parts of embodiments.

[0132] This disclosure can also be configured as follows:

[0133] (1) A method for detecting an object to be detected, comprising:

[0134] Acquire data containing the image of the object to be detected;

[0135] Analyze image data to determine the location information and tilt angle of the object to be detected;

[0136] The position information and tilt angle used to generate the prediction box are determined based on the position information and tilt angle of the object to be detected.

[0137] Generate a predicted bounding box that matches the object to be detected based on the position information and tilt angle used to generate the predicted bounding box; and

[0138] The predicted bounding box is used to detect the object to be detected.

[0139] (2) According to the method of (1), wherein generating a prediction box that matches the object to be detected based on the position information and tilt angle used to generate the prediction box includes:

[0140] Using a pre-defined single-shot multi-frame detector (SSD) neural network, multiple prediction boxes are generated based on the position information and tilt angle used to generate the prediction boxes;

[0141] Obtain the preset calibration box of the object to be detected; and

[0142] Identify the prediction box that matches the calibration box among multiple prediction boxes.

[0143] (3) According to the method in (2), determining the prediction box that matches the calibration box among multiple prediction boxes includes:

[0144] Determine if the tilt angle of the calibration frame is 0°;

[0145] When the tilt angle of the calibration box is 0°, determine the prediction box that matches the calibration box among the non-tilted prediction boxes among multiple prediction boxes; or

[0146] When the tilt angle of the calibration box is not 0°, the prediction box that matches the calibration box is determined among the tilted prediction boxes among multiple prediction boxes.

[0147] (4) According to the method of (2) or (3), wherein determining the prediction box that matches the calibration box among multiple prediction boxes includes:

[0148] Obtain at least one of the following: the size information, position information, tilt angle, and object category information within the calibration frame; and

[0149] The prediction box is adjusted based on at least one of the calibration box's size, position, tilt angle, and object category information to obtain a prediction box that matches the calibration box.

[0150] (5) According to the method of (4), wherein adjusting the prediction box based on at least one of the size information, position information, tilt angle and object category information of the calibration box to obtain a prediction box that matches the calibration box includes:

[0151] Obtain the overlap area between each prediction box and the calibration box in the multiple prediction boxes;

[0152] Among multiple prediction frames, the prediction frame corresponding to the largest overlap area is determined as the largest prediction frame; and

[0153] The maximum prediction box is adjusted based on at least one of the calibration box's size, position, tilt angle, and object category information to obtain a prediction box that matches the calibration box.

[0154] (6) An apparatus for detecting an object to be detected, comprising:

[0155] The image acquisition module is configured to acquire data containing an image of the object to be detected.

[0156] The image analysis module is configured to analyze image data to determine the position information and tilt angle of the object to be detected;

[0157] The prediction box information determination module is configured to determine the position information and tilt angle used to generate the prediction box based on the position information and tilt angle of the object to be detected.

[0158] The prediction bounding box generation module is configured to generate a prediction bounding box that matches the object to be detected, based on the position information and tilt angle used to generate the prediction bounding box; and

[0159] The detection module is configured to use predictive bounding boxes to detect the objects to be detected.

[0160] (7) The apparatus according to (6), characterized in that the prediction box generation module is further configured to:

[0161] Using a pre-defined single-shot multi-frame detector (SSD) neural network, multiple prediction boxes are generated based on the position information and tilt angle used to generate the prediction boxes;

[0162] Obtain the preset calibration box of the object to be detected; and

[0163] Identify the prediction box that matches the calibration box among multiple prediction boxes.

[0164] (8) The apparatus according to (7), characterized in that the prediction box generation module is further configured to:

[0165] Determine if the tilt angle of the calibration frame is 0°;

[0166] When the tilt angle of the calibration box is 0°, determine the prediction box that matches the calibration box among the non-tilted prediction boxes among multiple prediction boxes; or

[0167] When the tilt angle of the calibration box is not 0°, the prediction box that matches the calibration box is determined among the tilted prediction boxes among multiple prediction boxes.

[0168] (9) The apparatus according to (7) or (8), characterized in that the prediction box generation module is further configured to:

[0169] Obtain at least one of the following: the size information, position information, tilt angle, and object category information within the calibration frame; and

[0170] The prediction box is adjusted based on at least one of the calibration box's size, position, tilt angle, and object category information to obtain a prediction box that matches the calibration box.

[0171] (10) The apparatus according to claim (9), wherein the prediction box generation module is further configured to:

[0172] Obtain the overlap area between each prediction box and the calibration box in the multiple prediction boxes;

[0173] Among multiple prediction frames, the prediction frame corresponding to the largest overlap area is determined as the largest prediction frame; and

[0174] The maximum prediction box is adjusted based on at least one of the calibration box's size, position, tilt angle, and object category information to obtain a prediction box that matches the calibration box.

[0175] (11) An apparatus for detecting an object to be detected, comprising: a memory and a processor, the processor being connected to the memory, the memory storing program instructions to be executed by the processor, the processor being able to execute any one of (1) to (5) when executing the program instructions.

[0176] (12) A system for detecting an object to be detected, comprising: an image acquisition device for acquiring data containing an image of the object to be detected; and an apparatus for detecting the object to be detected according to any one of (6) to (11).

[0177] (13) A computer-readable storage medium, wherein a computer program including program instructions is stored on the computer-readable storage medium, the program instructions performing any one of (1) to (5) when executed by a processor.

[0178] (14) A processor, wherein the processor is used to run a program, wherein the program performs any one of (1) to (5) during runtime.

[0179] (15) A computer program product, wherein the computer program product includes a computer program stored on a non-transitory computer-readable storage medium, the computer program including program instructions that, when executed by a computer, cause the computer to perform any one of (1) to (5).

[0180] The above are merely preferred embodiments of this disclosure. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of this disclosure, and these improvements and modifications should also be considered within the scope of protection of this disclosure.

Claims

1. A method for detecting multiple objects to be inspected, wherein the objects to be inspected are irregular damage types on aircraft skin, characterized in that, include: Acquire data containing images of the plurality of objects to be detected; Analyze the data in the image to determine the position information and tilt angle of each of the plurality of objects to be detected; The position information and tilt angle for generating the prediction box are determined based on the position information and tilt angle of each of the plurality of objects to be detected. The prediction box is generated based on the position information and tilt angle used to generate the prediction box, and the prediction box is matched with each of the plurality of objects to be detected. as well as The predicted bounding box is used to detect the object to be detected. Generating a prediction box that matches each of the plurality of objects to be detected, based on the position information and tilt angle used to generate the prediction box, includes: Using a pre-defined single-shot multi-frame detector (SSD) neural network, multiple prediction frames are generated based on the position information and tilt angle used to generate the prediction frames; Obtain a preset calibration box for each of the plurality of objects to be detected; Obtain the overlap area between each of the multiple prediction boxes and any one of the calibration boxes; Among the multiple prediction frames, the prediction frame corresponding to the largest overlap area is determined as the largest prediction frame; Obtain at least one of the following: size information, position information, tilt angle, and object category information within each calibration frame; and The maximum prediction box is adjusted based on at least one of the size information, position information, tilt angle, and object category information of each calibration box to obtain a prediction box that matches each calibration box.

2. The method according to claim 1, characterized in that, Determining the prediction box that matches each of the calibration boxes among the plurality of prediction boxes includes: Determine whether the tilt angle of each of the calibration frames is 0°; When the tilt angle of each calibration box is 0°, a prediction box matching each calibration box is determined among the non-tilted prediction boxes in the plurality of prediction boxes; or When the tilt angle of the calibration box is not 0°, the prediction box that matches each of the calibration boxes is determined among the tilted prediction boxes of the plurality of prediction boxes.

3. An apparatus for detecting multiple objects to be inspected, wherein the objects to be inspected are irregular types of damage on aircraft skin, characterized in that, include: The image acquisition module is configured to acquire data containing images of the plurality of objects to be detected; The image analysis module is configured to analyze data from the image to determine the position information and tilt angle of each of the plurality of objects to be detected; The prediction box information determination module is configured to determine the position information and tilt angle used to generate the prediction box based on the position information and tilt angle of the object to be detected. The prediction box generation module is configured to generate a prediction box that matches each of the plurality of objects to be detected based on the position information and tilt angle used to generate the prediction box; as well as The detection module is configured to detect the object to be detected using the prediction bounding box. The prediction box generation module is further configured as follows: Using a pre-defined single-shot multi-frame detector (SSD) neural network, multiple prediction frames are generated based on the position information and tilt angle used to generate the prediction frames; Obtain a preset calibration box for each of the plurality of objects to be detected; Obtain the overlap area between each of the multiple prediction boxes and any one of the calibration boxes; Among the multiple prediction frames, the prediction frame corresponding to the largest overlap area is determined as the largest prediction frame; Obtain at least one of the following: size information, position information, tilt angle, and object category information within each calibration frame; and The maximum prediction box is adjusted based on at least one of the size information, position information, tilt angle, and object category information of each calibration box to obtain a prediction box that matches each calibration box.

4. The apparatus according to claim 3, characterized in that, The prediction box generation module is further configured to: Determine whether the tilt angle of each of the calibration frames is 0°; When the tilt angle of each calibration box is 0°, a prediction box matching each calibration box is determined among the non-tilted prediction boxes in the plurality of prediction boxes; or When the tilt angle of the calibration box is not 0°, the prediction box that matches each of the calibration boxes is determined among the tilted prediction boxes of the plurality of prediction boxes.

5. An apparatus for detecting an object to be detected, characterized in that, include: A memory and a processor, the processor being connected to the memory, the memory storing program instructions that the processor executes, the processor being capable of performing the method according to any one of claims 1 to 2 when executing the program instructions.

6. A system for detecting an object to be detected, characterized in that, include: Image acquisition device, used to acquire data containing an image of the object to be detected; And an apparatus for detecting a plurality of objects to be detected according to any one of claims 3 to 4.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program including program instructions that, when executed by a processor, perform the method according to any one of claims 1 to 2.

Images