Object Detection Method, Device, Electronic Device, and Storage Medium

Abstract

The present application discloses an object detection method, apparatus, electronic device, and storage medium. The object detection method includes: obtaining a first image, a standard image, and a detection configuration file; based on the first image and the standard image, obtaining a first feature parameter of the first image, a second feature parameter of the standard image, and a matching relationship between the first feature parameter and the second feature parameter; based on the detection configuration file, the first feature parameter, the second feature parameter, and the matching relationship, obtaining a second image of the object to be detected in the first image; and obtaining a detection result based on the second image. According to the detection configuration file, the first feature parameter of the first image, and the second feature parameter of the standard image, the present application realizes the matching of the first image and the standard image, and can realize the detection of different first images by adjusting the corresponding standard image and the detection configuration file.

Description

Technical Field

[0001] The present application relates to the field of visual detection, and particularly to a target detection method, apparatus, electronic device, and storage medium. Background Art

[0002] In the field of industrial visual quality inspection, the main application scenarios include target missing and leakage detection and product surface defect detection. Among them, screw missing detection belongs to the category of target missing and leakage detection. In the screw missing detection scenario, the scale change range of different products is large, and the proportion of screws on the commodity is very small.

[0003] For the solutions based on traditional image processing techniques, a single camera is commonly used to take multiple photos, or multiple cameras are used to take photos at multiple positions, and then traditional image processing techniques are used to perform different processing on different sub-images to detect the target screw. This requires high requirements for the detection environment and imaging quality. At the same time, there needs to be a high contrast between the defect area to be detected and the non-defect area, and the robustness of the solutions based on traditional image processing techniques is poor.

[0004] For the solutions based on deep learning computer vision, it is necessary to train a deep learning model through a large amount of labeled data to achieve end-to-end detection of the target screw, resulting in low detection efficiency. At the same time, it has high requirements for the performance of the computing device. Summary of the Invention

[0005] The present application provides at least one target detection method, apparatus, electronic device, and storage medium.

[0006] In the first aspect of the present application, a target detection method is provided. The target detection method includes:

[0007] Obtain a first image, a standard image, and a detection configuration file;

[0008] Based on the first image and the standard image, obtain the first feature parameter of the first image, the second feature parameter of the standard image, and the matching relationship between the first feature parameter and the second feature parameter;

[0009] Based on the detection configuration file, the first feature parameter, the second feature parameter, and the matching relationship, obtain a second image of the target to be detected in the first image;

[0010] Based on the second image, obtain a detection result.

[0011] In the second aspect of the present application, a target detection apparatus is provided. The target detection apparatus includes:

[0012] An acquisition module, configured to acquire a first image, a standard image, and a detection configuration file;

[0013] A calculation module, configured to obtain a first feature parameter of the first image, a second feature parameter of the standard image, and a matching relationship between the first feature parameter and the second feature parameter based on the first image and the standard image;

[0014] The calculation module is further configured to obtain a second image of the target to be detected in the first image based on the detection configuration file, the first feature parameter, the second feature parameter, and the matching relationship;

[0015] The calculation module is further configured to obtain a detection result based on the second image.

[0016] A third aspect of the present application provides an electronic device, including a memory and a processor coupled to each other, and the processor is configured to execute program instructions stored in the memory to implement the target detection method in the first aspect above.

[0017] A fourth aspect of the present application provides a computer-readable storage medium, on which program instructions are stored, and when the program instructions are executed by a processor, the target detection method in the first aspect above is implemented.

[0018] The beneficial effects of the present application are as follows: Different from the prior art, the present application realizes the matching between the first image and the standard image according to the detection configuration file, the first feature parameter of the first image, and the second feature parameter of the standard image, and realizes the detection of different first images by adjusting the corresponding standard image and the detection configuration file. At the same time, the second image of the target to be detected is obtained through the target detection method, and the second image of the target to be detected is further detected to realize the effective detection of the target to be detected on the surfaces of different products.

[0019] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and do not limit the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0021] Figure 1 is a schematic flowchart of an embodiment of the target detection method of the present application;

[0022] Figure 2 is Figure 1 a specific flowchart of step S11 in

[0023] Figure 3 is Figure 2 a specific flowchart of establishing the relationship between the standard image and the detection configuration file before step S114 in

[0024] Figure 4 is Figure 1 The specific process schematic diagram of step S12 in

[0025] Figure 5 is Figure 4 The specific process schematic diagram of step S125 in

[0026] Figure 6 is Figure 1 The specific process schematic diagram of step S13 in

[0027] Figure 7 is Figure 1 The specific process schematic diagram of step S14 in

[0028] Figure 8 It is the framework schematic diagram of an embodiment of the target detection device of the present application;

[0029] Figure 9 It is the framework schematic diagram of an embodiment of the electronic device of the present application;

[0030] Figure 10 It is the framework schematic diagram of an embodiment of the computer-readable storage medium of the present application. Specific Embodiments

[0031] To enable those skilled in the art to better understand the technical solutions of the present application, the target detection method, device, electronic device, and storage medium provided by the present application will be further described in detail below in conjunction with the accompanying drawings and specific embodiments. It can be understood that the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present application without creative efforts shall fall within the protection scope of the present application.

[0032] The terms "first", "second", etc. in the present application are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the listed steps or units, but optionally further includes steps or units not listed, or optionally further includes other steps or units inherent to these processes, methods, products, or devices.

[0033] Please refer to Figure 1 , Figure 1 It is the process schematic diagram of an embodiment of the target detection method of the present application.

[0034] The execution subject of the object detection method of this application can be an object detection device. For example, the object detection method can be executed by a terminal device, a server, or other processing devices. Among them, the object detection device can be a user equipment (UE), a mobile device, a user terminal, a terminal, a personal digital assistant (PDA), a handheld device, a computing device, a vehicle-mounted device, a wearable device, etc. In some possible implementation manners, the object detection method can be implemented by a processor invoking computer-readable instructions stored in a memory.

[0035] Specifically, the object detection method of this embodiment may include the following steps:

[0036] Step S11: Obtain a first image, a standard image, and a detection configuration file.

[0037] Among them, the object detection method of this embodiment is specifically applied to the screw missing detection scenario, and is used to detect the missing or damaged screws on the product surface.

[0038] Specifically, the first image obtained in step S11 is a detection image obtained by photographing the product to be tested, the standard image is an image pre-set according to the product to be tested, and the detection configuration file is a file storing information of all items to be detected of the product to be tested, that is, storing the position of each screw to be tested.

[0039] Among them, for the specific process of obtaining the first image, the standard image, and the detection configuration file, please continue to refer to Figure 2 , Figure 2 is Figure 1 the specific process schematic diagram of step S11 in

[0040] Step S111: Obtain the one-dimensional code of the product to be tested.

[0041] Among them, a one-dimensional code including the product information of the product to be tested, that is, a bar code, is correspondingly set on each product to be tested. A bar code is a mark composed of a group of regularly arranged bars, spaces, and corresponding characters. "Bar" refers to the part with a lower light reflectivity, and "space" refers to the part with a higher light reflectivity. The data composed of these bars and spaces express certain information and can be read by a specific device and converted into binary and decimal information compatible with a computer. Usually, for each product, its encoding is unique.

[0042] Step S112: Obtain the product information of the product to be tested based on the one-dimensional code.

[0043] Among them, in this embodiment, the commodity information corresponding to the product to be tested can be obtained by scanning the barcode. The commodity information may include the product category and the product model. By querying the commodity information, it can be obtained that the product to be tested is a certain model of a certain category of product. For example, the product model naming of electric water heaters basically starts with ES. Among them, E represents "electrical", that is, powered by electricity, and S represents "storage", that is, water storage. The two digits following ES represent the capacity. For example, 80 represents 80L. The letter following the number is either H or V, where H represents "horizontal", that is, a horizontal product.

[0044] Specifically, the commodity information is stored in the production information management system. In this embodiment, the MES system (Manufacturing Execution System) is specifically used. The MES system is a set of production information management systems for the workshop execution layer of manufacturing enterprises.

[0045] When the one-dimensional code scanning fails, "scanning failed" is displayed on the client and the relevant information is saved. Specifically, it can be displayed on the display screen of the client, and further, a long-bright indicating light prompts the scanning failure. Specifically, a long-bright yellow indicating light is used to prompt the production line personnel to handle it. Specifically, it may include adjusting the placement position of the product to be tested or replacing the scanning device, etc. At the same time, the next round of target detection is carried out, and the detection object can be the current product to be tested or the next product to be tested.

[0046] When the one-dimensional code scanning is successful, it is necessary to further request the MES system to obtain the commodity information of the product to be tested. When the request for the MES system fails, "request failed" is displayed on the client and the relevant information is saved. Specifically, it can be displayed on the display screen of the client, and further, a long-bright indicating light prompts the request failure. Specifically, a long-bright yellow indicating light is used to prompt the production line personnel to handle it. At the same time, the next round of target detection is carried out, and the detection object can be the current product to be tested or the next product to be tested.

[0047] When the request for the MES system is successful, the commodity information of the product to be tested is successfully obtained, and steps S113 and S114 are executed. Optionally, the commodity information is specifically the product model of the product to be tested.

[0048] Step S113: In response to successfully obtaining the commodity information, obtain the first image.

[0049] Among them, in this embodiment, the first image can be obtained by taking a picture of the product to be tested with a camera on the industrial production line.

[0050] Step S114: Based on the commodity information, obtain the standard image and the detection configuration file.

[0051] Among them, in this embodiment, the corresponding standard image and detection configuration file stored in the local database are retrieved according to the commodity information, where the standard image and the detection configuration file correspond to the product to be measured of the first image.

[0052] Optionally, steps S113 and S114 may be executed simultaneously or successively, and this embodiment does not limit the execution order of steps S113 and S114.

[0053] Among them, before obtaining the standard image and the detection configuration file, it is necessary to establish the relationship between the standard image and the detection configuration file. Please continue to refer to Figure 3 , Figure 3 Yes Figure 2 is the schematic flow chart of the specific process of establishing the relationship between the standard image and the detection configuration file before step S114 in

[0054] Step S21: Establish the standard image and the detection configuration file.

[0055] Among them, the standard image can be an image generated by modeling according to the product to be measured, and the modeling standard is to include all areas of the product to be measured and as little background as possible. The specific steps of modeling can be: select a standard image of the product to be measured, such as the front view of the product to be measured; blacken the irrelevant background, and the irrelevant background is specifically the part of the standard image that cannot display the surface of the product to be measured; obtain the standard image.

[0056] Furthermore, the detection configuration file is a file storing information on all items to be detected of the product to be measured, and is specifically obtained by detecting the standard image, that is, the detection configuration file includes the position information of all standard targets in the standard image, where the standard target is a screw.

[0057] Furthermore, the standard image and the detection configuration file are associated according to the model of the product to be measured corresponding to the standard image and stored in the local database. Among them, the local database includes the standard images and detection configuration files corresponding to products of multiple models.

[0058] Step S22: Modify the configurable items in the detection configuration file according to the preset image transformation type.

[0059] Among them, the detection configuration file includes configurable items, and the configurable items are related to the image transformation type. Specifically, in this embodiment, the detection configuration file can be a xml file, and the configurable item can be transType (transformation symbol). The transType in the xml file can be adjusted according to the image transformation involved in the detection task scenario. Specifically, the image transformation types include rigid transformation (rigid) and perspective transformation (homo), and the preset image transformation type is the image transformation involved in detecting a specific product to be measured.

[0060] In steps S111 - S114, the first image, standard image, and detection configuration file corresponding to the product to be tested are obtained, and the first image, standard image, and detection configuration file are input into the detection algorithm node to perform screw missing detection. Specifically, the detection algorithm process is as shown in steps S12 - S14.

[0061] Step S12: Based on the first image and the standard image, obtain the first feature parameters of the first image, the second feature parameters of the standard image, and the matching relationship between the first feature parameters and the second feature parameters.

[0062] Among them, the detection algorithm used in this embodiment is a screw detection algorithm based on depth - adaptive registration technology. Through this screw detection algorithm, image feature extraction (FeatureExtract) and feature matching (FeatureMatch) are performed on the first image and the standard image.

[0063] Among them, for the specific feature extraction and matching process, please continue to refer to Figure 4 , Figure 4 which Figure 1 is the specific process schematic diagram of step S12 in

[0064] Step S121: Obtain the first feature parameters of the first image.

[0065] Among them, the first feature parameters include the first key point O kpts-1 and the first sub - feature parameters. Specifically, the first sub - feature parameters are the first feature descriptor O desc-1 .

[0066] Specifically, in this embodiment, a VGG - style shared convolutional neural network encoder E is used. The first image I1 is input into the shared convolutional neural network encoder E to convert the first image I1 ∈ R1 H×W into an intermediate feature map where H c = H / 8, W c = W / 8, F > 1. The shared convolutional neural network encoder E decodes the intermediate feature map B1 through the key point decoder D kpts branch and the feature descriptor decoder D desc branch.

[0067] In the key point decoder D kpts branch of the shared convolutional neural network encoder E, first, the intermediate feature map B1 is decoded into Among them, 65 channels respectively correspond to non-overlapping local 8×8 pixel grids in the first image I1 plus a mark indicating the presence or absence of key points. Then, successively perform Softmax (logistic regression) operations at the channel level, an operation to remove the last 1 channel (Reduce, aggregation operation), and a pixel rearrangement operation (Shuffle) to obtain the scores of each point in the first image I1 as key points. The size of the feature map of this branch changes to Finally, through post-processing operations such as non-maximum suppression (NMS algorithm, Non-Maximum Suppression), filtering by the key point score threshold (keypoint-threshold), and removing boundary points (remove_border), the first key point O of the first image I1 is obtained kpts-1 .

[0068] The shared convolutional neural network encoder E is in the feature descriptor decoder D desc branch. First, decode the intermediate feature map B1 into where D is the dimension of the feature descriptor, and then obtain the feature descriptors corresponding to each point in the first image I1 through bicubic interpolation and L2 regularization (L2_normalization). The size of the feature map of this branch changes to Select the feature descriptors corresponding to the key points obtained through the key point branch to obtain the first feature descriptor O of the first image I1 desc-1 .

[0069] Step S122: Obtain the first matching parameter of the first image based on the first key point and the first sub-feature parameter.

[0070] Specifically, the first matching parameter of the first image I1 is the first matching descriptor f1.

[0071] Among them, in this embodiment, a multi-layer perceptron (MLP, Multilayer Perceptron) is used for feature point encoding to encode the first key point O kpts-1 and the first feature descriptor O desc-1 together. Then, through a multi-layer graph neural network (Graph Neural Networks, GNN), alternating aggregation of the self-attention mechanism and the cross-attention mechanism is combined to obtain the first matching descriptor f1.

[0072] Step S123: Obtain the second feature parameter of the standard image.

[0073] Among them, the second feature parameter includes the second key point O kpts-2With the second sub - feature parameter, specifically, the second sub - feature parameter is the second feature descriptor O desc-2 .

[0074] Specifically, in this embodiment, a VGG - style shared convolutional neural network encoder E is used. The standard image I2 is input into the shared convolutional neural network encoder E to convert the standard image I2 ∈ R1 H×W into an intermediate feature map where H c = H / 8, W c = W / 8, F > 1. The shared convolutional neural network encoder E decodes the intermediate feature map B2 through the key - point decoder D kpts branch and the feature - descriptor decoder D desc branch.

[0075] In the key - point decoder D kpts branch of the shared convolutional neural network encoder E, first, the intermediate feature map B2 is decoded into through a convolutional operation Conv, where 65 channels respectively correspond to non - overlapping 8×8 pixel grids in the standard image I2 plus a marker for the presence or absence of key - points. Then, a Softmax operation at the channel level, an operation to remove the last 1 channel, and a pixel rearrangement operation are sequentially performed to obtain the scores of each point in the standard image I2 as key - points. The size of the feature map in this branch changes as Finally, through post - processing operations such as non - maximum suppression, key - point score threshold filtering, and boundary - point removal, the second key - point O of the standard image I2 is obtained kpts-2 .

[0076] In the feature - descriptor decoder D desc branch of the shared convolutional neural network encoder E, first, the intermediate feature map B2 is decoded into through a convolutional operation Conv, where D is the dimension of the feature descriptor. Then, through bicubic interpolation and L2 regularization, the feature descriptor corresponding to each point in the standard image I2 is obtained. The size of the feature map in this branch changes as The feature descriptors corresponding to the key - points obtained through the key - point branch are selected to obtain the second feature descriptor O of the standard image I2 desc-2 .

[0077] Step S124: Based on the second key - point and the second sub - feature parameter, obtain the second matching parameter of the standard image

[0078] Specifically, the second matching parameter of the standard image I2 is the second matching descriptor f2

[0079] Among them, in this embodiment, a multi - layer perceptron is used for feature - point encoding, and the second key - point O kpts-2and the second feature descriptor O desc-2 They are encoded together. Then, through a multi-layer graph neural network, the self-attention mechanism and the cross-attention mechanism are alternately aggregated to obtain the second matching descriptor f2.

[0080] Optionally, step S121 and step S123 can be executed simultaneously or successively. This embodiment does not limit the execution order of step S121 and step S123.

[0081] Step S125: Based on the first matching parameter and the second matching parameter, obtain the matching relationship between the first key point and the second key point.

[0082] Among them, the shared convolutional neural network encoder E performs an inner product on the first matching descriptor f1 of the first image I1 and the second matching descriptor f2 of the standard image I2 to obtain a matching score S ∈ R M×N , where M and N are the numbers of key points extracted from the first image I1 and the standard image I2 respectively. Finally, the overall score is maximized through the sinkhorm algorithm, and then filtered using the matching score threshold to obtain the matching relationship between the finally extracted key points, that is, the first key point O kpts-1 and the second key point O kpts-2 between them. Optionally, the matching score S is specifically a confidence score. In this embodiment, the matching score threshold can be 0.5.

[0083] After obtaining the matching relationship between the first key point and the second key point according to step S125, it is necessary to perform anomaly detection on the matching relationship. This stage is the first-stage anomaly detection. For the specific anomaly detection process, please continue to refer to Figure 5 , Figure 5 which is Figure 4 the specific flow diagram of step S125. Specifically, it includes the following steps:

[0084] Step S1251: According to the first matching parameter and the second matching parameter, obtain the i-th point in the first key point and the j-th point in the second key point.

[0085] Among them, according to step S121 and step S123, M first key points O kpts-1 , and N second key points O kpts-2 can be obtained respectively. Further, according to the first matching parameter and the second matching parameter, obtain the i-th point in the first key point O kpts-1 and the j-th point in the second key point O kpts-2 .

[0086] Specifically, the i-th point in the first key point O kpts-1 and the j-th point in the second key point O kpts-2The object described by the j-th point in is the same, that is, the i-th point matches the j-th point, where i is an integer less than or equal to M, and j is an integer less than or equal to N.

[0087] Step S1252: Based on the i-th point and the j-th point, obtain the number of pairs of matching points between the first key point and the second key point. Among them, one i-th point and one j-th point form a pair of matching points. The first key point O kpts-1 There are multiple i-th points, and the second key point O kpts-2 There are multiple j-th points, that is, the first key point O kpts-1 and the second key point O kpts-2 There are multiple pairs of matching points, and the number of pairs of matching points is less than or equal to the minimum value of M and N.

[0088] Specifically, the matching relationship obtained in step S125 is the multiple pairs of matching points between the first key point O kpts-1 and the second key point O kpts-2 Step S1253: In response to the number of pairs of matching points being greater than or equal to the number of key points required for image transformation, output the matching relationship.

[0089] Among them, in order to achieve image change, a certain amount of key points are required for estimation. When it is determined that the number of pairs of matching points is greater than or equal to the number of key points required for image transformation, it is determined that there is no abnormal situation in the detection process, and the matching relationship obtained according to step S125 is output to execute step S13.

[0090] Step S1254: In response to the number of pairs of matching points being less than the number of key points required for image transformation, end the target detection.

[0091] Among them, when it is determined that the number of pairs of matching points is greater than or equal to the number of key points required for image transformation, it is determined that there is an abnormal situation in the detection process, and the current target detection is ended. Optionally, the abnormal situation may include that the product to be tested is blocked or the position of the product to be tested exceeds the detection area, etc.

[0092] When an abnormal situation is detected, display a detection abnormality on the client and save the relevant information. Specifically, it can be displayed on the display screen of the client, and further, a long-bright indicator light is used to prompt the detection abnormality. Specifically, a long-bright yellow indicator light is used to prompt the production line personnel to handle it. At the same time, a new round of target detection is carried out, and the detection object can be the current product to be tested or the next product to be tested.

[0093] Step S13: Based on the detection configuration file, the first feature parameter, the second feature parameter, and the matching relationship, obtain the second image of the target to be detected in the first image.

[0094]

[0095] ​Among them, based on the detection configuration file obtained in step S11, the first feature parameter, the second feature parameter, and the matching relationship obtained in step S12, image transformation is performed on the first image and the standard image, and the second image of the target to be measured in the first image can be obtained.

[0096] Among them, for the specific image transformation process, please continue to refer to Figure 6 , Figure 6 which Figure 1 is the specific process schematic diagram of step S13 in

[0097] Step S131: Based on the preset image transformation type, the first key points, the second key points, and the matching relationship, an inverse transformation matrix is obtained.

[0098] Among them, in this embodiment, according to the preset image transformation type included in the detection configuration file, and through the first key points, the second key points, and the matching relationship, the inverse transformation matrix (invertH) is calculated.

[0099] Step S132: Based on the inverse transformation matrix, the first mapping coordinates of the standard target in the standard image in the first image are obtained.

[0100] Step S133: Based on the first mapping coordinates, the area to be measured in the first image is obtained.

[0101] Among them, in this embodiment, according to the inverse transformation matrix, the position of the standard target in the standard image, that is, the position of the screw, is mapped onto the first image, and the first mapping coordinates of the screw in the standard image in the first image are obtained, which are the positions of the screws to be measured in the first image, and the area where the screws to be measured are located in the first image is the area to be measured in the first image.

[0102] In this embodiment, it is further determined that the center coordinates of the target to be measured in the area to be measured do not exceed the boundary of the first image. In response to the center coordinates of the target to be measured in the area to be measured exceeding the boundary of the first image, the corresponding one is deleted; in response to the center coordinates of the target to be measured in the area to be measured not exceeding the boundary of the first image, step S134 is executed.

[0103] Step S134: In response to the center coordinates of the target to be measured in the area to be measured not exceeding the boundary of the first image, the first mapping coordinates are corrected to obtain the second mapping coordinates.

[0104] Among them, the second mapping coordinates are the position coordinates of the target to be measured in the first image, that is, the position coordinates of the screw to be measured.

[0105] When the center coordinates of the target to be measured in the area to be measured do not exceed the boundary of the first image, that is, the center coordinates of the screw to be measured do not exceed the boundary of the first image, it is necessary to further correct the coordinates of the screw to be measured to ensure that the bounding box of the screw to be measured is within the boundary of the first image. The specific correction formula is as follows:

[0106] x lefttop = min(max(x lefttop , 0), width - b - 1) (1)

[0107] y lefttop = min(max(y lefttop , 0), height - b - 1) (2)

[0108] x rightbottom = x lefttop + b (3)

[0109] y rightbottom = y lefttop + b (4)

[0110] Among them, width in formula (1) and height in formula (2) are the width and height of the first image respectively, b is the side length of each detection box mapped back to the first image. Specifically, the detection box is a square box. x lefttop , y lefttop , x rightbottom and y rightbottom constitute the bounding box of the screw to be measured under the second mapping coordinates.

[0111] Optionally, after performing step S135, it is necessary to further perform anomaly detection in the second stage to determine whether the number of remaining screws to be measured is equal to the number of screws to be detected in the configuration item.

[0112] When the number of remaining screws to be measured is not equal to the number of screws to be detected in the configuration item, it is determined that there is an abnormal situation in the detection process, and the current object detection is ended. Display the detection anomaly on the client and save the relevant information. Specifically, it can be displayed on the display screen of the client, and further turn on the indicating light continuously to prompt the detection anomaly. Specifically, turn on the yellow indicating light continuously to prompt the production line personnel to handle it. At the same time, perform the next round of object detection, and the detection object can be the current product to be measured or the next product to be measured.

[0113] When the number of remaining screws to be measured is equal to the number of screws to be detected in the configuration item, it is determined that there is no abnormal situation in the detection process, and according to the second mapping coordinates obtained in step S134, perform step S135.

[0114] Step S135: Obtain the second image based on the second mapping coordinates.

[0115] Among them, the bounding boxes of the screws to be measured under the corrected second mapping coordinates are all within the boundaries of the first image, and the second image is the image of the screws to be measured captured according to the bounding boxes.

[0116] Step S14: Obtain a detection result based on the second image.

[0117] Wherein, the number of screws to be measured in the first image is at least one, then the number of second images is at least one, and one second image includes one object to be measured, that is, includes one screw to be measured.

[0118] In this embodiment, classification score detection is performed on the second image. For the specific detection process, please continue to refer to Figure 7 , Figure 7 which Figure 1 is the schematic flow chart of step S14 in

[0119] Step S141: Classify at least one second image.

[0120] Step S142: Evaluate at least one category of second images to obtain detection scores of at least one object to be measured.

[0121] Wherein, in this embodiment, a screw classification model is used to input at least one second image into the screw classification model. The screw classification model classifies and evaluates at least one second image to obtain detection scores of at least one object to be measured.

[0122] Step S143: Obtain a detection result according to the second image and the detection score.

[0123] Wherein, the detection result includes detection scores of at least one object to be measured and their corresponding second mapping coordinates.

[0124] Optionally, after executing step S143, confidence evaluation needs to be performed on all obtained detection scores.

[0125] When it is determined that the detection scores of all screws to be measured output are greater than a preset threshold, it is displayed on the client that the detection is normal, specifically, it can be displayed on the display screen of the client, and subsequent steps are executed. Optionally, in this embodiment, the preset threshold can be 0.5.

[0126] When it is determined that the detection scores of all screws to be measured output are not all greater than the preset threshold, it is displayed on the client that the detection is abnormal and relevant information is saved. Specifically, it can be displayed on the display screen of the client, and a warning light is further lit to indicate that the detection is abnormal. Specifically, a yellow warning light is lit to prompt the production line personnel to handle it. At the same time, the next round of target detection is performed, and the detection object can be the current product to be measured or the next product to be measured.

[0127] Optionally, after performing step S143, according to the second mapping coordinates corresponding to each target to be measured, the screws to be measured can be marked on the client display with a green detection border or a red detection border. Among them, the green detection border indicates the existence of the screw to be measured where the standard screw is mapped to this position, and the red detection border indicates the non-existence of the screw to be measured where the standard screw is mapped to this position, which proves that the number of screws of the product to be measured is missing.

[0128] When all the detection borders displayed on the client's display are green, the client displays that the product to be measured is qualified, and the indicating light stays on to prompt the completion of the detection, specifically, the green indicating light stays on. At the same time, the next round of target detection is carried out, and the detection object can be the next product to be measured.

[0129] When any one of the detection borders displayed on the client's display is red, the client displays that the product to be measured is unqualified, and the indicating light stays on to prompt the stop of the detection, specifically, the red indicating light stays on, and the production line processes the unqualified product to be measured. At the same time, the next round of target detection is carried out, and the detection object can be the next product to be measured.

[0130] After completing all the steps in the above embodiment of the target detection method, the detection result obtained in step S143 is uploaded to the MES system in real time to achieve data closed-loop, the current detection process ends, and the next round of target detection is carried out.

[0131] This application realizes the matching of the first image and the standard image according to the detection configuration file, the first feature parameters of the first image, and the second feature parameters of the standard image. By adjusting the corresponding standard image and the detection configuration file, the detection of different first images is realized. At the same time, the second image of the target to be measured is obtained through the target detection method, and the second image of the target to be measured is further detected to realize the effective detection of the target to be measured on the surfaces of different products, and to identify abnormal detections such as incomplete photographing and occlusion caused by non-standard production line operations.

[0132] In addition, the target detection method of this application is used in combination with the MES system to realize the free configuration of detection tasks and detection targets, realize real-time detection, abnormal detection recognition, real-time upload of detection data, real-time control of production status, problem traceability analysis, and achieve self-consistency of the whole process data.

[0133] Please continue to refer to Figure 8 , Figure 8 which is a schematic framework diagram of an embodiment of the target detection device of this application. The target detection device 40 includes an acquisition module 41 and a calculation module 42.

[0134] Among them, the acquisition module 41 is used to acquire the first image, the standard image, and the detection configuration file.

[0135] The calculation module 42 is configured to obtain a first feature parameter of the first image, a second feature parameter of the standard image, and a matching relationship between the first feature parameter and the second feature parameter based on the first image and the standard image.

[0136] The calculation module 42 is further configured to obtain a second image of the target to be detected in the first image based on the detection configuration file, the first feature parameter, the second feature parameter, and the matching relationship;

[0137] The calculation module 42 is further configured to obtain a detection result based on the second image.

[0138] Please continue to refer to Figure 9 , Figure 9 FIG. is a schematic framework diagram of an embodiment of an electronic device according to the present application. The electronic device 50 includes a memory 51 and a processor 52 that are coupled to each other. The processor 52 is configured to execute program instructions stored in the memory 51 to implement the steps in any of the above-described target detection method embodiments. In a specific implementation scenario, the electronic device 50 may include, but is not limited to, a microcomputer, a server. In addition, the electronic device 50 may further include mobile devices such as a laptop computer, a tablet computer, etc., which are not limited herein.

[0139] Specifically, the processor 52 is configured to control itself and the memory 51 to implement the steps in any of the above-described target detection method embodiments. The processor 52 may also be referred to as a CPU (Central Processing Unit). The processor 52 may be an integrated circuit chip with signal processing capabilities. The processor 52 may also be a general-purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. Additionally, the processor 52 may be implemented jointly by integrated circuit chips.

[0140] Please refer to Figure 10 , Figure 10 FIG. is a schematic framework diagram of an embodiment of a computer-readable storage medium according to the present application. The computer-readable storage medium 60 stores program instructions 61 that can be run by a processor. The program instructions 61 are configured to implement the steps in any of the above-described target detection method embodiments.

[0141] In some embodiments, the functions or modules included in the device provided in this embodiment can be used to execute the methods described in the above method embodiments. The specific implementation can refer to the description of the above method embodiments. For the sake of brevity, it will not be repeated here.

[0142] The above descriptions of the various embodiments tend to emphasize the differences between the various embodiments. Their similarities or similarities can be referred to each other. For the sake of brevity, they will not be repeated in this article.

[0143] In several embodiments provided in this application, it should be understood that the disclosed methods and devices can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of modules or units is only a logical function division. In actual implementation, there may be other division methods. For example, units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed coupling or direct coupling or communication connection to each other can be through some interfaces. The indirect coupling or communication connection of the device or unit can be in electrical, mechanical or other forms.

[0144] In addition, in each embodiment of this application, the functional units can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit. The above integrated unit can be implemented in the form of hardware or in the form of a software functional unit.

[0145] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to execute all or part of the steps of the methods in various embodiments of this application. The foregoing storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical discs that can store program codes.

[0146] The above are only the embodiments of this application, and do not limit the patent scope of this application. Any equivalent structure or equivalent process transformation made by using the content of the specification and drawings of this application, or directly or indirectly applied in other related technical fields, shall be equally included in the patent protection scope of this application.

Claims

1. A target detection method, characterized in that, Including: Obtain a first image, a standard image, and a detection configuration file; Based on the first image and the standard image, obtain a first feature parameter of the first image, a second feature parameter of the standard image, and a matching relationship between the first feature parameter and the second feature parameter; Based on the detection configuration file, the first feature parameter, the second feature parameter, and the matching relationship, obtain a second image of a target to be detected in the first image; Based on the second image, obtain a detection result; The step of obtaining the first feature parameter of the first image, the second feature parameter of the standard image, and the matching relationship between the first feature parameter and the second feature parameter based on the first image and the standard image includes: Obtain the first feature parameter of the first image; wherein, the first feature parameter includes a first key point and a first sub-feature parameter; Based on the first key point and the first sub-feature parameter, obtain a first matching parameter of the first image; Obtain the second feature parameter of the standard image; wherein, the second feature parameter includes a second key point and a second sub-feature parameter; Based on the second key point and the second sub-feature parameter, obtain a second matching parameter of the standard image; Based on the first matching parameter and the second matching parameter, obtain a matching relationship between the first key point and the second key point; The detection configuration file includes a preset image transformation type. The step of obtaining the second image of the target to be detected in the first image based on the detection configuration file, the first feature parameter, the second feature parameter, and the matching relationship includes: Based on the preset image transformation type, the first key point, the second key point, and the matching relationship, obtain an inverse transformation matrix; Based on the inverse transformation matrix, obtain a first mapping coordinate of a standard target in the standard image in the first image; Based on the first mapping coordinate, obtain a region to be detected in the first image; In response to the center coordinate of the target to be detected in the region to be detected not exceeding the boundary of the first image, correct the first mapping coordinate to obtain a second mapping coordinate; wherein, the second mapping coordinate is the position coordinate of the target to be detected in the first image; Based on the second mapping coordinate, obtain the second image.

2. The method according to claim 1, characterized in that, The step of obtaining the matching relationship between the first key point and the second key point based on the first matching parameter and the second matching parameter includes: According to the first matching parameter and the second matching parameter, obtain the i-th point in the first key point and the j-th point in the second key point; wherein, the i-th point in the first key point matches the j-th point in the second key point; Based on the i-th point and the j-th point, obtain the number of pairs of matching points between the first key point and the second key point; In response to the number of pairs of matching points being greater than or equal to the number of key points required for image transformation, output the matching relationship; In response to the number of pairs of matching points being less than the number of key points required for image transformation, end target detection.

3. The method according to claim 1, wherein The number of the second images is at least one, and one of the second images includes one of the targets to be detected. The step of obtaining a detection result based on the second images includes: Classifying at least one of the second images; Evaluating at least one category of the second images to obtain detection scores of at least one of the targets to be detected; Obtaining the detection result according to the second images and the detection scores; wherein, the detection result includes detection scores of at least one of the targets to be detected and the second mapping coordinates.

4. The method according to claim 1, wherein The steps of obtaining the first image, the standard image, and the detection configuration file include: Obtaining a one-dimensional code of a product to be detected; Obtaining product information of the product to be detected based on the one-dimensional code; In response to successfully obtaining the product information, obtaining the first image; Obtaining the standard image and the detection configuration file based on the product information.

5. The method according to claim 4, wherein The method further includes: Establishing the standard image and the detection configuration file; wherein, the detection configuration file includes position information of all standard targets in the standard image; Modifying configurable items in the detection configuration file according to a preset image transformation type.

6. A target detection device, characterized in that, Including: An acquisition module, configured to acquire a first image, a standard image, and a detection configuration file; A calculation module, configured to obtain a first feature parameter of the first image, a second feature parameter of the standard image, and a matching relationship between the first feature parameter and the second feature parameter based on the first image and the standard image; The calculation module is further configured to obtain a second image of a target to be detected in the first image based on the detection configuration file, the first feature parameter, the second feature parameter, and the matching relationship; The calculation module is further configured to obtain a detection result based on the second image; The acquisition module is further configured to acquire the first feature parameter of the first image; wherein, the first feature parameter includes first key points and first sub-feature parameters; The calculation module is further configured to obtain a first matching parameter of the first image based on the first key points and the first sub-feature parameters; acquire a second feature parameter of the standard image; wherein, the second feature parameter includes second key points and second sub-feature parameters; obtain a second matching parameter of the standard image based on the second key points and the second sub-feature parameters; and obtain a matching relationship between the first key points and the second key points based on the first matching parameter and the second matching parameter. The detection configuration file includes a preset image transformation type, and the calculation module is further configured to: obtain an inverse transformation matrix based on the preset image transformation type, the first key point, the second key point, and the matching relationship; obtain a first mapping coordinate of a standard target in the standard image in the first image based on the inverse transformation matrix; obtain a to-be-detected area of the first image based on the first mapping coordinate; correct the first mapping coordinate to obtain a second mapping coordinate in response to that the center coordinate of the to-be-detected target in the to-be-detected area does not exceed the boundary of the first image, where the second mapping coordinate is the position coordinate of the to-be-detected target in the first image; and obtain the second image based on the second mapping coordinate.

7. An electronic device, characterized in that, It includes a memory and a processor coupled to each other, and the processor is configured to execute program instructions stored in the memory to implement the object detection method according to any one of claims 1-5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, and when the computer program can be executed by a processor, it implements the object detection method according to any one of claims 1-5.

Images