Product defect detection method and device, storage medium, and electronic device

Abstract

The present disclosure belongs to the field of product detection technology, and relates to a method and device for detecting product defects, a storage medium, and an electronic device. The method includes: obtaining edge-side resources and task requirement resources corresponding to the product to be detected, and performing resource matching calculations on the edge-side resources and the task requirement resources to obtain a resource matching value; obtaining a matching threshold corresponding to the resource matching value, and comparing the resource matching value with the matching threshold to obtain a matching comparison result; and performing defect detection on the product to be detected according to the matching comparison result. The present disclosure performs defect detection on the product to be detected by calculating the resource matching value, thereby improving the correlation between the production line and the edge side, and also improving the matching degree between the defect detection task volume and the edge resources, making rational use of various resources on the edge side, reducing network bandwidth pressure and network transmission damage, and improving the accuracy and real-time performance of defect detection.

Description

Technical Field

[0001] The present disclosure relates to the technical field of product detection, and in particular to a product defect detection method and a product defect detection device, a computer-readable storage medium, and an electronic device. Background Art

[0002] Defect detection is an important part of the industrial manufacturing field, and has extremely high requirements for real-time and accuracy. With the application of edge computing in the industrial field, how to effectively use the edge computing architecture, improve the utilization of edge resources, reduce network pressure, and improve the accuracy and real-time performance of defect detection has become a research focus in the industrial field.

[0003] Defect detection images are usually uploaded to the cloud for defect detection. However, since defect detection images are uncompressed ultra-high-definition images, uploading them to the cloud will take up a lot of bandwidth, and network transmission damage will affect detection accuracy and real-time requirements. Although deploying smart gateways on the field side can avoid network transmission damage, it cannot meet the high-precision requirements of industrial defect detection due to limited computing resources.

[0004] In view of this, there is an urgent need in the art to develop a new method and device for detecting product defects.

[0005] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present disclosure, and therefore may include information that does not constitute the prior art known to ordinary technicians in the field. Summary of the invention

[0006] The purpose of the present disclosure is to provide a product defect detection method, a product defect detection device, a computer-readable storage medium and an electronic device, thereby at least to a certain extent overcoming the technical problems of low detection accuracy and high transmission damage caused by the limitations of related technologies.

[0007] Other features and advantages of the present disclosure will become apparent from the following detailed description, or may be learned in part by the practice of the present disclosure.

[0008] According to a first aspect of an embodiment of the present invention, a method for detecting product defects is provided, which is applied to the edge side of an edge computing architecture, and the method includes:

[0009] Acquire edge-side resources and task-required resources corresponding to the product to be detected, and perform resource matching calculation on the edge-side resources and the task-required resources to obtain a resource matching value;

[0010] Acquire a matching threshold value corresponding to the resource matching value, and compare the resource matching value with the matching threshold value to obtain a matching comparison result;

[0011] Defect detection is performed on the product to be detected according to the matching comparison result.

[0012] In an exemplary embodiment of the present invention, the task required resources include: required computing resources, required storage resources and required network resources.

[0013] The obtaining of edge side resources and task requirement resources corresponding to the product to be detected includes:

[0014] Acquire the detection task corresponding to the product to be detected and the detection task quantity of the detection task, and perform resource requirement calculation on the detection task, the required computing resources, the required storage resources and the required network resources according to the detection task quantity to obtain the task required resources;

[0015] The edge computing resources, the edge storage resources, and the edge network resources are obtained to determine that the edge computing resources, the edge storage resources, and the edge network resources are edge-side resources.

[0016] In an exemplary embodiment of the present invention, the resource matching value includes a computing resource matching value, a storage resource matching value and a network resource matching value.

[0017] The comparing the resource matching value with the matching threshold to obtain a matching comparison result includes:

[0018] Compare the matching threshold with the computing resource matching value, the storage resource matching value or the network resource matching value to obtain a matching comparison result; or

[0019] The computing resource matching value, the storage resource matching value and the network resource matching value are compared to obtain a target resource matching value, and the target resource matching value is compared with the matching threshold to obtain a matching comparison result.

[0020] In an exemplary embodiment of the present invention, the step of performing defect detection on the product to be detected according to the matching comparison result includes:

[0021] If the matching comparison result is that the computing resource matching value, the storage resource matching value or the network resource matching value is greater than or equal to the matching threshold, the edge side resources and the task requirement resources are calculated to obtain the target task quantity;

[0022] Defect detection is performed on the product to be inspected according to the target task quantity.

[0023] In an exemplary embodiment of the present invention, after performing parameter calculation on the edge side resources and the task requirement resources to obtain the target task quantity, the method further includes:

[0024] If the resource matching value corresponding to the target task quantity is less than the matching threshold, a defect detection model is used to perform defect detection on the product to be detected.

[0025] If the matching comparison result is that the target resource matching value is less than the matching threshold, a defect detection model is used to perform defect detection on the product to be detected.

[0026] In an exemplary embodiment of the present invention, the step of performing defect detection on the product to be detected by using a defect detection model includes:

[0027] Deploy the defect detection model on the terminal device at the edge side, and obtain a quantity threshold corresponding to the target task quantity or the detection task quantity, wherein the terminal device includes an intelligent gateway and an edge computing device;

[0028] If the target task quantity or the detection task quantity is less than the quantity threshold, performing defect detection on the product to be detected by using the defect detection model deployed on the intelligent gateway;

[0029] If the target task quantity or the detection task quantity is greater than or equal to the quantity threshold, defect detection is performed on the product to be inspected using the defect detection model deployed on the smart gateway and the edge computing device.

[0030] In an exemplary embodiment of the present invention, the method of performing defect detection on the product to be detected by using the defect detection model deployed on the intelligent gateway and the edge computing device includes:

[0031] Performing model layering processing on the defect detection model to obtain model layering parameters, and performing model parameter calculation on the model layering parameters to obtain gateway demand resources corresponding to the intelligent gateway;

[0032] Acquire a resource threshold of the intelligent gateway corresponding to the gateway demand resources, and compare the gateway resources with the gateway demand resources to obtain a gateway comparison result, wherein the gateway demand resources include gateway computing resources, gateway storage resources, and gateway network resources;

[0033] If the gateway comparison result is that the resource threshold is less than or equal to the gateway computing resources, the gateway storage resources or the gateway network resources, defect detection is performed on the product to be inspected in layers using the defect detection model deployed on the smart gateway and edge computing device.

[0034] According to a second aspect of an embodiment of the present invention, a device for detecting product defects is provided, which is applied to the edge side of an edge computing architecture, including:

[0035] A resource matching module is configured to obtain edge-side resources and task-required resources corresponding to the product to be detected, and perform resource matching calculation on the edge-side resources and the task-required resources to obtain a resource matching value;

[0036] A resource comparison module is configured to obtain a matching threshold corresponding to the resource matching value, and compare the resource matching value with the matching threshold to obtain a matching comparison result;

[0037] The product detection module is configured to perform defect detection on the product to be detected according to the matching comparison result.

[0038] According to a third aspect of an embodiment of the present invention, there is provided an electronic device, comprising: a processor and a memory; wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the method for detecting product defects in any of the above exemplary embodiments is implemented.

[0039] According to a fourth aspect of an embodiment of the present invention, there is provided a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the method for detecting product defects in any of the above exemplary embodiments is implemented.

[0040] It can be seen from the above technical solutions that the product defect detection method, product defect detection device, computer storage medium and electronic device in the exemplary embodiments of the present disclosure have at least the following advantages and positive effects:

[0041] In the method and apparatus provided in the exemplary embodiments of the present disclosure, defect detection is performed on the product to be inspected by calculating the resource matching value, thereby improving the correlation between the production line and the edge side, and also improving the matching degree between the defect detection task volume and the edge resources. The various resources on the edge side are rationally utilized, the network bandwidth pressure and network transmission damage are reduced, and the accuracy and real-time performance of defect detection are improved.

[0042] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] The accompanying drawings herein are incorporated into the specification and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the specification are used to explain the principles of the present disclosure. Obviously, the accompanying drawings described below are only some embodiments of the present disclosure, and for ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without creative work.

[0044] Figure 1 A schematic diagram schematically illustrates a flow chart of a method for detecting product defects in an exemplary embodiment of the present disclosure;

[0045] Figure 2 A schematic diagram schematically illustrates a flow chart of a method for acquiring edge-side resources and task-required resources in an exemplary embodiment of the present disclosure;

[0046] Figure 3 A flowchart schematically illustrates a method for obtaining a matching comparison result in an exemplary embodiment of the present disclosure;

[0047] Figure 4 A schematic flow chart schematically illustrates a method for performing defect detection on a product to be detected in an exemplary embodiment of the present disclosure;

[0048] Figure 5 A schematic flowchart of a method for performing defect detection on a product to be detected by using a defect detection model in an exemplary embodiment of the present disclosure is schematically shown;

[0049] Figure 6 A flowchart schematically illustrating a method for further utilizing a defect detection model to perform defect detection on a product to be detected in an exemplary embodiment of the present disclosure;

[0050] Figure 7 A schematic flowchart of a method for performing defect detection on a product to be detected by using defect detection models deployed on an intelligent gateway and an edge computing device simultaneously in an exemplary embodiment of the present disclosure is schematically shown;

[0051] Figure 8 A schematic diagram schematically illustrates a network structure of a convolutional neural network model in an exemplary embodiment of the present disclosure;

[0052] Fig. 9 A schematic diagram schematically illustrates a structure of a layered execution of a convolutional neural network in an exemplary embodiment of the present disclosure;

[0053] Fig.10 A system architecture diagram schematically illustrates a product defect detection system in an application scenario in an exemplary embodiment of the present disclosure;

[0054] Fig.11 A schematic diagram schematically illustrates a flow chart of a method for detecting product defects in an application scenario in an exemplary embodiment of the present disclosure;

[0055] Fig.12 A schematic diagram schematically shows the structure of a device for detecting product defects in an exemplary embodiment of the present disclosure;

[0056] Fig.13 An electronic device for implementing a method for detecting product defects in an exemplary embodiment of the present disclosure is schematically shown;

[0057] Fig.14A computer-readable storage medium for implementing a method for detecting product defects in an exemplary embodiment of the present disclosure is schematically shown. DETAILED DESCRIPTION

[0058] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as being limited to the examples set forth herein; on the contrary, these embodiments are provided so that the present disclosure will be more comprehensive and complete, and the concepts of the example embodiments are fully conveyed to those skilled in the art. The described features, structures, or characteristics may be combined in one or more embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present disclosure. However, those skilled in the art will appreciate that the technical solutions of the present disclosure may be practiced while omitting one or more of the specific details, or other methods, components, devices, steps, etc. may be adopted. In other cases, known technical solutions are not shown or described in detail to avoid obscuring various aspects of the present disclosure.

[0059] The terms "a", "an", "the" and "said" are used in this specification to indicate the presence of one or more elements / components / etc.; the terms "including" and "having" are used to express an open-ended inclusion and mean that additional elements / components / etc. may exist in addition to the listed elements / components / etc.; the terms "first" and "second" etc. are used only as labels and are not intended to limit the quantity of their objects.

[0060] In addition, the accompanying drawings are only schematic illustrations of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the drawings represent the same or similar parts, and their repeated descriptions will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically independent entities.

[0061] Defect detection is an important part of the industrial manufacturing field, and has extremely high requirements for real-time and accuracy. With the application of edge computing in the industrial field, how to effectively use the edge computing architecture, improve the utilization of edge resources, reduce network pressure, and improve the accuracy and real-time performance of defect detection has become a research focus in the industrial field.

[0062] Defect detection images are uncompressed ultra-high-definition images with a data rate of up to 100 megabits per second. Uploading them directly to the cloud or edge cloud will take up a lot of bandwidth. At the same time, network transmission damage will affect detection accuracy and real-time requirements.

[0063] When the smart gateway is deployed on-site, network transmission damage can be avoided and detection speed can be improved. However, due to limited computing resources, it is also unable to meet the high-precision requirements of industrial defect detection.

[0064] Based on this, the defect detection task is weakly correlated with the resource utilization of the edge. Usually, resource waste due to too low edge resource utilization or a decrease in detection accuracy due to overload of edge resource utilization will occur.

[0065] In response to the problems existing in the related art, the present disclosure proposes a product defect detection method applied to the edge side of the edge computing architecture. Figure 1 A flow chart showing a method for detecting product defects is shown in FIG. Figure 1 As shown, the method for detecting product defects includes at least the following steps:

[0066] Step S110: Acquire edge-side resources and task-required resources corresponding to the product to be detected, and perform resource matching calculation on the edge-side resources and the task-required resources to obtain a resource matching value.

[0067] Step S120: Obtain a matching threshold corresponding to the resource matching value, and compare the resource matching value with the matching threshold to obtain a matching comparison result.

[0068] Step S130: Perform defect detection on the product to be inspected according to the matching comparison result.

[0069] In the exemplary embodiment of the present disclosure, defect detection is performed on the product to be inspected by calculating the resource matching value, which improves the correlation between the production line and the edge side, and also improves the matching degree between the defect detection task volume and the edge resources. The various resources on the edge side are reasonably utilized, the network bandwidth pressure and network transmission damage are reduced, and the accuracy and real-time performance of defect detection are improved.

[0070] The following is a detailed description of each step of the product defect detection method.

[0071] In step S110, edge-side resources and task-required resources corresponding to the product to be detected are obtained, and resource matching calculation is performed on the edge-side resources and the task-required resources to obtain a resource matching value.

[0072] In an exemplary embodiment of the present disclosure, when the product to be inspected is an industrial product, an industrial camera can be used to photograph the product to be inspected. Furthermore, the intelligent gateway deployed on the edge side of the edge computing architecture can pull the video stream shot by the industrial camera and extract key frame images from the video stream to perceive and obtain the task requirement resources of the product to be inspected.

[0073] In an optional embodiment, the task required resources include: required computing resources, required storage resources and required network resources. Figure 2 A flow chart of a method for obtaining edge resources and task-required resources is shown, such as Figure 2As shown, the method includes at least the following steps: in step S210, the detection tasks corresponding to the product to be detected and the detection task quantity of the detection tasks are obtained, and according to the detection task quantity, resource requirements of the detection tasks, required computing resources, required storage resources and required network resources are calculated to obtain task required resources.

[0074] The task required resources can be calculated according to formula (1):

[0075] R=(R1,R2,…,R n )

[0076] =(r1c1+…+r n c1,r1c2+…+r n c2,r1c3+…+r n c3) (1)

[0077] Among them, R is the task required resource, r1, r2, r3, …, r n is the detection task, n represents the number of detection tasks, c1, c2, c3 are the requirements of each detection task for computing resources, storage resources and network resources, that is, the required computing resources, required storage resources and required network resources.

[0078] In step S220, edge computing resources, edge storage resources, and edge network resources are obtained to determine that the edge computing resources, edge storage resources, and edge network resources are edge-side resources.

[0079] The edge side resources can be determined according to formula (2):

[0080] N=(N1,N2,…,N m ) (2)

[0081] Where N is the edge side resources, including the gateway resources of the smart gateway deployed on the edge side And the MEC resources of edge computing devices deployed on the edge side in,

[0082] Since only edge computing resources, edge storage resources and edge network resources are considered, m=3.

[0083] In this exemplary embodiment, by sensing the current edge-side resources and the task requirement resources corresponding to the product to be inspected, a data basis and support is provided for the inspection scheduling of product defects, thereby improving the correlation between the production line and the edge side.

[0084] After the edge side resources and the task required resources are obtained, a resource matching calculation may be performed on the edge side resources and the task required resources to obtain a resource matching value.

[0085] Specifically, the resource matching calculation method can be implemented according to formula (3):

[0086]

[0087]

[0088] Among them, T is the resource matching value, T1 is the computing resource matching value included in the resource matching value, T2 is the storage resource matching value in the resource matching value, and T3 is the network resource matching value included in the resource matching value.

[0089] In step S120, a matching threshold corresponding to the resource matching value is obtained, and the resource matching value is compared with the matching threshold to obtain a matching comparison result.

[0090] In an exemplary embodiment of the present disclosure, after the resource matching value is calculated, a matching threshold corresponding to the resource matching value may be acquired.

[0091] The matching threshold is the decision threshold value of the edge resource when the demand for any resource reaches the edge resource, which can be set to a. The value of a can be determined according to the production line resources, and this exemplary embodiment does not make any special limitation on this.

[0092] By matching the threshold a, it can be determined whether the edge resources meet the requirements of the production line detection task.

[0093] Furthermore, the resource matching value may be compared with a matching threshold to obtain a corresponding matching comparison result.

[0094] In an optional embodiment, the resource matching value includes a computing resource matching value, a storage resource matching value, and a network resource matching value. Figure 3 A flow chart of a method for obtaining a matching comparison result is shown, such as Figure 3 As shown, the method at least includes the following steps: In step S310, the matching threshold is compared with the computing resource matching value, the storage resource matching value or the network resource matching value to obtain a matching comparison result.

[0095] Since the resource matching value includes three parts: computing resource matching value, storage resource matching value and network resource matching value, the three resource matching values ​​can be compared with the matching thresholds respectively to obtain the matching comparison result.

[0096] The matching comparison result may be that any resource matching value is greater than or equal to the matching threshold, or any resource matching value is less than the matching threshold.

[0097] In step S320, the computing resource matching value, the storage resource matching value and the network resource matching value are compared to obtain a target resource matching value, and the target resource matching value is compared with a matching threshold to obtain a matching comparison result.

[0098] Since the resource matching value includes three parts: computing resource matching value, storage resource matching value and network resource matching value, the computing resource matching value, storage resource matching value and network resource matching value can be compared first to obtain the target resource matching value, and then the target resource matching value is compared with the matching threshold to obtain the matching comparison result.

[0099] The matching comparison result may be that the target resource matching value is less than the matching threshold, or may be that the target resource matching value is greater than or equal to the matching threshold.

[0100] Among them, the target resource matching value can be the largest resource matching value among the computing resource matching value, storage resource matching value and network resource matching value, and other target resource matching value determination methods can also be set according to actual conditions. This exemplary embodiment does not make any special limitations on this.

[0101] In this exemplary embodiment, a variety of methods for comparing resource matching values ​​with matching thresholds are provided, which enriches the application scenarios of defect detection and also provides a basis and foundation for the defect detection method.

[0102] In step S130, defect detection is performed on the product to be inspected according to the matching comparison result.

[0103] In an exemplary embodiment of the present disclosure, after obtaining the matching comparison result, defect detection can be performed on the product to be inspected.

[0104] In an alternative embodiment, Figure 4 A schematic diagram of a process for performing defect detection on a product to be detected is shown. Figure 4 As shown, the method includes at least the following steps: in step S410, if the matching comparison result is that the computing resource matching value, the storage resource matching value or the network resource matching value is greater than or equal to the matching threshold, the edge side resources and the task requirement resources are calculated to obtain the target task quantity.

[0105] In the manner of comparing the resource matching value provided in step S310 with the matching threshold, the matching comparison result that can be obtained is that any one of the computing resource matching value, storage resource matching value and network resource matching value is greater than or equal to the matching threshold.

[0106] This kind of matching comparison result indicates that there is a shortage of resources on the edge side, and the production line tasks can be dynamically adjusted.

[0107] Specifically, the workload of the production line can be controlled, that is, the workload of inspection can be appropriately reduced to alleviate the inspection pressure on the edge side.

[0108] Then, the reduced number of target tasks can be calculated according to formula (4):

[0109] n * = argmax n∈[1,n] (f(N,R)) (4)

[0110] Among them, n * is the target number of tasks.

[0111] argmax is a function that finds the parameters of a function. When there is another function y=f(x), if the result x0=argmax(f(x)), it means that when the function f(x) takes x=x0, the maximum value of the range of values ​​of f(x) is obtained. Therefore, n * The number of tasks corresponding to the maximum value of (f(N,R).

[0112] In step S420, defect detection is performed on the product to be inspected according to the target task quantity.

[0113] After the target task quantity is calculated, the inspection task quantity may be adjusted to the size of the target task quantity, and defect inspection of the product to be inspected may be performed according to the target task quantity.

[0114] In this exemplary embodiment, production line tasks can be scheduled according to the matching comparison results to realize defect detection of products to be inspected, thereby improving the matching degree between the number of tasks and edge resources and realizing the rational use of edge resources.

[0115] In addition, after obtaining the target number of tasks, the resource matching values ​​corresponding to the number of tasks can be further matched to achieve defect detection of the product to be inspected through edge task allocation.

[0116] In an alternative embodiment, Figure 5 A flow chart of a method for performing defect detection on a product to be detected using a defect detection model is shown. Figure 5 As shown, the method includes at least the following steps: in step S510, if the resource matching value corresponding to the target task quantity is less than the matching threshold, the defect detection model is used to perform defect detection on the product to be detected.

[0117] When the number of detection tasks is equal to the number of target tasks, Figure 2 The corresponding resource matching value is obtained by the method shown and the calculation method of step S110. The resource matching value may also include a computing resource matching value, a storage resource matching value and a network resource matching value.

[0118] Further, according to the comparison method of step S320, a matching comparison result between the target resource matching value corresponding to the computing resource matching value, storage resource matching value and network resource matching value at this time and the matching threshold can be obtained.

[0119] When the matching comparison result at this time is that the target resource matching value is less than the matching threshold, the defect detection model can be used to perform defect detection on the product to be inspected.

[0120] In step S520, if the matching comparison result is that the target resource matching value is less than the matching threshold, the defect detection model is used to perform defect detection on the product to be detected.

[0121] In addition to the situation in step S510, when the matching comparison result in step S320 is that the target resource matching value is less than the matching threshold, the defect detection model can be used to perform defect detection on the product to be detected.

[0122] The method of performing defect detection on the product to be inspected by using the defect detection model in step S510 and step S520 can be implemented by the defect detection model deployed on the intelligent gateway and the edge computing device.

[0123] In an alternative embodiment, Figure 6 A flow chart of a method for further utilizing a defect detection model to perform defect detection on a product to be detected is shown, such as Figure 6 As shown, the method includes at least the following steps: in step S610, the defect detection model is deployed on the terminal device on the edge side, and a quantity threshold corresponding to the target task quantity or the detection task quantity is obtained, and the terminal device includes an intelligent gateway and an edge computing device.

[0124] Deploying the defect detection model on the edge side smart gateway and edge computing device may be to deploy a convolutional neural network on the edge side smart gateway and edge computing device. That is, the defect detection model may be a convolutional neural network model or other models, and this exemplary embodiment does not specifically limit this.

[0125] Among them, the intelligent gateway that deploys a part of the convolutional neural network model provides the first level edge detection, and the edge computing device that deploys a part of the convolutional neural network model provides the second level edge detection.

[0126] The edge computing device may be a device capable of implementing edge computing technology.

[0127] It is worth noting that the intelligent gateway must be deployed on the field side, but the edge computing device can be deployed on the field side, in the operator's computer room, or in a farther place. This exemplary embodiment does not specifically limit this.

[0128] The quantity threshold may be a threshold for determining whether the number of target tasks or the number of detection tasks at this time is more or less, and may be set according to actual conditions. Generally, the quantity threshold may be determined by a resource threshold of the intelligent gateway, or may be equal to the resource threshold, and this exemplary embodiment does not specifically limit this.

[0129] In step S620, if the target task quantity or the detection task quantity is less than the quantity threshold, the defect detection model deployed on the intelligent gateway is used to perform defect detection on the product to be inspected.

[0130] After the quantity threshold is obtained, the number of target tasks or the number of detection tasks may be compared with the quantity threshold.

[0131] When the comparison result is that the number of target tasks or the number of detection tasks is less than the number threshold, it indicates that there are fewer tasks at this time.

[0132] Furthermore, it is also possible to determine whether the computing resources, storage resources, and network resources provided by the intelligent gateway meet the requirements of the number of target tasks or the number of detection tasks.

[0133] When the computing resources, storage resources, and network resources provided by the intelligent gateway can meet the requirements of the number of target tasks or the number of detection tasks, the defect detection model deployed on the intelligent gateway can be used to perform defect detection on the product to be inspected.

[0134] In step S630, if the number of target tasks or the number of detection tasks is greater than or equal to the quantity threshold, defect detection is performed on the product to be inspected using the defect detection model deployed on the smart gateway and the edge computing device.

[0135] After the quantity threshold is obtained, the number of target tasks or the number of detection tasks may be compared with the quantity threshold.

[0136] When the comparison result is that the number of target tasks or the number of detection tasks is greater than or equal to the number threshold, it indicates that there are many tasks at this time. Therefore, the defect detection model deployed on the smart gateway and edge computing device can be used to perform defect detection on the product to be inspected.

[0137] In an alternative embodiment, Figure 7 The flowchart of the method for simultaneously using the defect detection models deployed on the intelligent gateway and the edge computing device to detect defects on the product to be inspected is shown as follows: Figure 7 As shown, the method includes at least the following steps: in step S710, the defect detection model is subjected to model layering processing to obtain model layering parameters, and the model layering parameters are subjected to model parameter calculation to obtain gateway demand resources corresponding to the intelligent gateway.

[0138] Figure 8 A schematic diagram of the network structure of the convolutional neural network model is shown, Figure 8 As shown in Figure 1, when the number of target tasks or detection tasks is n, any layer in the convolutional neural network model can be split into task splitting layers h1, h2, …, h n .

[0139] Therefore, model layering parameters such as the height of the convolution kernel can be obtained.

[0140] Furthermore, the gateway resource requirements of the intelligent gateway can be calculated according to formula (5), formula (6) and formula (7):

[0141]

[0142] Among them, TX is the gateway network resource in the gateway demand resources, τ is the processing time required for defect detection models such as convolutional neural network models, TD is the total amount of data transmitted from the smart gateway to the edge computing device, is the data depth of the input convolution layer, is the height of the input convolutional layer, For h n The data depth of the convolutional layer output, For h n The height of the convolutional layer output.

[0143]

[0144] Among them, TM is the memory required for intelligent gateway calculation, that is, the gateway storage resource. is the data depth of the input convolution layer, is the height of the input convolutional layer, is the height of the convolution kernel, For h n The data depth of the convolutional layer output, For h n The height of the convolutional layer output.

[0145]

[0146] Among them, TC is the computing resource required by the intelligent gateway, that is, the gateway computing resource. is the data depth of the input convolution layer, is the height of the input convolutional layer, is the height of the convolution kernel, For h n The data depth of the convolutional layer output, For hn The height of the convolutional layer output.

[0147] In step S720, a resource threshold of the intelligent gateway corresponding to the gateway demand resources is obtained, and the gateway resources are compared with the gateway demand resources to obtain a gateway comparison result. The gateway demand resources include gateway computing resources, gateway storage resources and gateway network resources.

[0148] Since the gateway demand resources include gateway computing resources, gateway storage resources and gateway network resources, the resource threshold of the intelligent gateway corresponding to the gateway demand resources can be obtained. The resource threshold can be based on the resource N of the intelligent gateway. g Sure.

[0149] Specifically, the resource threshold can be λN g .in, The introduction of λ can reserve computing power for one end of the smart gateway to avoid high concurrency of the smart gateway.

[0150] Furthermore, the resource threshold may be compared with any one of the gateway computing resources, the gateway storage resources, and the gateway network resources to obtain a gateway comparison result.

[0151] In step S730, if the gateway comparison result is that the resource threshold is less than or equal to the gateway computing resources, gateway storage resources or gateway network resources, the defect detection model deployed on the smart gateway and edge computing device is used to perform defect detection on the product to be inspected in a layered manner.

[0152] When the gateway comparison result is that the resource threshold is less than any one of the gateway computing resources, gateway storage resources, and gateway network resources, then h1, h2, …, h n The number of network layers of the defect detection model such as convolutional neural network executed on the smart gateway side for n tasks.

[0153] Fig. 9 A schematic diagram of the structure of the layered execution of the convolutional neural network is shown, Fig. 9 As shown in Figure 2, for the i-th defect detection task of the product to be inspected, the intelligent gateway executes the deployed convolutional neural network. Layer defect detection model, executed on edge computing devices Defect detection model for layers.

[0154] In this exemplary embodiment, the defect detection tasks performed on different terminal devices can be determined through different comparison results of the target task number or the detection task number and the quantity threshold, which gives full play to the near-field advantage of the intelligent gateway, rationally utilizes edge resources, reduces network pressure and network transmission damage, and improves the accuracy and real-time performance of defect detection.

[0155] The following is a detailed description of the method for detecting product defects in an embodiment of the present disclosure in conjunction with an application scenario.

[0156] Fig.10 The system architecture diagram of the product defect detection system in the application scenario is shown as follows: Fig.10 As shown, the system includes edge-side network resources, computing resources and storage resources, including a defect detection module, a resource perception module, a task scheduling module and a model training optimization module.

[0157] Among them, the resource perception module can judge the matching degree between edge resources and production line tasks according to the resource conditions of the intelligent gateway and edge computing equipment and the amount of production line detection tasks, and send the prediction results to the task scheduling module.

[0158] Specifically, the intelligent resource perception algorithm adopted by the resource perception module can be to obtain the detection tasks corresponding to the products to be detected and the number of detection tasks of the detection tasks, and perform resource requirement calculations on the detection tasks, required computing resources, required storage resources and required network resources according to the number of detection tasks to obtain the task requirement resources.

[0159] The task required resources can be calculated according to formula (1).

[0160] Obtain edge computing resources, edge storage resources, and edge network resources to determine that the edge computing resources, edge storage resources, and edge network resources are edge-side resources.

[0161] Among them, the edge side resources can be determined according to formula (2).

[0162] After the edge side resources and the task required resources are obtained, a resource matching calculation may be performed on the edge side resources and the task required resources to obtain a resource matching value.

[0163] Specifically, the resource matching calculation method can be implemented according to formula (3).

[0164] After calculating the resource matching value T, the resource perception module can transmit the resource matching value T to the task scheduling module and adjust it according to the situation of the edge side resources.

[0165] After the resource matching value is calculated, a matching threshold corresponding to the resource matching value may be obtained.

[0166] The matching threshold is the threshold value of the edge resource when the demand of any resource reaches the edge resource, which can be set to a. The value of a can be determined according to the production line resources, and this exemplary embodiment does not specifically limit this. The matching threshold a can be used to determine whether the edge side resources meet the requirements of the production line detection task.

[0167] Furthermore, the resource matching value may be compared with a matching threshold to obtain a corresponding matching comparison result.

[0168] First, the matching threshold is compared with the computing resource matching value, the storage resource matching value or the network resource matching value to obtain a matching comparison result.

[0169] Since the resource matching value includes three parts: computing resource matching value, storage resource matching value and network resource matching value, the three resource matching values ​​can be compared with the matching thresholds respectively to obtain the matching comparison result.

[0170] The matching comparison result may be that any resource matching value is greater than or equal to the matching threshold, or any resource matching value is less than the matching threshold.

[0171] Secondly, the computing resource matching value, the storage resource matching value and the network resource matching value are compared to obtain a target resource matching value, and the target resource matching value is compared with a matching threshold to obtain a matching comparison result.

[0172] Since the resource matching value includes three parts: computing resource matching value, storage resource matching value and network resource matching value, the computing resource matching value, storage resource matching value and network resource matching value can be compared first to obtain the target resource matching value, and then the target resource matching value is compared with the matching threshold to obtain the matching comparison result.

[0173] The matching comparison result may be that the target resource matching value is less than the matching threshold, or may be that the target resource matching value is greater than or equal to the matching threshold.

[0174] Among them, the target resource matching value can be the largest resource matching value among the computing resource matching value, storage resource matching value and network resource matching value, and other target resource matching value determination methods can also be set according to actual conditions. This exemplary embodiment does not make any special limitations on this.

[0175] If the matching comparison result is that the computing resource matching value, storage resource matching value or network resource matching value is greater than or equal to the matching threshold, the edge side resources and task demand resources are calculated to obtain the target task quantity.

[0176] In the first method of comparing the resource matching value provided with the matching threshold, the matching comparison result that can be obtained is that any one of the resource matching values ​​of the computing resource matching value, the storage resource matching value and the network resource matching value is greater than or equal to the matching threshold, so the production line task can be scheduled.

[0177] This kind of matching comparison result indicates that there is a shortage of resources on the edge side, and the production line tasks can be dynamically adjusted.

[0178] Specifically, the workload of the production line can be controlled, that is, the workload of inspection can be appropriately reduced to alleviate the inspection pressure on the edge side.

[0179] Then, the reduced target task number can be calculated according to formula (4).

[0180] After the target task quantity is calculated, the inspection task quantity may be adjusted to the size of the target task quantity, and defect inspection of the product to be inspected may be performed according to the target task quantity.

[0181] In addition, after obtaining the target number of tasks, the resource matching values ​​corresponding to the number of tasks can be further matched to achieve defect detection of the product to be inspected through edge task allocation.

[0182] If the resource matching value corresponding to the target task quantity is less than the matching threshold, the defect detection model is used to perform defect detection on the product to be inspected to achieve model hierarchical execution.

[0183] When the number of detection tasks is equal to the number of target tasks, the corresponding resource matching value can be calculated, which can also include a computing resource matching value, a storage resource matching value, and a network resource matching value.

[0184] Furthermore, a matching comparison result between the target resource matching value corresponding to the computing resource matching value, the storage resource matching value, and the network resource matching value at this time and the matching threshold is obtained.

[0185] When the matching comparison result at this time is that the target resource matching value is less than the matching threshold, the defect detection model can be used to perform defect detection on the product to be inspected.

[0186] Alternatively, if the matching comparison result is that the target resource matching value is less than the matching threshold, the defect detection model is used to perform defect detection on the product to be inspected.

[0187] When the matching comparison result is that the target resource matching value is less than the matching threshold, the defect detection model is used to perform defect detection on the product to be inspected to achieve model hierarchical execution.

[0188] The method of using the defect detection model to perform defect detection on the product to be inspected can be achieved by deploying the defect detection model on the intelligent gateway and the edge computing device.

[0189] The defect detection model is deployed on the terminal device at the edge side, and a quantity threshold corresponding to the target task quantity or the detection task quantity is obtained. The terminal device includes an intelligent gateway and an edge computing device.

[0190] Deploying the defect detection model on the edge side smart gateway and edge computing device may be to deploy a convolutional neural network on the edge side smart gateway and edge computing device. That is, the defect detection model may be a convolutional neural network model or other models, and this exemplary embodiment does not specifically limit this.

[0191] Among them, the intelligent gateway that deploys a part of the convolutional neural network model provides the first level edge detection, and the edge computing device that deploys a part of the convolutional neural network model provides the second level edge detection.

[0192] The edge computing device may be a device capable of implementing edge computing technology.

[0193] It is worth noting that the intelligent gateway must be deployed on the field side, but the edge computing device can be deployed on the field side, in the operator's computer room, or in a farther place. This exemplary embodiment does not specifically limit this.

[0194] The quantity threshold may be a threshold for determining whether the number of target tasks or the number of detection tasks at this time is more or less, and may be set according to actual conditions. Generally, the quantity threshold may be determined by a resource threshold of the intelligent gateway, or may be equal to the resource threshold, and this exemplary embodiment does not specifically limit this.

[0195] Furthermore, the number of neural network layers executed by the smart gateway and edge computing devices is intelligently allocated according to the target number of tasks or the number of detection tasks of the production line, and a strategy of prioritizing the use of smart gateway resources is adopted.

[0196] If the number of target tasks or the number of detection tasks is less than the quantity threshold, the defect detection model deployed on the intelligent gateway is used to perform defect detection on the product to be inspected.

[0197] After the quantity threshold is obtained, the number of target tasks or the number of detection tasks may be compared with the quantity threshold.

[0198] When the comparison result is that the number of target tasks or the number of detection tasks is less than the number threshold, it indicates that there are fewer tasks at this time.

[0199] Furthermore, it is also possible to determine whether the computing resources, storage resources, and network resources provided by the intelligent gateway meet the requirements of the number of target tasks or the number of detection tasks.

[0200] When the computing resources, storage resources, and network resources provided by the intelligent gateway can meet the requirements of the number of target tasks or the number of detection tasks, the defect detection model deployed on the intelligent gateway can be used to perform defect detection on the product to be inspected.

[0201] If the number of target tasks or the number of detection tasks is greater than or equal to the quantity threshold, the defect detection model deployed on the intelligent gateway and the edge computing device is used to perform defect detection on the product to be inspected.

[0202] After the quantity threshold is obtained, the number of target tasks or the number of detection tasks may be compared with the quantity threshold.

[0203] When the comparison result is that the number of target tasks or the number of detection tasks is greater than or equal to the number threshold, it indicates that there are many tasks at this time. Therefore, the defect detection model deployed on the smart gateway and edge computing device can be used to perform defect detection on the product to be inspected.

[0204] The defect detection model is subjected to model layering processing to obtain model layering parameters, and the model layering parameters are subjected to model parameter calculation to obtain gateway demand resources corresponding to the intelligent gateway.

[0205] Specifically, model layering parameters such as the height of the convolution kernel can be obtained.

[0206] Furthermore, the gateway demand resources of the intelligent gateway can be calculated according to formula (5), formula (6) and formula (7), and the gateway demand resources include gateway computing resources, gateway storage resources and gateway network resources.

[0207] A resource threshold of the intelligent gateway corresponding to the gateway demand resource is obtained, and the gateway resource is compared with the gateway demand resource to obtain a gateway comparison result.

[0208] Since the gateway demand resources include gateway computing resources, gateway storage resources and gateway network resources, the resource threshold of the intelligent gateway corresponding to the gateway demand resources can be obtained. The resource threshold can be based on the resource N of the intelligent gateway. g Sure.

[0209] Specifically, the resource threshold can be λN g .in, The introduction of λ can reserve computing power for one end of the smart gateway to avoid high concurrency of the smart gateway.

[0210] Furthermore, the resource threshold may be compared with any one of the gateway computing resources, the gateway storage resources, and the gateway network resources to obtain a gateway comparison result.

[0211] If the gateway comparison result is that the resource threshold is less than or equal to the gateway computing resources, gateway storage resources or gateway network resources, the defect detection model deployed on the smart gateway and edge computing device is used to perform defect detection on the product to be inspected in a layered manner.

[0212] When the gateway comparison result is that the resource threshold is less than any one of the gateway computing resources, gateway storage resources, and gateway network resources, then h1, h2, …, h n The number of network layers of the defect detection model such as convolutional neural network executed on the smart gateway side for n tasks.

[0213] For the i-th defect detection task of the product to be inspected, the smart gateway executes the deployed convolutional neural network Layer defect detection model, executed on edge computing devices Defect detection model for layers.

[0214] exist Fig.10 In the system architecture diagram shown, the defect detection module is deployed on the edge side, including the intelligent gateway deployed on the field side and the edge computing device deployed in the region. The defect detection module uploads the usage of computing resources, storage resources, and network resources on the edge side to the resource perception module, providing a basis for subsequent task scheduling.

[0215] The defect detection module connects to the model training and optimization model, supports the transmission of image feature data to the cloud side, and deploys and updates the defect detection model sent by the cloud side.

[0216] The model training optimization module can train and update the model based on the image feature data uploaded by the edge side, and compress, prune, quantize and other operations on the defect detection model to obtain a more streamlined defect detection model while ensuring the accuracy of defect detection. In addition, the streamlined defect detection model can be sent to the edge side for model update and deployment.

[0217] In the product defect detection system, a cloud-edge collaborative processing method is adopted, and cloud-edge devices are used reasonably to improve resource utilization. In addition, the edge computing architecture is adopted, and the edge side uses a near-field intelligent gateway and edge computing device collaboration to reduce network bandwidth pressure and network transmission damage, and improve the accuracy and real-time performance of defect detection.

[0218] In response to the problem of weak correlation between production line tasks and edge resources in related technologies, this product defect detection system also introduced a resource perception module and a task allocation module, and proposed an intelligent resource perception algorithm to improve the matching degree between task volume and edge resources.

[0219] In addition, a task allocation module was introduced, and a hierarchical strategy for defect detection models was proposed. Defect detection models such as convolutional neural networks were deployed in a split manner. The resources of smart gateways and edge computing devices were coordinated to achieve rational use of edge resources.

[0220] Fig.11 A flow chart of a method for detecting product defects in an application scenario is shown, such as Fig.11 As shown, in step S1110, the production line task volume.

[0221] When the product to be inspected is an industrial product, an industrial camera can be used to shoot the product to be inspected. Furthermore, the smart gateway deployed on the edge side of the edge computing architecture can pull the video stream shot by the industrial camera and extract key frame images from the video stream to perceive and obtain the task requirements of the product to be inspected.

[0222] In step S1120, the service matches the edge resource value T m .

[0223] The detection tasks corresponding to the product to be detected and the detection task quantity of the detection tasks are obtained, and according to the detection task quantity, resource requirement calculation is performed on the detection tasks, required computing resources, required storage resources and required network resources to obtain task required resources.

[0224] The task required resources can be calculated according to formula (1).

[0225] Obtain edge computing resources, edge storage resources, and edge network resources to determine that the edge computing resources, edge storage resources, and edge network resources are edge-side resources.

[0226] Among them, the edge side resources can be determined according to formula (2).

[0227] After the edge side resources and the task required resources are obtained, a resource matching calculation may be performed on the edge side resources and the task required resources to obtain a resource matching value.

[0228] Specifically, the resource matching calculation method can be implemented according to formula (3).

[0229] In the resource matching value T m In the example, m=3. T1 is a computing resource matching value included in the resource matching value, T2 is a storage resource matching value included in the resource matching value, and T3 is a network resource matching value included in the resource matching value.

[0230] Furthermore, it can also be sent to the task scheduling module.

[0231] In step S1130, it is determined that max(T)<a.

[0232] After the resource matching value is calculated, a matching threshold corresponding to the resource matching value may be obtained.

[0233] The matching threshold is the threshold value of the edge resource when the demand of any resource reaches the edge resource, which can be set to a. The value of a can be determined according to the production line resources, and this exemplary embodiment does not specifically limit this. The matching threshold a can be used to determine whether the edge side resources meet the requirements of the production line detection task.

[0234] Furthermore, the resource matching value may be compared with a matching threshold to obtain a corresponding matching comparison result.

[0235] Specifically, the computing resource matching value, the storage resource matching value and the network resource matching value are compared to obtain a target resource matching value, and the target resource matching value is compared with a matching threshold to obtain a matching comparison result.

[0236] Since the resource matching value includes three parts: computing resource matching value, storage resource matching value and network resource matching value, the computing resource matching value, storage resource matching value and network resource matching value can be compared first to obtain the target resource matching value, and then the target resource matching value is compared with the matching threshold to obtain the matching comparison result.

[0237] The matching comparison result may be that the target resource matching value is less than the matching threshold, or may be that the target resource matching value is greater than or equal to the matching threshold.

[0238] Among them, the target resource matching value can be the largest resource matching value among the computing resource matching value, storage resource matching value and network resource matching value, and other target resource matching value determination methods can also be set according to actual conditions. This exemplary embodiment does not make any special limitations on this.

[0239] In step S1140, the adaptation task amount n is calculated. * .

[0240] When any Tm≥a, that is, when the matching comparison result is that the target resource matching value is greater than or equal to the matching threshold, it indicates that there is a shortage of resources on the edge side, and the production line tasks can be dynamically adjusted.

[0241] Specifically, the workload of the production line can be controlled, that is, the workload of inspection can be appropriately reduced to alleviate the inspection pressure on the edge side.

[0242] Then, the number of target tasks can be calculated according to formula (4) to obtain the target task number n * .

[0243] Furthermore, the number of target tasks n * Send it to the OT system to adjust the production line so that the task scheduling module can synchronously perform edge task allocation.

[0244] In step S1150, it is determined that R < N g.

[0245] When the matching comparison result is that the target resource matching value is greater than or equal to the matching threshold, it may be further determined whether the number of target tasks or the number of detection tasks is less than the number threshold.

[0246] In step S1160, the task is divided into layers h1, h2, ..., h n .

[0247] When the comparison result is that the number of target tasks or the number of detection tasks is less than the number threshold, it indicates that there are fewer tasks at this time.

[0248] Furthermore, it is also possible to determine whether the computing resources, storage resources, and network resources provided by the intelligent gateway meet the requirements of the number of target tasks or the number of detection tasks.

[0249] When the computing resources, storage resources, and network resources provided by the intelligent gateway can meet the requirements of the number of target tasks or the number of detection tasks, the defect detection model deployed on the intelligent gateway can be used to perform defect detection on the product to be inspected.

[0250] The defect detection model can be deployed on the terminal device at the edge, and a convolutional neural network can be deployed on the intelligent gateway and edge computing device at the edge. That is, the defect detection model can be a convolutional neural network model or other models, and this exemplary embodiment does not specifically limit this.

[0251] Among them, the intelligent gateway that deploys a part of the convolutional neural network model provides the first level edge detection, and the edge computing device that deploys a part of the convolutional neural network model provides the second level edge detection.

[0252] The edge computing device may be a device capable of implementing edge computing technology.

[0253] It is worth noting that the intelligent gateway must be deployed on the field side, but the edge computing device can be deployed on the field side, in the operator's computer room, or in a farther place. This exemplary embodiment does not specifically limit this.

[0254] Furthermore, it is also possible to determine whether the computing resources, storage resources, and network resources provided by the intelligent gateway meet the requirements of the number of target tasks or the number of detection tasks.

[0255] When the computing resources, storage resources, and network resources provided by the intelligent gateway can meet the requirements of the number of target tasks or the number of detection tasks, the defect detection model deployed on the intelligent gateway can be used to perform defect detection on the product to be inspected.

[0256] If the number of target tasks or the number of detection tasks is greater than or equal to the quantity threshold, the defect detection model deployed on the intelligent gateway and the edge computing device is used to perform defect detection on the product to be inspected.

[0257] After the quantity threshold is obtained, the number of target tasks or the number of detection tasks may be compared with the quantity threshold.

[0258] When the comparison result is that the number of target tasks or the number of detection tasks is greater than or equal to the number threshold, it indicates that there are many tasks at this time. Therefore, the defect detection model deployed on the smart gateway and edge computing device can be used to perform defect detection on the product to be inspected.

[0259] The defect detection model is subjected to model layering processing to obtain model layering parameters, and the model layering parameters are subjected to model parameter calculation to obtain gateway demand resources corresponding to the intelligent gateway.

[0260] When the number of target tasks or detection tasks is n, any layer in the convolutional neural network model can be divided into task segmentation layers h1, h2, …, h n .

[0261] Therefore, model layering parameters such as the height of the convolution kernel can be obtained.

[0262] In step S1170, TX, TM, and TC are calculated.

[0263] The gateway required resources of the intelligent gateway are calculated according to formula (5), formula (6) and formula (7). The gateway required resources include gateway computing resources TC, gateway storage resources TM and gateway network resources TX.

[0264] In step S1180, defect detection is performed.

[0265] A resource threshold of the intelligent gateway corresponding to the gateway demand resources is obtained, and the gateway resources are compared with the gateway demand resources to obtain a gateway comparison result, where the gateway demand resources include gateway computing resources, gateway storage resources, and gateway network resources.

[0266] Since the gateway demand resources include gateway computing resources, gateway storage resources and gateway network resources, the resource threshold of the intelligent gateway corresponding to the gateway demand resources can be obtained. The resource threshold can be based on the resource N of the intelligent gateway. g Sure.

[0267] Specifically, the resource threshold can be λN g .in, The introduction of λ can reserve computing power for one end of the smart gateway to avoid high concurrency of the smart gateway.

[0268] Furthermore, the resource threshold may be compared with any one of the gateway computing resources, the gateway storage resources, and the gateway network resources to obtain a gateway comparison result.

[0269] If the gateway comparison result is that the resource threshold is less than or equal to the gateway computing resources, gateway storage resources or gateway network resources, the defect detection model deployed on the smart gateway and edge computing device is used to perform defect detection on the product to be inspected in a layered manner.

[0270] When the gateway comparison result is that the resource threshold is less than any one of the gateway computing resources, gateway storage resources, and gateway network resources, then h1, h2, …, h n The number of network layers of the defect detection model such as convolutional neural network executed on the smart gateway side for n tasks.

[0271] For the i-th defect detection task of the product to be inspected, the smart gateway executes the deployed convolutional neural network Layer defect detection model, executed on edge computing devices Defect detection model for layers.

[0272] In step S1190, data upload and model training are optimized.

[0273] The image feature data uploaded by the edge side is used for model training and updating, and the defect detection model is compressed, pruned, quantized, and other operations are performed to obtain a more streamlined defect detection model while ensuring the accuracy of defect detection. In addition, the streamlined defect detection model can be sent to the edge side for model updating and deployment.

[0274] The product defect detection method in this application scenario introduces a resource perception module and a task scheduling module, and proposes an intelligent perception algorithm. In addition, the matching value is calculated by the production line task volume and edge resources, and the production line task volume is adjusted according to the matching value. The edge computing method is adopted to deploy the defect detection model in layers on the intelligent gateway and edge computing devices, and the model is layered according to the resources consumed by the neural network reasoning, giving full play to the near-field advantage of the intelligent gateway.

[0275] The matching value is calculated based on the computing resources, storage resources, network resources required for the production line tasks and the existing resources on the edge side, and uploaded to the task scheduling module, which improves the correlation between the production line and the edge side.

[0276] If the matching value does not meet the requirements, the production line tasks are dynamically adjusted according to the matching value until the requirements are met. If the matching value meets the requirements, defect detection models such as convolutional neural networks are deployed in layers according to the resources consumed by model reasoning, edge resources are used rationally, and network pressure is reduced.

[0277] Fig.12 A schematic diagram of the structure of a device for detecting product defects is shown in FIG. Fig.12 As shown, the product defect detection device 1200 applied to the edge side of the edge computing architecture may include: a resource matching module 1210, a resource comparison module 1220 and a product detection module 1230. Among them:

[0278] The resource matching module 1210 is configured to obtain edge-side resources and task-required resources corresponding to the product to be detected, and perform resource matching calculation on the edge-side resources and the task-required resources to obtain a resource matching value;

[0279] The resource comparison module 1220 is configured to obtain a matching threshold corresponding to the resource matching value, and compare the resource matching value with the matching threshold to obtain a matching comparison result;

[0280] The product detection module 1230 is configured to perform defect detection on the product to be detected according to the matching comparison result.

[0281] In an exemplary embodiment of the present invention, the task required resources include: required computing resources, required storage resources and required network resources.

[0282] The obtaining of edge side resources and task requirement resources corresponding to the product to be detected includes:

[0283] Acquire the detection task corresponding to the product to be detected and the detection task quantity of the detection task, and perform resource requirement calculation on the detection task, the required computing resources, the required storage resources and the required network resources according to the detection task quantity to obtain the task required resources;

[0284] The edge computing resources, the edge storage resources, and the edge network resources are obtained to determine that the edge computing resources, the edge storage resources, and the edge network resources are edge-side resources.

[0285] In an exemplary embodiment of the present invention, the resource matching value includes a computing resource matching value, a storage resource matching value and a network resource matching value.

[0286] The comparing the resource matching value with the matching threshold to obtain a matching comparison result includes:

[0287] Compare the matching threshold with the computing resource matching value, the storage resource matching value or the network resource matching value to obtain a matching comparison result; or

[0288] The computing resource matching value, the storage resource matching value and the network resource matching value are compared to obtain a target resource matching value, and the target resource matching value is compared with the matching threshold to obtain a matching comparison result.

[0289] In an exemplary embodiment of the present invention, the step of performing defect detection on the product to be detected according to the matching comparison result includes:

[0290] If the matching comparison result is that the computing resource matching value, the storage resource matching value or the network resource matching value is greater than or equal to the matching threshold, the edge side resources and the task requirement resources are calculated to obtain the target task quantity;

[0291] Defect detection is performed on the product to be inspected according to the target task quantity.

[0292] In an exemplary embodiment of the present invention, after performing parameter calculation on the edge side resources and the task requirement resources to obtain the target task quantity, the method further includes:

[0293] If the resource matching value corresponding to the target task quantity is less than the matching threshold, a defect detection model is used to perform defect detection on the product to be detected.

[0294] If the matching comparison result is that the target resource matching value is less than the matching threshold, a defect detection model is used to perform defect detection on the product to be detected.

[0295] In an exemplary embodiment of the present invention, the step of performing defect detection on the product to be detected by using a defect detection model includes:

[0296] Deploy the defect detection model on the terminal device at the edge side, and obtain a quantity threshold corresponding to the target task quantity or the detection task quantity, wherein the terminal device includes an intelligent gateway and an edge computing device;

[0297] If the target task quantity or the detection task quantity is less than the quantity threshold, performing defect detection on the product to be detected by using the defect detection model deployed on the intelligent gateway;

[0298] If the target task quantity or the detection task quantity is greater than or equal to the quantity threshold, defect detection is performed on the product to be inspected using the defect detection model deployed on the smart gateway and the edge computing device.

[0299] In an exemplary embodiment of the present invention, the method of performing defect detection on the product to be detected by using the defect detection model deployed on the intelligent gateway and the edge computing device includes:

[0300] Performing model layering processing on the defect detection model to obtain model layering parameters, and performing model parameter calculation on the model layering parameters to obtain gateway demand resources corresponding to the intelligent gateway;

[0301] Acquire a resource threshold of the intelligent gateway corresponding to the gateway demand resources, and compare the gateway resources with the gateway demand resources to obtain a gateway comparison result, wherein the gateway demand resources include gateway computing resources, gateway storage resources, and gateway network resources;

[0302] If the gateway comparison result is that the resource threshold is less than or equal to the gateway computing resources, the gateway storage resources or the gateway network resources, defect detection is performed on the product to be inspected in layers using the defect detection model deployed on the smart gateway and edge computing device.

[0303] The specific details of the above-mentioned product defect detection device 1200 have been described in detail in the corresponding product defect detection method, so they will not be repeated here.

[0304] It should be noted that, although several modules or units of the device for detecting product defects 1200 are mentioned in the above detailed description, such division is not mandatory. In fact, according to an embodiment of the present disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided into multiple modules or units to be embodied.

[0305] In addition, in an exemplary embodiment of the present disclosure, an electronic device capable of implementing the above method is also provided.

[0306] Refer to the following Fig.13 13 to 14. An electronic device 1300 according to such an embodiment of the present invention is described. Fig.13 The electronic device 1300 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0307] like Fig.13 As shown, the electronic device 1300 is in the form of a general computing device. The components of the electronic device 1300 may include, but are not limited to: at least one processing unit 1310, at least one storage unit 1320, a bus 1330 connecting different system components (including the storage unit 1320 and the processing unit 1310), and a display unit 1340.

[0308] The storage unit stores program codes, which can be executed by the processing unit 1310, so that the processing unit 1310 executes the steps according to various exemplary embodiments of the present invention described in the above “Exemplary Method” section of this specification.

[0309] The storage unit 1320 may include a readable medium in the form of a volatile storage unit, such as a random access memory unit (RAM) 1321 and / or a cache memory unit 1322 , and may further include a read-only memory unit (ROM) 1323 .

[0310] The storage unit 1320 may also include a program / utility 1324 having a set (at least one) of program modules 1325, such program modules 1325 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0311] Bus 1330 may represent one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

[0312] The electronic device 1300 may also communicate with one or more external devices 1500 (e.g., keyboards, pointing devices, Bluetooth devices, etc.), may also communicate with one or more devices that enable a user to interact with the electronic device 1300, and / or communicate with any device that enables the electronic device 1300 to communicate with one or more other computing devices (e.g., routers, modems, etc.). Such communication may be performed via an input / output (I / O) interface 1350. Furthermore, the electronic device 1300 may also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) via a network adapter 1360. As shown, the network adapter 1360 communicates with other modules of the electronic device 1300 via a bus 1330. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 1300, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0313] Through the description of the above embodiments, it is easy for those skilled in the art to understand that the example embodiments described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the embodiment of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to the embodiment of the present disclosure.

[0314] In an exemplary embodiment of the present disclosure, a computer-readable storage medium is also provided, on which a program product capable of implementing the above method of the present specification is stored. In some possible embodiments, various aspects of the present invention can also be implemented in the form of a program product, which includes a program code, and when the program product is run on a terminal device, the program code is used to enable the terminal device to perform the steps according to various exemplary embodiments of the present invention described in the above "Exemplary Method" section of the present specification.

[0315] refer to Fig.14 As shown, a program product 1400 for implementing the above method according to an embodiment of the present invention is described, which can adopt a portable compact disk read-only memory (CD-ROM) and include program code, and can be run on a terminal device, such as a personal computer. However, the program product of the present invention is not limited thereto. In this document, a readable storage medium can be any tangible medium containing or storing a program, which can be used by or in combination with an instruction execution system, an apparatus or a device.

[0316] The program product may use any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0317] Computer readable signal media may include data signals propagated in baseband or as part of a carrier wave, in which readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Readable signal media may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0318] The program code embodied on the readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the foregoing.

[0319] Program code for performing the operations of the present invention may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., through the Internet using an Internet service provider).

[0320] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary technical means in the art that are not disclosed in the present disclosure. The specification and examples are to be considered as exemplary only, and the true scope and spirit of the present disclosure are indicated by the claims.

Claims

1. A method for detecting product defects, characterized in that: Applied to the edge side of the edge computing architecture, the method includes: Acquire edge-side resources and task-required resources corresponding to the product to be detected, and perform resource matching calculation on the edge-side resources and the task-required resources to obtain resource matching values; wherein the resource matching values ​​include computing resource matching values, storage resource matching values, and network resource matching values; Acquire a matching threshold value corresponding to the resource matching value, and compare the resource matching value with the matching threshold value to obtain a matching comparison result; Perform defect detection on the product to be detected according to the matching comparison result; Wherein, performing defect detection on the product to be detected according to the matching comparison result includes: If the matching comparison result is that the computing resource matching value, the storage resource matching value or the network resource matching value is greater than or equal to the matching threshold, or if the matching comparison result is that the target resource matching value is greater than or equal to the matching threshold, the edge side resources and the task requirement resources are calculated to obtain the target task quantity; Perform defect detection on the product to be inspected according to the target task quantity; The target resource matching value is obtained by comparing the computing resource matching value, the storage resource matching value and the network resource matching value.

2. The method for detecting product defects according to claim 1, characterized in that: The task required resources include: required computing resources, required storage resources and required network resources. The obtaining of edge side resources and task requirement resources corresponding to the product to be detected includes: Acquire the detection task corresponding to the product to be detected and the detection task quantity of the detection task, and perform resource requirement calculation on the detection task, the required computing resources, the required storage resources and the required network resources according to the detection task quantity to obtain the task requirement resources; Acquire edge computing resources, edge storage resources, and edge network resources to determine that the edge computing resources, the edge storage resources, and the edge network resources are edge-side resources.

3. The method for detecting product defects according to claim 2, characterized in that: After the edge side resources and the task requirement resources are calculated to obtain the target task quantity, the method further includes: If the resource matching value corresponding to the target task quantity is less than the matching threshold, using the defect detection model to perform defect detection on the product to be detected; If the matching comparison result is that the target resource matching value is less than the matching threshold, a defect detection model is used to perform defect detection on the product to be detected.

4. The method for detecting product defects according to claim 3, characterized in that: The method of performing defect detection on the product to be detected by using the defect detection model includes: Deploy the defect detection model on the terminal device at the edge side, and obtain a quantity threshold corresponding to the target task quantity or the detection task quantity, wherein the terminal device includes an intelligent gateway and an edge computing device; If the target task quantity or the detection task quantity is less than the quantity threshold, performing defect detection on the product to be detected by using the defect detection model deployed on the intelligent gateway; If the target task quantity or the detection task quantity is greater than or equal to the quantity threshold, defect detection is performed on the product to be inspected using the defect detection model deployed on the intelligent gateway and the edge computing device.

5. The method for detecting product defects according to claim 4, characterized in that: The method of performing defect detection on the product to be detected by using the defect detection model deployed on the intelligent gateway and the edge computing device includes: Performing model layering processing on the defect detection model to obtain model layering parameters, and performing model parameter calculation on the model layering parameters to obtain gateway demand resources corresponding to the intelligent gateway; Acquire a resource threshold of the intelligent gateway corresponding to the gateway demand resources, and compare the resource threshold with the gateway demand resources to obtain a gateway comparison result, wherein the gateway demand resources include gateway computing resources, gateway storage resources, and gateway network resources; If the gateway comparison result is that the resource threshold is less than or equal to the gateway computing resources, the gateway storage resources or the gateway network resources, defect detection is performed on the product to be inspected in layers using the defect detection model deployed on the smart gateway and edge computing device.

6. A device for detecting product defects, characterized in that: Applied to the edge side of edge computing architecture, including: A resource matching module is configured to obtain edge-side resources and task-required resources corresponding to the product to be detected, and perform resource matching calculation on the edge-side resources and the task-required resources to obtain a resource matching value; wherein the resource matching value includes a computing resource matching value, a storage resource matching value, and a network resource matching value; A resource comparison module is configured to obtain a matching threshold corresponding to the resource matching value, and compare the resource matching value with the matching threshold to obtain a matching comparison result; A product detection module, configured to perform defect detection on the product to be detected according to the matching comparison result; Wherein, the product detection module is configured as follows: If the matching comparison result is that the computing resource matching value, the storage resource matching value or the network resource matching value is greater than or equal to the matching threshold, or if the matching comparison result is that the target resource matching value is greater than or equal to the matching threshold, the edge side resources and the task requirement resources are calculated to obtain the target task quantity; Perform defect detection on the product to be inspected according to the target task quantity; The target resource matching value is obtained by comparing the computing resource matching value, the storage resource matching value and the network resource matching value.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for detecting product defects described in any one of claims 1 to 5 is implemented.

8. An electronic device, characterized in that: include: processor; A memory, configured to store executable instructions of the processor; The processor is configured to execute the product defect detection method according to any one of claims 1 to 5 by executing the executable instructions.

Images