Product detection method, medium, equipment and system based on production line centralized re-judgment

Abstract

The invention provides a product detection method based on production line centralized re-judgment, a medium, equipment and a system. The method comprises the following steps: acquiring an imaging picture of a product; identifying a defect condition in the imaging picture; determining the process flow direction of a product corresponding to the imaging picture according to the defect condition of the imaging picture; responding to the product qualification, and enabling the product to flow to a qualification process; in response to the fact that the product is unqualified, the imaging picture corresponding to the product is made to flow to a centralized re-judgment process; the centralized re-judgment process is balanced and optimized according to system safety and efficiency improvement; and determining the final process flow direction of the unqualified product according to the concentrated re-judgment result. The flow direction of the final process of the product can be adjusted by combining the conditions of qualified and unqualified products, the labor cost problem of a production line is solved, and the production efficiency is improved.

Description

Technical Field

[0001] The present application belongs to the technical field of product production inspection, and relates to a defect detection method, and in particular to a product inspection method, medium, equipment and system based on centralized re-judgment of a production line. Background Art

[0002] At present, many defects will occur in the process of product manufacturing, which seriously affect product quality. Therefore, factories will arrange detection equipment in various links of assembly line production to identify and control defects in product pictures.

[0003] There are two existing methods for identifying defects in images: one is that manual quality inspectors completely rely on human eyes to identify defects in each image. In the production and manufacturing process of products, enterprises generally have a large number of assembly lines, and each assembly line has many processes that need to be inspected. This will have two impacts on the enterprise. On the one hand, the enterprise needs to hire a large number of quality inspection workers, which has high labor costs; on the other hand, defect identification is purely dependent on manual labor, which will lead to high defect omissions and misjudgments. The product quality of each production line will also be difficult to improve due to the uneven proficiency of people; the second is to use visual inspection algorithms to assist manual inspection. Although visual inspection algorithms can help manual quality inspectors find some defects that are difficult to find, in some application scenarios, the accuracy of visual algorithms cannot reach the level of fully automatic identification, and each image still needs to be manually confirmed twice. In this scenario, the algorithm first identifies defects in each image once, and each production line is then reconfirmed by manual quality inspectors. Although this method reduces the difficulty of manual defect identification, each production line still needs to arrange personnel for reconfirmation. For enterprises, labor costs have not been reduced and they still need to bear high labor costs. Summary of the invention

[0004] The present application provides a product inspection method, medium, equipment and system based on centralized re-inspection of a production line, which is used to solve the problem of how to adjust the final process flow of products on the production line through flexible centralized re-inspection according to the degree of product qualification.

[0005] In a first aspect, the present application provides a product inspection method based on centralized re-judgment of a production line, the method comprising: obtaining an image of a product; identifying defect conditions in the image; the defect conditions comprising defect type and defect degree; determining the process flow of a product corresponding to the image according to the defect conditions of the image; in response to the product being qualified, directing the product to a qualified process; in response to the product being unqualified, directing the image corresponding to the product to a centralized re-judgment process; balancing and optimizing the centralized re-judgment process based on system safety and efficiency improvement; determining the final process flow of unqualified products according to the result of the centralized re-judgment; in response to the result of the centralized re-judgment being qualified, directing the unqualified products to a qualified process; in response to the result of the centralized re-judgment being unqualified, directing the unqualified products to a rework process or to offline.

[0006] In an implementation of the first aspect, the step of directing the imaging image corresponding to the product to flow to the centralized re-inspection process in response to the product being unqualified includes: in response to the product being unqualified, sending the imaging image of the unqualified product to the centralized re-inspection equipment of the centralized re-inspection station, so that the centralized re-inspection equipment displays the imaging image of the unqualified product to the quality inspector at the centralized re-inspection station, and obtains the centralized re-inspection result generated on the centralized re-inspection equipment after the quality inspector's re-inspection operation.

[0007] In an implementation of the first aspect, the method further includes determining the number of centralized re-judgment devices and the number of personnel saved in the centralized re-judgment process: determining the number of centralized re-judgment devices according to the number of all inspection equipment on the production line, the number of images of each inspection equipment, the product yield of machine vision inspection, the accuracy of visual inspection, the production and manufacturing standards, the maximum number of quality inspectors for quality inspectors, the required production capacity, and the number of people saved; wherein the number of centralized re-judgment devices ranges from 0 to the number of all inspection equipment; after starting the centralized re-judgment process, the number of quality inspectors required for m inspection equipment is changed to n quality inspectors retained, that is, the number of manpower saved is mn, and the cost reduction is

[0008] In an implementation of the first aspect, the steps of balancing and optimizing the centralized re-judgment process according to system security include: setting a master-slave redundancy mechanism, and deploying an additional set of centralized re-judgment equipment next to each centralized re-judgment station, one master and one slave, to prevent the host from crashing or the production line inspection equipment from being disconnected from the host due to a network failure, and the inspection equipment fails to obtain a response result from the host for many times, in which case the slave is enabled for centralized re-judgment; setting a timeout default processing mechanism, in response to an abnormality in the centralized re-judgment equipment receiving the imaging image, the default is to use unqualified as the re-judgment result of the centralized re-judgment process.

[0009] In an implementation of the first aspect, the step of identifying defect conditions in the imaging image includes: providing a computing power scheduling and management mechanism for the computing power of visual inspection and recognition, grouping and managing the computing power clusters according to the needs of product and model adaptation, and recording the idle status and belonging group of all computing power in real time when performing visual inspection and recognition. When an abnormality occurs in the computing power used or there is a new task, a query will be made in the adaptable computing power and then scheduling will be performed.

[0010] In an implementation of the first aspect, the steps of balancing and optimizing the centralized re-judgment process based on efficiency improvement include: combining with the manual dynamic intervention mechanism, determining the intervention results of the imaging pictures corresponding to the product, and using the intervention results as the re-judgment results of the centralized re-judgment process; in response to the fact that the centralized re-judgment quality inspectors do not have time to re-judgment the received imaging pictures, using a cache queue and a free selection mechanism to establish a cache queue in the order of receipt for all imaging pictures to be judged and the defect conditions of visual inspection, so that all imaging pictures that have not been inspected in time are queued in turn for centralized re-judgment; in response to the existence of multiple inspection surfaces for the same product, multiple imaging pictures are formed, and the multiple imaging pictures are independently processed separately during visual inspection and combined and displayed during re-judgment through a multi-image processing and matching mechanism.

[0011] In an implementation of the first aspect, the method further includes: utilizing a load balancing mechanism to intelligently dispatch and manage the imaging images in the centralized re-evaluation process; the load balancing mechanism refers to automatically balancing the workload among multiple centralized re-evaluation devices, or in response to the temporary absence of some centralized re-evaluation quality inspectors, managing and scheduling the centralized re-evaluation devices, and balancing the workload among multiple centralized re-evaluation quality inspectors.

[0012] In a second aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which implements the described method when the computer program is executed by a detection device.

[0013] In a third aspect, the present application provides a detection device, comprising: a processor and a memory; the memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory, so that the detection device performs the described method.

[0014] In a fourth aspect, the present application provides a detection system, the system comprising: an imaging device, the detection equipment and a centralized re-judgment device; the detection equipment is respectively connected to the imaging device and the centralized re-judgment device in communication; the imaging device collects an imaging picture of the product; the detection equipment obtains the imaging picture of the product from the imaging device; identifies the defect condition in the imaging picture; the defect condition includes the defect type and the defect degree; determines the process flow of the product corresponding to the imaging picture according to the defect condition of the imaging picture; in response to the product being qualified, the product is directed to the qualified process; in response to the product being unqualified, the imaging picture corresponding to the product is directed to the centralized re-judgment process of the centralized re-judgment device; the detection equipment determines the final process flow of the unqualified product according to the result of the centralized re-judgment fed back by the centralized re-judgment device; in response to the product being unqualified, the detection equipment sends the imaging picture of the unqualified product to the centralized re-judgment device at the centralized re-judgment station, the centralized re-judgment device displays the imaging picture of the unqualified product to the quality inspector at the centralized re-judgment station, and obtains the result of the centralized re-judgment generated by the quality inspector on the display interface, and the centralized re-judgment device feeds back the result of the centralized re-judgment to the detection equipment.

[0015] As described above, the product inspection method, medium, device and system based on centralized re-judgment of production lines described in this application have the following beneficial effects:

[0016] This application provides a centralized re-judgment (one person, multiple machines) method for the application scenario of image defect recognition in multiple production lines or multiple workstations, which can greatly reduce the number of quality inspection personnel on the production line and significantly reduce the labor cost of the production line of the manufacturing enterprise. At the same time, combining centralized re-judgment with AI (Artificial Intelligence) image recognition algorithm can greatly reduce the missed judgment rate and false judgment rate caused by inaccurate image defect recognition. Furthermore, by reducing the missed judgment rate, the product yield is significantly improved; by reducing the false judgment rate, the production efficiency is improved.

[0017] In addition, the present application can flexibly deploy and switch the centralized re-inspection process. It is realized that in multiple production lines, there can be one or more centralized re-inspection equipment. Under the original one-to-one manual review plan, the number of quality inspectors is reduced to one person corresponding to multiple machines. Multiple machines means multiple inspection equipment machines in all inspection stations or all inspection equipment machines in all inspection stations.

[0018] This application is for the entire system. If any unit fails or the network connection between units is interrupted, it will affect the production line. In order to ensure safe production and improve the security of the system, this application provides master-slave redundancy mechanism, timeout default processing mechanism, computing power scheduling and other exception handling mechanisms.

[0019] This application takes comprehensive consideration and balance between improving system efficiency, detection efficiency, production efficiency and meeting the needs of special enterprise scenarios, and provides computing power scheduling and management mechanisms, manual dynamic intervention mechanisms, cache queues and free selection mechanisms, multi-image processing and matching mechanisms. In addition, it also provides a load balancing mechanism that can intelligently dispatch and manage the imaging images of the centralized re-judgment process, and realize automatic balancing and distribution of workload among multiple centralized re-judgment equipment and multiple centralized re-judgment quality inspectors, or deal with the situation where some centralized re-judgment quality inspectors are temporarily away from their posts. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 Shown is a schematic diagram of an application scenario of the product inspection method based on centralized re-judgment of production lines described in an embodiment of the present application.

[0021] Figure 2 Shown is a principle flow chart of the product inspection method based on centralized re-judgment of the production line described in an embodiment of the present application.

[0022] Figure 3 Shown is a defect identification schematic diagram of a product inspection method based on centralized re-judgment of a production line as described in an embodiment of the present application.

[0023] Figure 4 Shown is a detection schematic diagram of the product detection method based on centralized re-judgment of the production line described in an embodiment of the present application.

[0024] Figure 5 Shown is a detection flow chart of the product detection method based on centralized re-judgment of the production line described in an embodiment of the present application.

[0025] Figure 6 Shown is a schematic diagram of the workstation deployment of the product inspection method based on centralized re-judgment of the production line described in an embodiment of the present application.

[0026] Figure 7 Shown is a schematic diagram of the default timeout processing of the product detection method based on centralized re-judgment of the production line described in an embodiment of the present application.

[0027] Figure 8 Shown is a schematic diagram of computing power scheduling and management of a product inspection method based on centralized re-judgment of production lines as described in an embodiment of the present application.

[0028] Fig. 9 Shown is a schematic diagram of a cache queue and free selection mechanism, and a multi-image independent processing and matching mechanism of a product inspection method based on centralized re-judgment of a production line as described in an embodiment of the present application.

[0029] Fig.10 Shown is a schematic diagram of the structural connection of the detection device described in an embodiment of the present application.

[0030] Fig.11 Shown is a schematic diagram of the structure of the detection system described in an embodiment of the present application.

[0031] Component number description

[0032] 1. Testing equipment

[0033] 11 Processor

[0034] 12 Memory

[0035] 13 Communication Interface

[0036] 14 System Bus

[0037] 2 Imaging device

[0038] 3. Centralized review equipment

[0039] Steps S21 to S25 DETAILED DESCRIPTION

[0040] The following describes the embodiments of the present application through specific examples, and those skilled in the art can easily understand other advantages and effects of the present application from the contents disclosed in this specification. The present application can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present application. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict.

[0041] It should be noted that the illustrations provided in the following embodiments are only used to illustrate the basic concept of the present application in a schematic manner, and therefore the illustrations only show components related to the present application rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed arbitrarily, and the component layout may also be more complicated.

[0042] The following embodiments of the present application provide product inspection methods, media, equipment and systems based on centralized re-inspection of production lines, including but not limited to application scenarios of image defect recognition applied to multiple production lines or multiple workstations. This application scenario will be described below as an example.

[0043] like Figure 1As shown, this embodiment shows an application scenario of image defect recognition at an inspection station of a production line. The produced products are conveyed by a conveyor belt as objects to be inspected. When the conveyor belt is transported to the imaging position of a camera installed at the inspection station, they become inspection objects. The camera transmits the image of the inspection object to the inspection equipment of the inspection station. The algorithm of the inspection equipment confirms whether the inspection object meets the requirements. If it meets the requirements, it enters the corresponding qualified process. If it does not meet the requirements, it enters the corresponding process of centralized re-judgment of the product inspection method based on centralized re-judgment of the production line described in this application, and the quality inspector at the centralized re-judgment station conducts a secondary review. Figure 1 The example provides a scenario of a production and inspection process at one inspection station. In actual applications, images of unqualified components from multiple production lines or multiple stations can be collected at the same re-inspection station for re-inspection. The product can be a photovoltaic module or other product suitable for defect detection through images. This application takes photovoltaic modules as an example, but is not limited to photovoltaic modules or the photovoltaic industry.

[0044] The technical solutions in the embodiments of the present application will be described in detail below in conjunction with the drawings in the embodiments of the present application.

[0045] See also Figure 2 , which is a flow chart showing the principle of the product detection method based on the centralized re-judgment of the production line according to the embodiment of the present application. Figure 2 As shown, this embodiment provides a product detection method based on centralized re-judgment of a production line, and the method specifically includes the following steps:

[0046] S21, obtaining an image of the product.

[0047] Specifically, the product to be inspected is photographed by a camera or other imaging device to obtain an image of the product.

[0048] S22, identifying defect conditions in the image; the defect conditions include defect type and defect degree.

[0049] Specifically, the recognition algorithm for the defect condition in the imaging picture can adopt a commonly used AI vision algorithm, or a combination of a traditional machine vision algorithm and an AI vision algorithm. When using the AI ​​vision algorithm, each image can be recognized once, which can reduce the defect missed detection rate and false detection rate.

[0050] See also Figure 3 , which is a schematic diagram of defect recognition of a product inspection method based on centralized re-judgment of a production line according to an embodiment of the present application. Figure 3 As shown, the inspection device first images the objects to be inspected at the upper left and lower left, and then identifies the defects in the image. Figure 3If the rectangular box in the middle is a defect, the object to be inspected on the upper left contains the rectangular box, does not meet the quality standards, and is judged as NG (defective product). The NG inspection object will flow to the corresponding re-inspection process; the object to be inspected on the lower left does not contain the rectangular box, meets the quality standards, and is judged as OK (good product). The OK inspection object will flow directly to the next production process.

[0051] In one embodiment, the product includes a photovoltaic module; the step of identifying a defect condition in the image includes:

[0052] (1) Identify the defect type in the imaging image; the defect type includes cold solder joints, hidden cracks, broken grids, broken pieces and foreign matter.

[0053] In practical applications, the defect types may also include the following defects that need to be controlled due to various production factors:

[0054] If the defect is wafer splicing, it may directly lead to product scrapping and downgrading, and needs to be controlled; if the defect is short circuit or serious hidden cracks, it may affect the power generation efficiency and needs to be controlled; if the defect is dirt on the component surface or residual glue, it will affect the appearance of the product and will also be controlled; if the defect is foreign matter, assuming it does not affect the current process, it may also have a serious impact on the next process, and also needs to be controlled in the current process.

[0055] For photovoltaic modules, on the one hand, the raw material is silicon wafers, which are very brittle and fragile, which leads to a lot of defects in the production process of photovoltaic modules. On the other hand, the overall size of photovoltaic modules is relatively large, about 2.2 meters * 1.2 meters, the actual object is large, and the field of view of the production picture is also large. On such a large product, it is difficult to find the defects that need to be controlled in a short time. One reason is that there are many defects, and another reason is that some defects are relatively small, such as small foreign matter, small hidden cracks, small dirt, and the length may be only a few millimeters (about only 0.000005 of the module area).

[0056] (2) Based on the defect type, determine the quality standard control condition corresponding to the defect degree. Specifically, taking photovoltaic modules as an example, the quality standard or quality control standard of module manufacturing enterprises is relatively complex, and there will be different control standards for each defect, and there will also be different quality standards for different customer orders.

[0057] In one embodiment, the defect type is only cold solder joint; the step of determining the quality standard control condition corresponding to the degree of the defect based on the defect type includes:

[0058] a1. In response to the cold solder joint area being smaller than the first cold solder joint ratio, the defect is ignored.

[0059] Specifically, the first cold solder joint ratio is 2% for example, and the cold solder joint area is less than 2%, and the defect is ignored.

[0060] a2. In response to the cold solder joint area being greater than the first cold solder joint ratio and the cold solder joint area being less than the second cold solder joint ratio, 5 defects are allowed to occur for one photovoltaic module.

[0061] Specifically, the second cold solder joint ratio is 5% for example, the cold solder joint area is greater than 2%, and the cold solder joint area is less than 5%, and 5 defects are allowed to occur in one photovoltaic module.

[0062] a3. In response to the cold solder joint area being greater than the second cold solder joint ratio, the photovoltaic module is determined to be unqualified.

[0063] Specifically, if the cold solder joint area is greater than 5%, the photovoltaic module is judged to be unqualified.

[0064] a4. In response to the cold solder joint length being greater than the third cold solder joint ratio, the photovoltaic module is determined to be unqualified.

[0065] Specifically, the third cold solder joint ratio is 10% for example, and the cold solder joint length is greater than 10%, and the photovoltaic module is determined to be unqualified.

[0066] In another embodiment, the defect type is only hidden crack; the step of determining the quality standard control condition corresponding to the defect degree based on the defect type includes:

[0067] b1. In response to the linear crack length being less than the first crack ratio, the defect is ignored.

[0068] Specifically, the first hidden crack ratio is exemplified as 2%, and the length of the linear hidden crack is less than 2%, and the defect is ignored.

[0069] b2. In response to the linear crack length being greater than the first crack ratio and the linear crack length being less than the second crack ratio, three defects are allowed to occur in one photovoltaic module.

[0070] Specifically, the second hidden crack ratio is 5% for example, the linear hidden crack length is greater than 2%, and the linear hidden crack length is less than 5%, and 3 defects are allowed to appear in a photovoltaic module.

[0071] b3. In response to the linear crack length being greater than the second crack ratio, the photovoltaic module is determined to be unqualified.

[0072] Specifically, if the length of the linear crack is greater than 5%, the photovoltaic module is judged to be unqualified.

[0073] In actual applications, a photovoltaic module may have more than one defect, but a combination of multiple defects. In this case, it is necessary to determine the type and degree of each defect and whether it meets the corresponding quality standard control conditions.

[0074] (3) In response to the defect degree of the photovoltaic module satisfying the quality standard control condition, the photovoltaic module is determined to be qualified; in response to the defect degree of the photovoltaic module not satisfying the quality standard control condition, the photovoltaic module is determined to be unqualified.

[0075] S23, determining the process flow of the product corresponding to the image according to the defect status of the image.

[0076] Specifically, the defect conditions of the imaging image include defect type and defect degree; the defect type includes cold solder joint, hidden crack, broken grid, broken pieces, foreign matter, parallel pieces, short circuit, component surface dirt and residual glue. The defect degree is determined according to the different defect proportion ranges in which the defect is located.

[0077] S24, in response to the product being qualified, the product is directed to the qualified process; in response to the product being unqualified, the imaging pictures corresponding to the product are directed to the centralized re-judgment process. Specifically, the centralized re-judgment process optimizes and balances between improved system security, improved efficiency, and special needs. Thus, the pictures of qualified products do not need to be manually reconfirmed, and the pictures of unqualified products are manually re-judged and confirmed, which can significantly reduce the number of pictures that require manual re-judgment. Furthermore, through centralized re-judgment, the pictures of products that are initially judged to be unqualified from different production lines or different inspection stations are concentrated at one station for manual centralized re-judgment, which can save a lot of labor costs.

[0078] In one embodiment, in response to the product being unqualified, the step of sending the imaged image corresponding to the product to a centralized re-judgment process includes:

[0079] In response to the product being unqualified, an image of the unqualified product is sent to a centralized re-inspection device at a centralized re-inspection station, so that the centralized re-inspection device displays the image of the unqualified product to the quality inspector at the centralized re-inspection station, and obtains the centralized re-inspection result generated on the centralized re-inspection device after the quality inspector's re-inspection operation.

[0080] S25, based on the results of centralized re-evaluation, determine the final process flow of unqualified products.

[0081] Specifically, the step of determining the final process flow of unqualified products based on the results of centralized re-judgment includes: in response to the result of centralized re-judgment being qualified, the unqualified products are flowed to the qualified process; in response to the result of centralized re-judgment being unqualified, the unqualified products are flowed to the rework process or offline.

[0082] See also Figure 4 and Figure 5, respectively showing a detection schematic diagram of a product detection method based on centralized re-judgment of a production line according to an embodiment of the present application and a detection flow chart of a product detection method based on centralized re-judgment of a production line according to an embodiment of the present application. Figure 4 and Figure 5 As shown, the factory has multiple production lines or multiple inspection stations (inspection station 1, inspection station 2...inspection station n), each production line or each inspection station has an object to be inspected, and also has corresponding inspection equipment. The camera software or the camera module of the software in the inspection equipment takes pictures of the object to be inspected entering the inspection area to generate a picture of the object to be inspected. The defects in the picture are identified by the AI ​​visual algorithm. For multiple production lines or multiple inspection stations, the AI ​​algorithm first identifies the inspection pictures of each production line. The AI ​​algorithm detects the pictures that meet the requirements. The algorithm will directly return the results to the inspection equipment software of the inspection station, and then control the inspection object to flow to the qualified process. The AI ​​algorithm detects the pictures that do not meet the requirements. The inspection system aggregates the unqualified object pictures of each line to a centralized re-judgment station, and conducts centralized re-judgment manually, and returns the results of the manual re-judgment (OK / NG) to the software of the inspection equipment on the inspection station, and then controls the inspection object to enter the corresponding process. Therefore, multiple production lines or multiple stations only need to be equipped with one quality inspector to complete the picture recognition work.

[0083] In one embodiment, the method further includes determining the number of centralized re-judgment equipment and the number of personnel saved in the centralized re-judgment process: determining the number of centralized re-judgment equipment according to the number of all inspection equipment on the production line, the number of images of each inspection equipment, the product yield of machine vision inspection, the accuracy of visual inspection, the production and manufacturing standards, the maximum number of quality inspectors, the required production capacity and the number of people saved; wherein the number of centralized re-judgment equipment ranges from 0 to the number of all inspection equipment; after starting the centralized re-judgment process, the number of quality inspectors required for m inspection equipment is changed to n quality inspectors retained, that is, the number of manpower saved is mn, and the cost reduction is

[0084] Since the centralized retrial of this application is a modular software program, it can be flexibly deployed and switched to meet different centralized retrial requirements. Figure 6 The centralized re-judgment equipment can be directly deployed in the computer of the production line inspection equipment, or it can be uniformly deployed in a centralized control room far away from the production site (and the inspection equipment on the production line transmits data through the network). The two deployment methods each have their own advantages and disadvantages. If deployed at the production site, the online quality inspectors can observe the abnormal situation of the production line at any time and deal with it in time during the centralized re-judgment. If deployed in a centralized control room, all the quality inspectors who conduct centralized re-judgment can work together to deal with some other situations, which is also convenient for management.

[0085] In multiple production lines, there can be one or more centralized re-inspection devices. For example, in multiple production lines, suppose there are m inspection devices, and each device requires one quality inspector, then the number of quality inspectors is also m, and the number of centralized re-inspection devices is k. Since the main function of the centralized re-inspection equipment is to save quality inspectors on the inspection equipment and reduce the company's labor costs, then the value range of k is usually: 0≤k≤m. In the actual promotion of the visual inspection system labor-saving project, according to the company's labor-saving needs and the progress of the project, k will have different values, and the labor costs brought to the company will also be different. Usually, the number of centralized re-inspections is generally equal to the number of personnel that the company finally retains on the production line, that is, if the number of quality inspectors changes from m to n, then k=n, that is:

[0086]

[0087] The factors that generally affect n include: the number of all inspection equipment on the production line, the number of images of each equipment, product yield, manufacturing standards, the maximum number of quality inspectors, the accuracy of the visual inspection system, the required production capacity, and the number of people saved. Assume that the number of all production line inspection equipment is m equip , the defective product ratio is q, then at the same time, the maximum number of defective products is m equip *q, the more equipment there is on the production line, the more pictures to be re-judged, the higher the defective product ratio, the more pictures to be re-judged. Assume that the number of pictures for each device is m pic , the total number of defective product images of all production line equipment is m equip *q*m pic , assuming that the defective product coefficient after the visual inspection module (including the processing of quality sorting to production standards) is t, and the difference in yield and visual inspection results of different images is not considered, then at the same time, the number of defective images after the visual inspection system is m equip *q*m pic *t, t represents the comprehensive level of visual inspection accuracy and quality sorting accuracy. If missed judgment is 0 and false judgment is 0, then the visual inspection system detects all truly defective products. At this time, t = 1. However, in fact, due to missed judgments in the visual inspection system, the number of defective images is small, and false judgments increase the number of defective images. Generally, false judgments are more than missed judgments. Therefore, in fact, t>1; when the maximum number of quality inspectors is m humanpic , then the number of retained quality inspectors Of course, n is also affected by production capacity demand. When an enterprise wants to produce more products, it needs a faster production line, which means that each image needs to wait for a shorter time for centralized review. Therefore, by increasing the number of quality inspectors n, the number of quality inspections for each quality inspector m can be reduced. humanpic , to speed up the centralized re-judgment and improve production capacity.

[0088] In the process of launching the visual inspection system labor-saving project, manufacturers usually verify the capabilities of the visual inspection system first and track the product quality after the system is launched. They often do not directly save quality inspectors to prevent production or quality accidents. Generally, they adopt a step-by-step approach to gradually save manpower. Usually, as long as the results of visual algorithm detection are better than those of manual labor, labor-saving can be gradually enabled. For example, in the early stage of visual inspection system deployment, the number of quality inspectors remains unchanged and is still m. Since there is no need to save manpower, the centralized re-judgment function is not required at this time, so k can be 0. But in fact, in the early stage of deployment, the centralized re-judgment device software program can be deployed on each detection device computer, and then the centralized re-judgment function can be selected according to the needs. At this time, from the deployment perspective, k=m, but the centralized re-judgment function is not turned on. This has two advantages: 1. In this application, combined with the defect detection function, although the centralized re-judgment device does not have the centralized re-judgment function at this time, it can be converted into a defect auxiliary identification module, and the results of defect visual detection (usually defect type and location) are presented to the quality inspector in a striking manner, providing a reference for the quality inspector's online real-time quality inspection, reducing the workload of the quality inspector or reducing the missed judgment of defects; 2. After each detection device is deployed with a centralized re-judgment device, the workstation (quality inspector) of any detection device can be used as the workstation (quality inspector) of the centralized re-judgment. , there is no need to equip a special workstation (quality inspector) for centralized re-judgment. For example, there are 10 production lines, 10 inspection equipment, and 10 inspection workstations for 10 quality inspectors. The enterprise can install software programs such as centralized re-judgment on the computers of these 10 devices. When centralized re-judgment is not needed, the centralized re-judgment function is not turned on. When the enterprise needs 1 centralized re-judgment workstation and turns on centralized re-judgment, it can select any one of the 10 workstations to turn on the centralized re-judgment function. The quality inspector of this workstation remains unchanged, and the quality inspectors of other workstations can be omitted. All the pictures of the 10 inspection equipment after visual inspection are sent to this selected workstation. In this way, in actual use, according to the needs of the production line, you can choose to switch to the centralized re-judgment point at any inspection equipment, without being restricted by the fixed production site and personnel. You can select 1 or more of the m centralized re-judgment equipment to turn on the centralized re-judgment function as the centralized re-judgment point.

[0089] As enterprises recognize the data of visual inspection systems and promote the labor-saving project, they will gradually start the centralized re-judgment function. Assuming that m inspection equipment previously required m quality inspectors, the enterprise decided to retain only n quality inspectors, that is, the number of manpower saved is mn, and the cost reduction is Then, the quality inspectors at these n centralized re-judgment stations need to complete the judgment of all m inspection equipment images, and n centralized re-judgment equipment needs to be activated. For example, in a manufacturing enterprise, there are 10 production lines, each production line is equipped with 1 inspection equipment, and each inspection equipment needs to be equipped with 1 quality inspector, so a total of 10 quality inspectors are needed. The enterprise decides to save 80% of the manpower and only retains 2 quality inspectors. Correspondingly, 2 centralized re-judgment equipment needs to be activated to complete the re-judgment of all images of the 10 production lines. According to the needs of the enterprise, the workload of the 10 production lines can be divided equally by 2 quality inspectors in a 5+5 manner, or it can be allocated in a 4+6 or other manner. Later, when only 1 centralized re-judgment quality inspector is needed, only 1 centralized re-judgment system needs to be activated. At this time, the ratio of manpower saved reaches 90%. When the product yield, production and manufacturing standards, visual inspection system accuracy and demand capacity reach a balance point, the enterprise may no longer need centralized re-judgment quality inspectors. At this time, the number of centralized re-judgments activated is 0, and the ratio of manpower saved reaches 100%, achieving unmanned automation. This application adopts a modular centralized re-judgment program that can be deployed at multiple points. The re-judgment points can be switched as needed, or a centralized re-judgment point can be opened or closed as needed, which can meet the different manpower-saving needs of enterprises at different times.

[0090] Since the inspection equipment, visual inspection modules, centralized review equipment and quality inspectors constitute an overall system, and the various hardware and software are linked through the network or other means to send images and data, human-computer interaction is also required between the centralized review equipment and the quality inspectors. Therefore, in the entire system, if any unit fails or the network connection between units is interrupted, it will affect the production line. At the same time, since the quality inspectors have some operational requirements for the system, these requirements may conflict with improving the inspection efficiency and system efficiency. In order to ensure safe production, comprehensive consideration and balance are made between improving system efficiency, inspection efficiency, production efficiency and meeting personnel operation needs. This application provides master-slave redundancy mechanism, timeout default processing mechanism, computing power scheduling and management mechanism, manual dynamic intervention mechanism, cache queue and free selection mechanism, as well as multi-image processing and matching mechanism and other exception handling mechanisms.

[0091] In one embodiment, the step of balancing and optimizing the centralized re-judgment process according to system security includes: setting a master-slave redundancy mechanism and a timeout default processing mechanism.

[0092] The master-slave redundancy mechanism refers to deploying an additional set of centralized re-judgment equipment next to each centralized re-judgment station, one master and one slave, to prevent the host from crashing or the production line inspection equipment from being disconnected from the host due to a network failure. If the inspection equipment fails to get the response result from the host for many times, the slave machine will be enabled for centralized re-judgment. Figure 5 shown.

[0093] Specifically, when an enterprise needs n centralized re-judgment devices, it can deploy an additional centralized re-judgment device system next to each centralized re-judgment station, one master and one slave. In this way, the total number of centralized re-judgment devices is 2n. Under normal circumstances, the images of all inspection devices will be sent to the main centralized re-judgment device (referred to as the host) through the network after passing through the visual inspection module. When the host crashes, or the production line inspection equipment is disconnected from the host due to a network failure, when the inspection equipment fails to get the response result from the host for many times, the redundant standby centralized re-judgment device (referred to as the slave) will be enabled, and the images to be re-judged of all inspection devices under the original host will be sent to the slave. Through this master-slave redundancy and automatic switching mechanism, safe production is guaranteed.

[0094] The timeout default processing mechanism refers to the response in the system that when a unit fails or the link network between units is abnormal, resulting in the next step not being able to get the result returned by the previous step, the default processing result will be used as the result between system units by default, such as Figure 7 shown.

[0095] Specifically, the visual inspection module needs to receive pictures from the inspection equipment and give the results of the visual inspection. If the visual inspection module does not receive the picture or does not give the result within the specified time due to network reasons, in this method, the visual inspection module will usually default to NG (bad) treatment for this picture; similarly, the centralized re-judgment module may not receive the picture or give the re-judgment result due to network reasons, or due to abnormalities caused by some factors. In this method, there is also a default NG mechanism. Of course, the default treatment is NG (defective product) or OK (good product) and can be freely set. In some cases, if the abnormality persists, the default NG may cause too many misjudgments of the production line.

[0096] In one embodiment, the step of identifying the defect condition in the image includes: providing a computing power scheduling and computing power management mechanism for the computing power of visual detection and recognition, such as Figure 8 As shown in the figure, computing power clusters are grouped and managed according to the needs of product and model adaptation. When visual inspection and recognition are performed, the idle status and group to which all computing power belongs are recorded in real time. When an abnormality occurs in the computing power used or there is a new task, a query is made in the adaptable computing power before scheduling.

[0097] Specifically, there are usually two ways to deploy computing power for visual inspection modules. One is to deploy computing power close to the end side, and it is often deployed directly on the computer of the inspection equipment. The advantage of this is that it can avoid failures and delays caused by network transmission. The disadvantage is that computing power can only meet the needs of its own inspection equipment and cannot participate in computing power resource scheduling to meet the computing power requirements of other inspection equipment. Another way is server cluster deployment or cloud deployment, and a computing power pool composed of computing power units. For computing power clusters, this application provides computing power scheduling. When a new computing task arrives, it will give priority to selecting spare computing power resources for calculation to improve detection efficiency. When a computing unit fails, it can also switch to other computing power. In addition to scheduling, this application also provides a computing power management mechanism. Since enterprises may need to call different AI models in the computing power cluster when producing different products, when scheduling computing power, it is necessary to schedule in the model pool that adapts to the current product. Therefore, computing power management can group and manage computing power clusters according to the needs of product and model adaptation. When performing visual inspection, the idle status and belonging group of all computing power will be recorded in real time. When there is a new task, it will be queried in the adaptable computing power and then scheduled. Quality inspectors can freely group, manage, switch, upload and update models in the computing power pool through the computing power management window of the centralized review human-computer interaction interface. In this way, while making full use of computing power resources, it can also meet the needs of different products for different model algorithms. As mentioned earlier, visual inspection also has a default timeout processing mechanism. Occasional timeout exceptions can meet production inspection needs through computing power resource scheduling. However, when a computing power unit continues to have abnormalities, it will pose a risk to production. This application will record the abnormal conditions of the computing power unit. When the abnormality of a computing power unit continues to occur and exceeds the set value, for example, the computing power unit returns the default NG for three consecutive times, the computing power management will mark it as an abnormality and directly remove it from the computing power cluster to improve the scheduling efficiency of the computing power cluster itself. At the same time, continuous abnormal conditions will remind quality inspectors in the centralized re-judgment human-computer interaction interface, which is convenient for relevant personnel to check and handle software, network or hardware. Quality inspectors can refresh or repair the abnormal computing power through the centralized re-judgment computing power management window.

[0098] In one embodiment, the present application makes comprehensive considerations and balances the optimization between improving system efficiency, detection efficiency, production efficiency and meeting the needs of special scenarios of enterprises. Specifically, the centralized re-judgment process is balanced and optimized according to the efficiency improvement steps, including: combining the manual dynamic intervention mechanism, determining the intervention result of the imaging image corresponding to the product, and using the intervention result as the re-judgment result of the centralized re-judgment process; in response to the fact that the centralized re-judgment quality inspector has no time to re-judgment the received imaging images, all the imaging images to be judged and the defect conditions of visual inspection are established and saved in the order of receipt through a cache queue and a free selection mechanism, so that all the imaging images that have not been inspected in time are queued in turn for centralized re-judgment; in response to the existence of multiple inspection surfaces for the same product, multiple imaging images are formed, and the multiple imaging images are independently processed separately during visual inspection and combined and displayed during re-judgment through a multi-image processing and matching mechanism.

[0099] Specifically, with regard to the manual dynamic intervention mechanism, according to the needs of the enterprise to save manpower, the centralized re-evaluation device m has different values. When the value is 0, it is a fully automatic unmanned mode. At this time, there is no need for quality inspectors to participate in the centralized re-evaluation. When the enterprise requires both unmanned automated operation and manual quality inspectors to perform final intervention and re-evaluation when needed, centralized re-evaluation provides a manual dynamic intervention mechanism. During normal unmanned operation, the centralized re-evaluation device will also display a picture of the visual inspection results. Combined with the default timeout processing mechanism, within the set time, if there is no manual re-evaluation, the visual inspection results will be automatically processed. If within the set time, the manual quality inspector re-evaluates the picture and dynamically intervenes in the result, the centralized re-evaluation will issue an OK or NG signal based on the result of the manual intervention.

[0100] Specifically, with regard to the cache queue and free selection mechanism, when quality inspectors conduct centralized re-inspections of multiple production lines, due to the influence of low product yield, many defects, fast cycle time and slow personnel judgment, the personnel often do not have time to make judgments. In order to minimize the impact of manual re-inspection speed on the production line, the centralized re-inspection equipment provides a queue cache and free selection mechanism. For example, when the quality inspector has not yet finished judging the previous picture, the inspection equipment and visual inspection may have completed the imaging and visual judgment of a new picture, and sent the picture and results to the centralized re-inspection equipment waiting for personnel to re-judge. At this time, the centralized re-inspection equipment will build a cache queue and save all the pictures to be judged, visual inspection recognition and quality sorting results and other information in the order of receipt. In this way, all pictures are queued in turn waiting for quality inspectors to re-judge. The centralized review interface also provides a small queue preview window, where you can see how many images are in the queue, how long they have been waiting, and which production lines they are from. In the queue window, quality inspectors can freely choose which production line to view and review first. This can meet some special needs of production lines, rather than having to review images in the order they arrive, such as Fig. 9 As shown, component 1, component 2, and component 3 are sent to the centralized re-judgment system in the order of detection by the imaging and visual inspection systems, waiting for manual quality inspectors to make judgments. Under normal circumstances, quality inspectors will make judgments in order of priority. However, if component 3 and component 1 and component 2 are not from the same production line, and component 3 has special requirements and needs to be judged first, the quality inspector can give priority to component 3 for judgment.

[0101] Specifically, for multi-image processing and matching mechanisms, such as Fig. 9As shown. Since the object to be inspected may have multiple inspection surfaces, forming multiple images, at each moment, each production line inspection object will generate multiple images waiting for re-judgment, so centralized re-judgment needs to support re-judgment of multiple images of the same inspection object. For example, in the solar panel manufacturing process, some processes only need to detect EL defects. As mentioned earlier, hidden cracks, cold solder joints, etc. are EL defects. At this time, each production line only has one EL image that needs centralized re-judgment, while some processes need to detect EL defects, front appearance defects, and back appearance defects. As mentioned earlier, foreign matter and dirt are appearance defects. In this case, each component of each production line may have 3 images that need centralized re-judgment, but due to different imaging schemes of the inspection equipment for different inspection surfaces, the imaging speed is different. Different inspection surfaces have different defects, resulting in different visual inspection results and speeds. Therefore, the present application provides a multi-image processing and matching mechanism, which can independently process and merge images of different production lines. For example, among these three pictures, the EL image visual inspection result may be OK, while the appearance image inspection result is NG. It is also possible that the EL visual inspection result is fast, while the front appearance visual inspection result is slow. In order to maximize the use of computing power resources, the present application can process different surface images of different objects to be inspected separately and independently, instead of completing the inspection of three images of an object to be inspected and then inspecting the three images of the next object. Therefore, it does not distinguish whether the three images belong to the same object to be inspected. The one that comes earlier will be processed first. This makes full use of computing power and reduces the system's waiting time for processing. For example, due to the large size of the image, in order to facilitate manual re-judgment and the needs of some enterprises, there may be three centralized re-judgment windows at the production site, each of which corresponds to a detection surface (EL, front appearance, and back appearance, all from multiple production lines). As mentioned earlier, there may be multiple components queued up for each centralized re-judgment, and when the quality inspector determines the EL of a specific component in the EL centralized re-judgment window, he may not want to see the front appearance pictures from other components in the front appearance centralized re-judgment window. He just wants to see the EL, front appearance, and back appearance pictures from the same component when judging this component, and does not pay attention to the processing speed of these three detection surfaces and their respective OK / NG results (if there is only one detection surface, in the manual centralized re-judgment scenario of a single image, OK usually does not need to be displayed). Therefore, the centralized re-judgment device of this application saves the results of each detection surface on the basis of the independent and separate processing of the above-mentioned visual recognition detection, and adds a matching mechanism to display 3 pictures at the same time, which not only improves the efficiency of computing power utilization, but also meets the needs of enterprises.

[0102] In one embodiment, the method further includes: using a load balancing mechanism to intelligently distribute and manage the imaging images in the centralized re-judgment process. The load balancing mechanism refers to automatically balancing the workload among multiple centralized re-judgment devices, or managing and scheduling the centralized re-judgment devices to balance the workload among multiple centralized re-judgment quality inspectors in response to the temporary absence of some centralized re-judgment quality inspectors, such as Figure 5 shown.

[0103] Specifically, the load balancing mechanism is mainly used to automatically balance the workload among multiple centralized re-judgment devices, or to deal with the temporary absence of some centralized re-judgment quality inspectors. It is equivalent to adding a management and scheduling module on the upper layer of the centralized re-judgment device to balance the workload among multiple centralized re-judgment quality inspectors. For example, there are 10 inspection devices on 10 production lines, and the centralized re-judgment system is deployed in a centralized control room far away from the production line. The enterprise has enabled two centralized re-judgment systems. Assume that the two re-judgment systems are A and B, and two quality inspectors are responsible for re-inspecting the images of the 10 inspection devices after visual inspection. The first case is to balance the workload. Ideally, each quality inspector is responsible for the centralized re-judgment of images of 5 production lines, but in reality, due to the influence of various factors such as the difference in production capacity between different production lines, the difference in quality of the objects to be inspected, the difference in quality standards, or the difference in ability and quality between quality inspectors, the number of images waiting for re-judgment by the quality inspector of the centralized re-judgment system A may be greater than the number of images waiting for re-judgment by the quality inspector of the centralized re-judgment system B, which will lead to a difference in workload between the two quality inspectors. For example, assuming that the production line corresponding to system A temporarily arranges to increase production orders, resulting in a higher demand capacity for system A than system B, then system A needs to re-judge more pictures than system B; assuming that the production equipment of some production lines corresponding to system A fails at a certain moment, resulting in a higher defective product ratio than system B, then there are also more NG original pictures, and system A needs to re-judge more pictures than system B; assuming that some production lines corresponding to system A have tightened quality standards at the request of their end customers, then system A has more NG pictures after quality sorting, and system A needs to re-judge more pictures than system B; assuming that the above factors are the same, and only because the quality inspector of the centralized re-judgement system A is sick or an inexperienced new employee, resulting in a slower judgment speed, then the waiting time of centralized re-judgement system A is longer than that of centralized re-judgement system B. There are many other factors, which may cause the number of images or speed of images processed by the quality inspectors corresponding to the two centralized re-judgment systems to be different, which will affect the on-site production. The load balancing mechanism of the intelligent dispatching system will schedule and balance the tasks between the two centralized re-judgment systems. According to the number of real-time images waiting for re-judgment on the two centralized re-judgment systems, the subsequent centralized re-judgment tasks from 10 detection devices will be automatically scheduled. Since the number of images waiting for re-judgment in the centralized re-judgment system B is small, the intelligent dispatching system will send more images to the centralized re-judgment system B. This will balance the workload of the two quality inspectors on the one hand, and reduce the waiting time for re-judgment in the centralized re-judgment system A on the other hand, and improve the response speed of the production line detection equipment. In another case, when a quality inspector of a centralized re-judgment system needs to leave his post temporarily, he can selectively transfer the production line he is responsible for to other centralized re-judgment systems to meet the temporary leave needs. When returning to work, the intelligent dispatching will resume the normal load balancing mode.Since the customers of manufacturing companies have different demands for products, even the same product may have different quality standards on different production lines. Some quality standards are very complicated and difficult for quality inspectors to remember. Although the centralized re-judgment system supports re-judgment of products with different quality standards together and provides corresponding prompts, if the quality inspectors at the centralized re-judgment station switch back and forth between different standards, it will also affect the accuracy of the judgment. Therefore, companies generally put products and production lines with the same quality standards together for re-judgment. The same is true when the intelligent dispatch system schedules tasks between different centralized re-judgment systems.

[0104] Combination Figures 4 to 9 It can be seen that visual inspection and recognition transmits unqualified imaging pictures to the centralized re-judgment equipment and outputs the results of the centralized re-judgment. If the result of the centralized re-judgment is qualified, the product will flow to the qualified process; if it is unqualified, the product will flow to the unqualified process. In the visual inspection and recognition stage, computing power scheduling, computing power management mechanism and timeout default processing mechanism are introduced. When determining the re-judgment priority of unqualified imaging pictures, a cache queue and free selection mechanism are introduced. In the centralized re-judgment process, a master-slave redundancy mechanism, a load balancing mechanism, and a multi-image processing and matching mechanism are introduced. For the final determination of the centralized re-judgment results, a manual dynamic intervention mechanism and a timeout default processing mechanism are introduced.

[0105] Therefore, the overall technical solution of this application is to balance and optimize the centralized re-judgment process according to the improvement of system security, efficiency and special needs. Special needs refer to the operational needs of quality inspectors, such as which component they want to prioritize (corresponding to image caching and free selection mechanism), the need to temporarily leave their posts (corresponding to load balancing), judging the same component (corresponding to multi-image processing and matching), the adaptation between models and products, and the free switching and management of models (corresponding to computing power management). In actual applications, if there are no such special needs, the system efficiency may still be improved, but if these needs are to be met, it may affect the system efficiency, detection efficiency or production efficiency, so optimization and balance must be done to meet the needs and maximize efficiency, so the above-mentioned mechanisms are proposed.

[0106] The present application provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a detection device, the method described is implemented. The method includes: obtaining an image of a product; identifying the defect condition in the image; the defect condition includes the defect type and the defect degree; determining the process flow of the product corresponding to the image according to the defect condition of the image; in response to the product being qualified, directing the product to a qualified process; in response to the product being unqualified, directing the image corresponding to the product to a centralized re-judgment process; and determining the final process flow of the unqualified product according to the result of the centralized re-judgment.

[0107] A person of ordinary skill in the art can understand that all or part of the steps in the method for implementing the above-mentioned embodiment can be completed by instructing a processor through a program, and the program can be stored in a computer-readable storage medium, and the storage medium is a non-transitory medium, such as a random access memory, a read-only memory, a flash memory, a hard disk, a solid-state hard disk, a magnetic tape, a floppy disk, an optical disc, and any combination thereof. The above-mentioned storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or a data center that includes one or more available media. The available medium can be a magnetic medium (for example, a floppy disk, a hard disk, a tape), an optical medium (for example, a digital video disc (DVD)), or a semiconductor medium (for example, a solid-state disk (SSD)), etc.

[0108] See also Fig.10 , which is a schematic diagram showing the structural connection of the detection device described in the embodiment of the present application. Fig.10 As shown, this embodiment provides a detection device 1, including: a processor 11, a memory 12, a communication interface 13 and / or a system bus 14. The memory 12 and the communication interface 13 are connected to the processor 11 through the system bus 14 and complete communication with each other, the memory 12 is used to store computer programs, the communication interface 13 is used to communicate with other devices, and the processor 11 is used to run the computer program, so that the electronic device 1 performs each step of the product detection method based on the centralized re-judgment of the production line.

[0109] The above-mentioned processor 11 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components.

[0110] The above-mentioned memory 12 may include a Random Access Memory (RAM), and may also include a non-volatile memory, such as at least one disk memory.

[0111] The above-mentioned system bus 14 may be a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The system bus 14 can be divided into an address bus, a data bus, a control bus, etc. The communication interface is used to implement communication between the database access device and other devices (such as clients, read-write libraries, and read-only libraries).

[0112] Please refer to Fig.11 , which shows the structural schematic diagram of the detection system described in the embodiment of the present application. As Fig.11 shown, this embodiment provides a detection system, including: a detection device 1, an imaging device 2, and a centralized rejudgment device 3; the detection device 1 is respectively communicatively connected to the imaging device 2 and the centralized rejudgment device 3.

[0113] The imaging device 2 acquires imaging pictures of the product.

[0114] The detection device 1 obtains the imaging pictures of the product from the imaging device 2; identifies the defect conditions in the imaging pictures; the defect conditions include defect types and defect degrees; determines the process flow of the product corresponding to the imaging pictures according to the defect conditions of the imaging pictures; in response to the product being qualified, makes the product flow to the qualified process; in response to the product being unqualified, makes the imaging pictures corresponding to the product flow to the centralized rejudgment process of the centralized rejudgment device 3.

[0115] The detection device 1 determines the final process flow of the unqualified product according to the result of the centralized rejudgment feedback by the centralized rejudgment device 3.

[0116] In one embodiment, in response to the product being unqualified, the detection device sends the imaging pictures of the unqualified product to the centralized rejudgment device at the centralized rejudgment station. The centralized rejudgment device displays the imaging pictures of the unqualified product to the quality inspector at the centralized rejudgment station, and obtains the result of the centralized rejudgment generated by the quality inspector on the display interface. The centralized rejudgment device feeds back the result of the centralized rejudgment to the detection device.

[0117] Specifically, in response to the result of the centralized re-judgment being qualified, the detection equipment sends a first flow direction signal to the control device corresponding to the assembly line, so that the control device controls the assembly line to flow the unqualified products to the qualified process according to the first flow direction signal; in response to the result of the centralized re-judgment being unqualified, the detection equipment sends a second flow direction signal to the control device corresponding to the assembly line, so that the control device controls the assembly line to flow the unqualified products to the rework process or offline according to the second flow direction signal.

[0118] In different embodiments, the detection equipment 1 may include an imaging device 2, and the imaging device 2 may also be a separate device independent of the detection equipment 1; the centralized re-judgment device 3 may be a detection device at any detection station, or it may be a separate device independent of the detection equipment 1 and located at an additional centralized re-judgment station.

[0119] The protection scope of the product inspection method based on centralized re-judgment of the production line described in the embodiment of the present application is not limited to the execution order of the steps listed in this embodiment. All solutions implemented by adding, reducing or replacing steps in the prior art based on the principles of the present application are included in the protection scope of the present application.

[0120] The product defect detection system provided in the embodiments of the present application can implement the product detection method based on centralized re-judgment of the production line described in the present application, but the implementation device of the product detection method based on centralized re-judgment of the production line described in the present application includes but is not limited to the structure of the product defect detection system listed in the present embodiment. All structural deformations and replacements of the prior art made according to the principles of the present application are included in the protection scope of the present application.

[0121] The descriptions of the processes or structures corresponding to the above-mentioned figures have different emphases. For parts that are not described in detail in a certain process or structure, please refer to the relevant descriptions of other processes or structures.

[0122] The above embodiments are merely illustrative of the principles and effects of the present application and are not intended to limit the present application. Anyone familiar with the technology may modify or change the above embodiments without violating the spirit and scope of the present application. Therefore, all equivalent modifications or changes made by a person of ordinary skill in the art without departing from the spirit and technical ideas disclosed in the present application shall still be covered by the claims of the present application.

Claims

1. A product inspection method based on centralized re-judgment of a production line, characterized in that: The method comprises: Get an image of the product; Identify the defect condition in the image; the defect condition includes defect type and defect degree; Determine the process flow of the product corresponding to the image according to the defect condition of the image; In response to the product being qualified, the product is directed to a qualified process; in response to the product being unqualified, the image corresponding to the product is directed to a centralized re-judgment process; the centralized re-judgment process is balanced and optimized based on system security and efficiency improvement; According to the results of the centralized re-judgment, the final process flow of the unqualified products is determined; in response to the result of the centralized re-judgment being qualified, the unqualified products are flowed to the qualified process; in response to the result of the centralized re-judgment being unqualified, the unqualified products are flowed to the rework process or offline.

2. The method according to claim 1, characterized in that In response to the product being unqualified, the step of sending the imaged picture corresponding to the product to a centralized re-judgment process includes: In response to the product being unqualified, an image of the unqualified product is sent to a centralized re-inspection device at a centralized re-inspection station, so that the centralized re-inspection device displays the image of the unqualified product to the quality inspector at the centralized re-inspection station, and obtains the centralized re-inspection result generated on the centralized re-inspection device after the quality inspector's re-inspection operation.

3. The method according to claim 2, characterized in that The method further includes determining the number of centralized re-judgment devices and the number of personnel saved in the centralized re-judgment process: Determine the number of centralized re-inspection equipment based on the number of all inspection equipment on the production line, the number of images of each inspection equipment, the product yield of machine vision inspection, the accuracy of visual inspection, production and manufacturing standards, the maximum number of quality inspectors, required production capacity, and the number of people saved; The number of centralized re-judgment equipment ranges from 0 to the number of all testing equipment. After the centralized re-judgment process is started, the number of quality inspectors required for m testing equipment is changed to n quality inspectors, that is, the number of manpower saved is mn, and the cost reduction is 4. The method according to claim 1, characterized in that The steps of balancing and optimizing the centralized re-judgment process according to system security include: Set up a master-slave redundancy mechanism and deploy an additional set of centralized re-judgment equipment next to each centralized re-judgment station, one master and one slave, to prevent the host from crashing or the production line inspection equipment from being disconnected from the host due to a network failure. If the inspection equipment fails to get a response from the host for many times, the slave machine will be enabled for centralized re-judgment. A timeout default processing mechanism is set, and in response to an abnormality in the imaging image received by the centralized re-judgment device, unqualified is used as the re-judgment result of the centralized re-judgment process by default.

5. The method according to claim 1, characterized in that: The step of identifying the defect condition in the imaging picture comprises: A computing power scheduling and management mechanism is provided for the computing power of visual inspection and recognition. The computing power clusters are grouped and managed according to the needs of product and model adaptation. When performing visual inspection and recognition, the idle status and group to which all computing power belongs are recorded in real time. When an abnormality occurs in the computing power used or there is a new task, a query will be made in the adaptable computing power before scheduling.

6. The method according to claim 1, characterized in that The steps of balancing and optimizing the centralized re-judgment process according to efficiency improvement include: Combined with the manual dynamic intervention mechanism, the intervention result of the imaging picture corresponding to the product is determined, and the intervention result is used as the re-judgment result of the centralized re-judgment process; In response to the fact that the quality inspectors of the centralized re-judgment process do not have time to re-judge the received imaging images, a cache queue and free selection mechanism are used to establish a cache queue and save all the imaging images to be judged and the defect conditions of visual inspection in the order of receipt, so that all the imaging images that have not been inspected in time are queued up in turn for centralized re-judgment; In response to the existence of multiple inspection surfaces for the same product, multiple imaging pictures are formed. Through the multi-image processing and matching mechanism, the multiple imaging pictures are independently processed during visual inspection and combined and displayed during re-judgment.

7. The method according to claim 1, characterized in that The method further includes: utilizing a load balancing mechanism to intelligently dispatch and manage the imaging images in the centralized re-judgment process; The load balancing mechanism refers to automatically balancing the workload among multiple centralized re-judgment devices, or coping with the situation where some centralized re-judgment quality inspectors are temporarily away from their posts, so as to manage and schedule the centralized re-judgment devices and balance the workload among multiple centralized re-judgment quality inspectors.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a detection device, the method according to any one of claims 1 to 7 is implemented.

9. A detection device, characterized in that: The detection device includes: a processor and a memory; The memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory, so that the detection device performs the method according to any one of claims 1 to 7.

10. A detection system, characterized in that: The system comprises: an imaging device, the detection device according to claim 9 and a centralized re-judgment device; the detection device is respectively connected to the imaging device and the centralized re-judgment device for communication; The imaging device collects an image of the product; The inspection device obtains an image of the product by the imaging device; identifies the defect status in the image; the defect status includes the defect type and the defect degree; determines the process flow of the product corresponding to the image according to the defect status of the image; in response to the product being qualified, the product is directed to the qualified process; in response to the product being unqualified, the image corresponding to the product is directed to the centralized re-judgment process of the centralized re-judgment device; The detection device determines the final process flow of the unqualified products according to the results of the centralized re-judgment fed back by the centralized re-judgment device; In response to the product being unqualified, the inspection device sends the imaging picture of the unqualified product to the centralized re-inspection device at the centralized re-inspection station, the centralized re-inspection device displays the imaging picture of the unqualified product to the quality inspector at the centralized re-inspection station, and obtains the result of the centralized re-inspection generated by the quality inspector on the display interface, and the centralized re-inspection device feeds back the result of the centralized re-inspection to the inspection device.

Images