Quality control method and system of production line and manufacturing platform

Abstract

The embodiment of the invention provides a quality control method and system of a production line and a manufacturing platform, belongs to the technical field of manufacturing, and aims to improve the quality inspection efficiency and automation level in the production line, and the method comprises the steps: responding to the triggering of a first preset condition, and carrying out the quality inspection of a plurality of to-be-inspected objects produced on a target production line; feeding back the quality inspection result to a control object associated with the target production line, wherein the control object is used for controlling the technological process of the target production line; and in response to a second preset condition triggered on the control object, regulating and controlling the target production line based on the quality inspection result of the at least one object to be inspected, wherein the quality inspection comprises the steps of performing at least one time of image acquisition on a to-be-inspected object produced by the target production line, and inputting at least one acquired image into a defect detection model so as to perform quality inspection on the to-be-inspected object.

Description

Quality control methods, systems and manufacturing platforms for production lines Technical Field

[0001] The present disclosure relates to the field of manufacturing technology, and in particular to a quality control method, system and manufacturing platform for a production line. Background Art

[0002] In the field of manufacturing, multiple production lines are involved in the preparation of devices. The devices produced may include display panels, chips, etc. During the manufacturing process of these devices, and after the devices are manufactured, the devices will be quality inspected to eliminate unqualified devices so that the devices are qualified products when they leave the factory.

[0003] Currently, the efficiency of device quality inspection in the manufacturing field is low.

[0004] Overview

[0005] In a first aspect of the present disclosure, a quality control method for a production line is provided, the method comprising:

[0006] In response to a triggering of a first preset condition, performing quality inspection on a plurality of objects to be inspected produced on the target production line;

[0007] Feeding back the quality inspection result to a control object associated with the target production line, wherein the control object is used to control the process flow of the target production line;

[0008] In response to a second preset condition being triggered on the control object, regulating the target production line based on a quality inspection result of at least one of the objects to be inspected;

[0009] The quality inspection includes: performing at least one image acquisition on the object to be inspected produced by the target production line, and inputting at least one acquired image into a defect detection model to perform quality inspection on the object to be inspected.

[0010] In order to solve the above problems, the second aspect of the present disclosure provides a quality control method for a production line, the method comprising:

[0011] Capturing at least one image of the object to be inspected produced on the target production line;

[0012] Inputting the collected at least one image into a defect detection model to perform quality inspection on the object to be inspected through the defect detection model;

[0013] Based on the quality inspection result of at least one of the objects to be inspected on the target production line, the target production line is regulated.

[0014] A third aspect of the present disclosure provides a quality control system comprising an infrastructure layer and a quality control layer;

[0015] The infrastructure layer includes a plurality of first devices, where different first devices are used to provide different services required for quality inspection;

[0016] The quality control layer communicates with the infrastructure layer and is used to execute the quality control method described in the first aspect or the second aspect.

[0017] A fourth aspect of the present disclosure provides a manufacturing platform comprising a computer integrated system and the quality control system according to the third aspect, wherein the quality control system is integrated into the computer integrated system, and a production information management system is provided in the computer integrated system;

[0018] Wherein, the quality control system communicates with the production information management system;

[0019] The quality control system is configured to execute the quality control method of the first aspect or the second aspect and send debugging instructions to the production information management system when regulating the target production line;

[0020] The production information management system is used to regulate and control the process equipment in the target production line according to the debugging instructions.

[0021] According to a fifth aspect of the present disclosure, a computer-readable storage medium is provided, in which a computer program stored enables a processor to execute the quality control method for a production line as described in the embodiment of the first aspect of the present disclosure. According to a computer-readable storage medium, a computer program stored enables a processor to execute the quality control method for a production line as described in the embodiment of the first aspect of the present disclosure.

[0022] An embodiment of the present disclosure also discloses an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the quality control method for the production line as described in the embodiment of the first aspect.

[0023] In the technical solution of the embodiment of the present disclosure, quality inspection can be performed on multiple objects to be inspected produced on the target production line in response to a triggered quality inspection function; and the quality inspection results of the objects to be inspected can be displayed on the display interface, and a debugging function can be triggered based on the displayed quality inspection results, and the target production line can be regulated based on the quality inspection result of at least one object to be inspected; wherein, the quality inspection of the present disclosure includes: performing at least one image capture on the objects to be inspected on the target production line, and inputting at least one captured image into a defect detection model to perform quality inspection on the objects to be inspected.

[0024] Since it is possible to start quality inspection on the objects to be inspected on the target production line in response to the triggered quality inspection function, and the results of the quality inspection can be fed back to the control object associated with the target production line, the control object at least includes the client on the target production line and the display interface corresponding to the target production line; and in response to the debugging function triggered on the control object, the target production line is regulated based on the quality inspection results of at least one object to be inspected, so that the quality inspection of the objects to be inspected disclosed in the present invention can be deeply implemented in the production line for preparing the objects to be inspected, so as to adjust the process flow of the production line in combination with the quality inspection situation of the objects to be inspected, so as to solve the yield problem of the objects to be inspected from the source of preparing the objects to be inspected, such as process parameters, process equipment, etc., and reduce the production cost consumed by the discarded objects to be inspected due to defective objects to be inspected.

[0025] The above description is only an overview of the technical solution of the present disclosure. In order to more clearly understand the technical means of the present disclosure, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present disclosure more obvious and easy to understand, the specific implementation methods of the present disclosure are listed below.

[0026] BRIEF DESCRIPTION OF THE DRAWINGS

[0027] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure or related technologies, the following is a brief introduction to the drawings required for the description of the embodiments or related technologies. Obviously, the drawings described below are some embodiments of the present disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without inventive efforts. It should be noted that the scales in the drawings are for illustration only and do not represent the actual scale.

[0028] FIG1 shows a schematic diagram of an implementation environment of a manufacturing platform in an embodiment of the present disclosure;

[0029] FIG2 shows a schematic diagram of a framework of a quality control system according to an embodiment of the present disclosure;

[0030] FIG2 a shows a schematic diagram of a list of servers managed by a detection device management unit;

[0031] Figures 2b-2d show schematic diagrams of interfaces for managing user information by a user management unit;

[0032] FIG2e shows a schematic diagram of the division of labor of modules related to quality inspection in the quality control system;

[0033] FIG3 shows a schematic flow chart of steps of a quality control method for a production line according to an embodiment of the present disclosure;

[0034] FIG4 shows a schematic diagram of a first configuration interface in an embodiment of the present disclosure;

[0035] FIG5 shows a schematic diagram of a strategic framework for performing quality inspection under the quality control system shown in FIG2 ;

[0036] FIG6 shows a flow chart of the steps for building a defect detection model;

[0037] FIG7 shows a schematic diagram of the third configuration interface when constructing a defect detection model;

[0038] FIG8 shows an interface for acquiring a second data set;

[0039] FIG9 is a schematic diagram showing a flow chart of steps for obtaining a second data set in an embodiment of the present disclosure;

[0040] Figures 10a-10c are schematic diagrams of interfaces for performing statistics on quality inspection results;

[0041] FIG11 shows a flowchart of steps for performing statistics on quality inspection results in an embodiment of the present disclosure;

[0042] FIG12 is a schematic diagram showing a flow chart of steps for regulating a target production line in an embodiment of the present disclosure;

[0043] FIG13 shows a complete flow chart of quality control in an embodiment of the present disclosure;

[0044] FIG14 shows a schematic diagram of an interface setting of a quality control system in an embodiment of the present disclosure;

[0045] FIG15 is a schematic diagram showing a flow chart of steps of a quality control method for a production line according to an embodiment of the present disclosure;

[0046] FIG16 shows a schematic flow chart of the steps for real-time adjustment of the quality inspection process in an embodiment of the present disclosure.

[0047] Detailed description

[0048] To make the objectives, technical solutions, and advantages of the embodiments of the present disclosure more clear, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present disclosure, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present disclosure without making any creative efforts shall fall within the scope of protection of the present disclosure.

[0049] In the related technology, in the field of production and manufacturing, it is necessary to conduct quality inspection on the produced devices. One quality inspection method is to use manual quality inspection of the devices, which relies on the experience of the workers and has low accuracy and is not suitable for devices with relatively subtle defects. Another quality inspection method is to use artificial intelligence for quality inspection, such as taking a photo of the device and then using a professional neural network model to inspect the device. However, the current solution of using artificial intelligence for quality inspection has not been deeply implemented in the factory and combined with the production line, and it is still isolated from the production line.

[0050] Among them, applying artificial intelligence to factory quality inspection needs to face the following practical problems:

[0051] 1. Production lines generally involve multiple industrial control systems, such as the control systems for process equipment and the transmission control systems. Each industrial control system performs different functions. When applying artificial intelligence for quality inspection, it is necessary to combine artificial intelligence with multiple industrial control systems, which requires additional system integration and development costs.

[0052] 2. In the manufacturing field, production efficiency is a very important device production indicator. This means that it is necessary to promptly detect devices that fail quality inspection and defects in the production line, which requires a high level of real-time quality inspection. However, in related technologies, due to the reasons mentioned in point 1, the coupling between quality inspection and production lines is difficult, resulting in poor real-time quality inspection.

[0053] 3. In the manufacturing sector, integrated management and control of industrial control equipment on multiple production lines is required to improve the level of automated control of production lines. However, current industrial control equipment is generally independent of each other and controlled by its own system. Industrial control equipment between production lines is not interconnected. This means that when implementing artificial intelligence, targeted development must be considered for different production lines, resulting in high development costs.

[0054] 4. When using AI for quality inspection, a large amount of labeled data is required. In the manufacturing field, the devices to be inspected may be finished products or semi-finished products, and the data generally needs to be kept confidential. This means that large amounts of labeled data are difficult to obtain. At the same time, the labeled data needs to be kept confidential, which has become a major problem for the implementation of AI for quality inspection in factories.

[0055] 5. In the manufacturing field, such as the manufacturing of chips, display panels and other devices, the technical personnel who conduct quality inspections have a narrow range of professional levels and a low level of application of artificial intelligence, which makes them unable to skillfully use artificial intelligence for quality inspections. As a result, when artificial intelligence is implemented in factories, the professional level of workers is required to be high, which invisibly raises the technical threshold for implementation.

[0056] Based on the above practical problems, the application of artificial intelligence in device quality inspection in factories is currently still at the level of theoretical research, with few actual implementations. After implementation, it also faces the above practical problems, resulting in high maintenance costs after implementation, and its contribution to improving the manufacturing efficiency and product yield of factory production devices is still relatively low.

[0057] In view of this, the present disclosure proposes a solution that can deeply implement artificial intelligence into the production line of the factory, reduce the technical threshold for implementation, and improve the efficiency of quality inspection. Specifically, a quality control method for the production line is proposed. This quality control method can directly respond to the user's trigger, use a neural network model to perform quality inspection on the object to be inspected, and feed back the quality inspection results to the production line, so that the production line can adjust the process flow and other parameters on the production line based on the quality inspection results, thereby achieving the purpose of timely rectification and debugging of the shortcomings of the production line based on the quality inspection results, thereby ensuring that the process of the object to be inspected is adjusted in time during the production process, thereby ensuring the production quality of the object to be inspected; thus, the quality control method of the present disclosure can have the following advantages:

[0058] 1. When using artificial intelligence (defect detection model) for quality inspection, the quality inspection results are fed back to the control objects associated with the target production line, achieving mutual integration between quality inspection and the target production line. At the same time, it can directly respond to the second preset condition triggered on the control object of the target production line and directly perform regulation. When artificial intelligence is implemented on the production line, as long as a communication connection path is developed between the quality inspection process and the control objects corresponding to the target production line, the system integration development cost is relatively low;

[0059] 2. Improve the coupling between the quality inspection of the objects to be inspected and the production line. The real-time quality inspection is more real-time, so as to adjust the shortcomings of the production line in real time.

[0060] 3. When AI is applied to quality inspection of objects, relatively simple trigger conditions, such as first and second preset conditions, can be set. This facilitates worker control of the AI ​​and reduces the required expertise in AI. This allows workers to operate the AI ​​for quality inspection with minimal technical barriers. Furthermore, the inspection results can be used to control production lines with minimal technical barriers. Regarding this third point, the present disclosure further reduces the technical barrier to use by simplifying the parameter settings required for AI. See the subsequent embodiments for details.

[0061] The quality control method of the production line disclosed herein is described in detail below.

[0062] Example 1

[0063] To gain a deeper understanding of the quality control method proposed in this disclosure, first, the system framework designed for the quality control method in the embodiments of this disclosure is described:

[0064] 1 and 2 , FIG1 shows a schematic diagram of an implementation environment of a manufacturing platform proposed in the present disclosure, and FIG2 shows a schematic diagram of a framework of a quality control system in the manufacturing platform. The manufacturing platform includes a quality control system and a computer integration system, wherein the quality control system is integrated into the computer integration system, and the computer integration system is provided with a production information management system MES;

[0065] Among them, the quality control system can communicate with the production information management system;

[0066] The quality control system can be used to execute the quality control method of the subsequent embodiment 2 and send debugging instructions to the production information management system when regulating the target production line;

[0067] Among them, the production information management system can be used to control the process equipment in the target production line according to debugging instructions.

[0068] The manufacturing platform may include multiple production lines, and different production lines may be used to prepare different objects to be inspected, or different production lines may be used to perform different processes required in the preparation process of the objects to be inspected.

[0069] Among them, the quality control system can be used to perform quality control on multiple production lines. Specifically, quality inspection can be carried out on the objects to be inspected based on the images obtained by image acquisition of the objects to be inspected completed by each production line. According to the quality inspection results, the process equipment of the production line can be adjusted to achieve the purpose of centralized control of the production quality of multiple production lines.

[0070] Among them, the quality control system is mainly integrated into the computer integrated system, which can act as an independent system to interact with the production information management system. The production information management system integrates the management threads of multiple production lines to control the start, stop and process flow of the production lines through the management threads. The production line generally has some process equipment, and different process equipment can complete different process preparations in the production of the object to be inspected.

[0071] In some embodiments, the manufacturing platform further includes: a plurality of second devices, the plurality of second devices communicate with the quality control system and are used to collect information on the objects to be inspected on the production line, where the information collection includes at least image collection.

[0072] In some specific implementations of this embodiment, multiple second devices can be controlled by a production information management system, which may include some or all of the process equipment on the production line. The second equipment is mainly used to collect information related to quality inspection during quality inspection. As shown in Figure 2, the industrial equipment included in the production line may include: a cutting machine, a PLC, a sensor, a camera, a light source and a robotic arm. The PLC (Programmable Logic Controller), a sensor, a camera, a light source and a robotic arm are second devices for information collection. The cutting machine is a device for cutting display panels. It may not be used for information collection during quality inspection. However, it can respond to the control instructions fed back by the quality control system based on the quality inspection results, update its own process parameters, and thus realize the control of the cutting process.

[0073] In some other specific implementations of this embodiment, multiple second devices may also include a server for controlling and managing process equipment. The server can control and manage multiple process equipment that perform the same information collection on multiple production lines. For example, there are three production lines for panel cutting. If two cameras are set up on each production line, there are at least six cameras, and the six cameras can be controlled and managed by one server.

[0074] Among them, the servers in the multiple second devices can be deployed in the production information management system 300, which can serve as a node device to control and manage the equipment used for information collection on multiple production lines.

[0075] In this way, when the quality control system is implemented, multiple second devices on the production line can be connected to the quality control system and uniformly managed by the quality control system. The specific management and control can be as described in the subsequent embodiments: parameter configuration, status monitoring, etc. In this way, whether it is for quality inspection or production line regulation, there is no need for the industrial control equipment between the production lines to be connected to each other, but the control layer level can be connected through the quality control system. Therefore, the development cost of the quality control system when it is implemented is reduced, which is sufficient to deal with the practical problem of the above point 3.

[0076] In yet other embodiments, the plurality of second devices include: an image acquisition device, a controller, a sensor, and a robotic arm; the controller is connected to the image acquisition device via the robotic arm, and the sensor is connected to the controller;

[0077] A sensor is used to detect whether the object to be inspected has arrived;

[0078] The controller is used to control the image acquisition device to move when the object to be inspected arrives, so as to capture at least one image of the object to be inspected by the image acquisition device.

[0079] Among them, when the sensor detects the arrival of the object to be inspected, it can send a trigger signal to the controller. According to the trigger signal, the controller controls the robotic arm to drive the image acquisition device to move according to the pre-configured parameters corresponding to the control. During the movement, the image acquisition device can perform at least one image acquisition on the area to be inspected of the object to be inspected according to the pre-configured parameters corresponding to the image acquisition device.

[0080] The parameters corresponding to the image acquisition device may include image acquisition frequency, resolution, etc.

[0081] Therefore, it is only necessary to open up the communication between artificial intelligence and the industrial control equipment in the industrial control system, and to use the existing equipment in the industrial control system, thereby reducing the system development cost.

[0082] It should be noted that the objects to be inspected mentioned in the present disclosure can be various devices produced by the target production line, such as devices for electronic equipment, devices for production and daily life, etc. Among the devices for electronic equipment, they can include display panel devices such as display panels and array substrates, and can also include microelectronic devices such as diodes and semiconductors; among the devices for production and daily life, they can include ceramics, stainless steel utensils, molds, glass and other utensils, can include hardware, and can also include textiles such as cloth.

[0083] As shown in FIG2 , a schematic diagram of a framework of a quality control system in an example is shown, wherein the quality control system includes an infrastructure layer and a quality control layer;

[0084] The infrastructure layer includes a plurality of first devices, and different first devices are used to provide different services required for quality inspection; the quality control layer communicates with the infrastructure layer and is used to execute quality control methods.

[0085] Among them, the quality control layer can be understood as the software hub in the quality control system, which is used to perform quality control in the production line in combination with computer programs based on the hardware support provided by the infrastructure layer.

[0086] In this embodiment, the first device may include: a communication bridge device that establishes communication between the second device in the production information management system and the quality control layer, such as a gateway and an industrial control server, as well as a server for providing algorithm services, data analysis services and storage services for quality inspection.

[0087] In some embodiments, the quality control layer can also be communicatively connected with multiple second devices on the production line, for accessing the multiple second devices and for configuring parameters related to quality inspection for the multiple second devices; wherein the multiple second devices are used to collect information on the objects to be inspected on the production line, and the information collection includes at least image collection.

[0088] As described in the above embodiment, the second device can be controlled by a production information management system. It can be part of or all of the process equipment on the production line, primarily used to collect information related to quality inspection. It can also include a server that controls and manages the process equipment.

[0089] Among them, the second device and the first device can be the same in hardware structure, but the second device in this disclosure specifically refers to the equipment used for image acquisition, production line on-site control, etc. on the production line, while the first device in this disclosure specifically refers to the server, cluster, etc. that supports the quality control system.

[0090] When adopting this implementation method, the quality control system can reuse the existing industrial control equipment on the production line to collect the information required for quality inspection through the existing industrial control equipment, thereby reducing the implementation cost when the quality control system is implemented on the production line. Since there is no need to introduce new equipment, there is no need to carry out access, protocol matching and other development for the newly introduced equipment, thereby achieving implementation with a low technical threshold.

[0091] The infrastructure layer is used to provide the first device required for the quality control system to perform quality control. In one example, the first device may include a gateway and a server. The gateway may include at least one, and the server may also include at least one.

[0092] Different servers are used to provide different business services for the production line, wherein the business services at least include: control services and quality inspection services.

[0093] Among them, the control service may refer to: controlling the process equipment of the production line based on the quality inspection result of at least one object to be inspected on the production line; the quality inspection service may refer to: quality inspection of the objects to be inspected on the production line.

[0094] As shown in FIG2 , the server providing control services in the first device is called an industrial control server, and the server providing quality inspection services is called a computing power server.

[0095] As shown in Figure 2, in the infrastructure layer, multiple first devices may include gateways and industrial control servers, both of which are used to build a communication bridge between the second device in the production information management system and the quality control layer; specifically, for the gateway, it is used to establish a communication link between the second device and the computing power server, and to establish a communication connection between the process equipment in the production line and the industrial control server. The process equipment here can include both the second device and the equipment used for device production.

[0096] For example, the image information collected by the second device on the object to be inspected can be sent to the computing power server via the gateway, and the computing power server performs quality inspection based on the image; based on the quality inspection results, the quality control layer outputs control instructions to the industrial control server, which is sent by the industrial control server to the process equipment on the corresponding production line in the production information management system, such as a cutting machine, to adjust the process parameters of the cutting machine. In addition, the quality control layer can also send instructions to the industrial control server to adjust the configuration parameters of the second device, and the industrial control server can send the configuration parameters to the second device so that the second device collects information based on the configuration parameters. In particular, when the industrial control server sends the configuration parameters to the second device, it can send the configuration parameters to the server used to control the process equipment in the first device, thereby sending the configuration parameters to the equipment on the corresponding production line through the server.

[0097] In some embodiments, at least one server includes a quality inspection server configured with a defect detection model, and the quality inspection server is used to input the image into the defect detection model after receiving the image captured by the second device, so as to perform quality inspection on multiple objects to be inspected; wherein, the second device is located on the production line.

[0098] In this embodiment, at least one server may include the aforementioned computing server, i.e., the quality inspection server, and may also include a data server for data storage and analysis and the aforementioned industrial control server. In some other examples, a training server for model training and updating may also be included. The quality inspection server and the industrial control server may be deployed at the nodes of the quality control system, i.e., distributedly deployed around the nodes, such as near the second device. This is suitable for distributed factories, such as branch factories distributed in different regions, so that the quality inspection server and the second device form a local area network. The data server for data storage and analysis and the training server may be deployed in the cloud to provide additional services or functions.

[0099] Of course, quality inspection servers and training servers can also be deployed in the cloud to provide fast quality inspection services through the computing resources integrated in the cloud, while data servers and industrial control servers that provide data storage and analysis can be deployed at the nodes of the quality control system, that is, distributedly deployed around the nodes, such as near the second device, thereby providing faster quality inspection result queries.

[0100] As shown in Figure 5, at least one server may include a data server for data storage and analysis. The quality inspection server can communicate with the data server and send the quality inspection results to the data server for storage. The data server can support statistics, queries and retrieval of the quality inspection results based on the stored quality inspection results.

[0101] Among them, different quality inspection servers can be configured with different defect detection models. In this case, multiple quality inspection servers can be used to perform quality inspection on defects of different types of objects to be inspected. It can be understood that the types of defects that need to be inspected for objects to be inspected produced on different production lines can be the same. In this case, different production lines can share the same defect detection model for quality inspection; or, different quality inspection servers can be used to perform quality inspection on objects to be inspected produced by different production lines. In this case, each production line can correspond to an independent quality inspection server for independent quality inspection.

[0102] Among them, different quality inspection servers can be configured with defect detection models with different detection accuracy. In this case, the defect detection model with lower detection accuracy can be used for rough inspection of the object to be inspected during quality inspection, and the defect detection model with higher detection accuracy can be used for fine inspection of the object to be inspected during quality inspection. Therefore, the quality inspection efficiency can be improved through rough inspection and fine inspection.

[0103] As shown in Figure 5, at least one server may include a data server ③ for data storage and analysis, and the quality inspection server may include multiple first quality inspection servers ① and second quality inspection servers ②, wherein the multiple first quality inspection servers ① can be used to perform rough inspection on the objects to be inspected, and the second quality inspection server ② can perform fine inspection on the objects to be inspected that fail the quality inspection.

[0104] Among them, Figure 5 shows four first quality inspection servers ①, which can be deployed on different production lines to perform rough inspection on the same defect of the objects to be inspected produced by different production lines, or can be deployed on the same production line to perform rough inspection on different defects of the objects to be inspected produced on the production line; or, the four first quality inspection servers ① can be deployed on the same production line to perform distributed rough inspection on the defects of multiple objects to be inspected produced on the production line.

[0105] Among them, the second quality inspection server ② can be used to perform precision inspection on objects to be inspected with different defect types. It can be deployed at the main node to perform precision inspection on objects to be inspected that have failed quality inspection on multiple different production lines.

[0106] As shown in FIG2 , in one embodiment, the quality control layer may include an information management module, a detection module, and a model management module; wherein:

[0107] The information management module can be used to perform at least one of the following information processing: processing data generated during quality inspection, processing status information of multiple first devices, and processing user information of users;

[0108] The detection module can be used to perform at least one of the following in the quality inspection of the object to be inspected: parameter configuration in the quality inspection, quality inspection of the object to be inspected, quality inspection error correction of the object to be inspected, and access configuration of multiple devices;

[0109] The model management module can be used to build defect detection models.

[0110] The data generated during the quality inspection may include: image data obtained by capturing images of the object to be inspected during the quality inspection, quality inspection result data, and parameter data of parameter configuration to be performed during the quality inspection, etc.

[0111] The status information of the various first devices may refer to status information generated by monitoring the first devices, and processing thereof may include displaying, analyzing, and issuing alarms on the status information;

[0112] The user information may refer to the information of the user who logs into the quality control system, including user account information, login information, etc. The processing thereof may include: permission processing, account modification processing, etc. It should be noted that the user here may include ordinary logged-in users and administrator users.

[0113] In some embodiments, the information management module may further include a user management unit, which may be used to manage user permissions and activate corresponding functions in quality control according to the user permissions.

[0114] Specifically, the user management unit may be configured to, in response to a user triggering operation on a target function among the multiple functions, determine whether the target function complies with the user's authority, and activate the target function if the target function complies with the user's authority;

[0115] Among them, multiple functions are used to perform the following operations after being activated:

[0116] Parameter configuration of multiple first devices, debugging of process equipment of production lines, statistics of quality inspection results, updating of defect detection models, associating configuration parameters of target production lines with associated production lines, and error correction of misdetected objects to be inspected.

[0117] Specifically, in some examples, the operations that users can perform in the quality control system include the following categories:

[0118] Quality inspection: including initiating quality inspection of the production line, setting the configuration parameters (configuration parameters of the first device) required for quality inspection (corresponding to parameter configuration of multiple first devices), verifying the quality inspection results (corresponding to error correction of the inspected objects), statistics, analysis and query of quality inspection results, and migration of quality inspection (corresponding to associating the configuration parameters of the target production line with the associated production lines);

[0119] Infrastructure layer management: including access, deployment, status monitoring, operation, network configuration, and security protection of the first devices in the infrastructure layer (including gateways, industrial control servers, quality inspection servers, and data servers);

[0120] Production line control: including statistical analysis of quality inspection results, control of process equipment (including secondary equipment and servers) on the production line, and adjustment of configuration parameters of secondary equipment;

[0121] Algorithm control category: including the deployment, update (corresponding to the update of the defect detection model), construction, configuration of model parameters, and construction of training samples used by the defect detection model (involving data labeling).

[0122] It should be noted that the function activated by an operation permission included in each major category is called the target function. Among them, for ordinary logged-in users, the permissions they can operate may be different from those of administrator users, or they may overlap with each other.

[0123] For example, the operation permissions that ordinary logged-in users have in the quality inspection category may include: setting the configuration parameters required for quality inspection (configuration parameters of the second device) and verifying the quality inspection results; while the operation permissions that administrator users have in the quality inspection category may include: initiating quality inspection of the production line, and statistics, analysis, and querying the quality inspection results. Alternatively, in some cases, the operation permissions that ordinary logged-in users have in the quality inspection category may include: initiating quality inspection of the production line, setting the configuration parameters required for quality inspection (configuration parameters of the second device), and statistics, analysis of the quality inspection results; while the operation permissions that administrator users have in the quality inspection category, in addition to the above-mentioned operation permissions (operation permissions of ordinary users), may also include: verifying the quality inspection results and querying the statistical results of quality inspection.

[0124] As another example, the operation permissions performed by ordinary logged-in users in the infrastructure layer management category may include: status monitoring, operation, and network configuration of the first device in the infrastructure layer (including gateways, industrial control servers, quality inspection servers, and data servers). In addition to the above operation permissions, administrator users can also include device access, security protection, etc.

[0125] For example, a normal user might not have access to production line control, while an administrator might have access. A normal user might not have access to algorithm control, while an administrator might have access. Alternatively, a normal user might only have limited control rights within the production line control category, such as adjusting the configuration parameters of a second device. A normal user might also only have limited control rights within the algorithm control category, such as constructing training samples for a defect detection model (involving data annotation).

[0126] Of course, since the technical threshold for model deployment is lowered in the embodiments of the present disclosure, ordinary users can also easily deploy, update, build, and configure model parameters of the defect detection model, and build training samples used by the defect detection model (involving data labeling). Therefore, administrator users can authorize ordinary users to obtain the above-mentioned operation permissions.

[0127] 2b-2d, a schematic diagram of the interface for managing user information by the user management unit is shown. As shown in FIG2b, both ordinary logged-in users and administrator users can include user name, user type, user ID, password, registration date, remarks, etc., wherein the management user can edit the user information of each ordinary user, change the password, etc.; as shown in FIG2c, a schematic diagram of the interface for the operation information generated by the user during the operation is shown, including user name, log type, log details, event server ID, etc., which is used to count the operations performed by users under different users and the servers where the operations are located; as shown in FIG2d, the statistics of the quality inspection work of each user by the user management unit are shown, including: user ID, number of images, number of NGs, and manual re-inspection time, which is used to record the quality inspection work performed by users on the production line.

[0128] It should be noted that Figures 2b to 2d are merely exemplary illustrations and are not specifically limited herein.

[0129] As described above, the user's authority to the target function among the multiple functions may be granted by the administrator user with the highest authority among the administrator users.

[0130] The user management unit can create users and assign permissions based on the administrator's needs. It then responds to users on the production line registering industrial control devices and registers them into the management system. After configuring the initial connection between the industrial control devices and the server, users enter and debug production material information, industrial control device parameters, and algorithm parameters before beginning testing. This process is performed according to permissions.

[0131] In this way, the quality control process can be broken down into multiple sub-functions so that users with corresponding technical levels can use the corresponding sub-functions, thereby achieving the purpose of controlling the quality of the production line based on the workers' technical level and improving the control level and efficiency.

[0132] In some embodiments, the data generated during the quality inspection may include at least one of the following: quality inspection results and images collected during the quality inspection;

[0133] Accordingly, the processing of data generated during quality inspection includes at least one of the following processes: storage, statistics, retrieval, annotation, and query.

[0134] In this embodiment, storage may refer to the storage of quality inspection results and images, statistics may refer to the statistics of quality inspection results, and retrieval may include retrieval of quality inspection results of a certain object to be inspected, retrieval of quality inspection results of a certain inspection period, etc.

[0135] As shown in FIG2 , in one example, the information management module may include, in addition to the above-mentioned user management unit, a data management unit, a device management unit, and a data analysis unit;

[0136] The data management unit is used to perform data statistics, data retrieval, data storage, data annotation and multi-terminal query. In other words, the above-mentioned processing of the data generated in the quality inspection can be data statistics, data retrieval, data storage, data annotation and multi-terminal query.

[0137] Among them, the equipment management unit is used to perform functions such as status monitoring, configuration strategy, policy issuance, and production line control. Among them, configuration strategy and policy issuance may involve the above-mentioned: different servers can be configured with different defect detection models to perform quality inspection on different types of defects, or perform rough and fine inspection on the quality inspection of the objects to be inspected; among them, production line control can communicate with the industrial control server to adjust the process parameters of multiple second devices on the production line.

[0138] Status monitoring can include monitoring the status of gateways and servers, and of course, can also include monitoring the status of secondary equipment on the production line. Monitoring the status of gateways and servers can include: whether the network connection is secure, whether the network connection is normal, whether the communication between servers is secure, etc.

[0139] The data analysis unit may be used to perform statistical analysis and data docking of the quality inspection results. The data docking may refer to docking the quality inspection results of different production lines of the same object to be inspected.

[0140] Referring to Figure 2a, a list diagram of the device management unit managing the server is shown. As shown in Figure 2a, it includes the settings of the server type, such as whether it is a fine inspection server or a coarse inspection server, including the server code, server online status, IP address, the number of objects that have been inspected and are awaiting inspection, etc.

[0141] As shown in FIG2 , in one example, the detection module may include an industrial control equipment configuration unit, an algorithm configuration unit, a material configuration unit, an equipment monitoring unit, and a quality inspection unit;

[0142] The industrial control device configuration unit may be used to perform debugging of the industrial control device, configuration of device parameters, protocol parsing, and device access; wherein the industrial control device may include a first device and a second device, and the configuration of device parameters may refer to configuration of parameters of the first device and the second device;

[0143] Among them, the algorithm configuration unit can be used to perform real machine debugging and parameter configuration. The parameter configuration here can refer to the configuration of model parameters;

[0144] Among them, the material configuration unit can be used to execute product parameter entry and quality inspection strategy configuration. Product parameters can refer to the parameters of the object to be inspected, such as the size of the object to be inspected, the area requiring quality inspection, etc. The configuration of the quality inspection strategy can refer to whether to conduct batch quality inspection on the object to be inspected, fine inspection or rough inspection, etc.

[0145] Among them, the quality inspection unit is used to perform defect detection on the area to be inspected of the object to be inspected, real-time quality inspection monitoring, verification of quality inspection results, and statistics of quality inspection results;

[0146] Among them, the equipment monitoring unit is used to monitor the working conditions of each device and can issue a linkage alarm for faulty equipment. The linkage alarm here can mean that all modules associated with the faulty equipment are alarmed. For example, if the faulty equipment is a camera for image acquisition, the linkage alarm can be: an alarm in the quality inspection unit, an alarm in the industrial control equipment configuration unit, etc.

[0147] As shown in Figure 2, the model management module can also be called the AI ​​development platform, which can be used to support data labeling, model development, model training and model publishing.

[0148] Among them, data labeling can refer to the re-labeling of the objects to be inspected that are falsely detected or missed after quality inspection. False detection indicates that the defect detection model cannot accurately detect the object to be inspected. Therefore, the image of the object to be inspected can be called a negative sample or a difficult sample. It can be re-labeled as a training sample to update the defect detection model.

[0149] Among them, model development may include the construction of a defect detection model, the updating of a defect detection model, and the migration of a defect detection model as described in subsequent embodiments, so as to obtain a migration model adapted to an associated production line, wherein the defects of the objects to be inspected to be detected by the associated production line are the same or similar to the defect types of the objects to be inspected to be detected by the target production line.

[0150] Among them, model release can refer to the deployment of the defect detection model after obtaining the defect detection model, for example, deployment to a server. As mentioned above, different servers can deploy defect detection models for different defect types, and different servers can also deploy defect detection models with different detection accuracy.

[0151] In a specific example, the modules related to quality inspection in the above-mentioned quality control system are analyzed and explained. Referring to Figure 2e, a schematic diagram of the division of labor of the modules related to quality inspection in the quality control system is shown. As shown in Figure 2e, in general, it includes automated training, quality inspection client and quality inspection server. The quality inspection server is the server configured with the defect detection model mentioned above; automated training is the above-mentioned model management module; the quality inspection client is the above-mentioned detection module, and the quality inspection server is used for quality inspection.

[0152] Automated training includes data labeling, automated training, production line real machine debugging, and service release. Service release is model release. These functions will be explained in detail in the subsequent Example 2.

[0153] Specifically, automated training can be achieved through the model management module. In response to a triggered model building function, at least one trigger control is triggered. Different trigger controls are used to trigger different configuration windows during the model building process. Subsequently, multiple configuration options are output in different configuration windows for the user to select. Different configuration options correspond to different preset configuration parameters. The preset configuration parameters are used to provide parameters such as the operating environment, training configuration, and algorithm for the training model. In this way, users can perform automated model training through the model management module.

[0154] At the same time, the model management module can also be used to create a new annotation data set, i.e., the second data set, in response to the triggered model update function. Afterwards, a variety of configuration options for model update are output in different configuration windows for users to choose from. Different configuration options correspond to different preset configuration parameters. The preset configuration parameters are used to provide the training model with operating environment, training configuration, algorithm and other parameters, thereby realizing the automatic update of the defect detection model.

[0155] Of course, the model management module can also be used to provide the associated production line with the required model updates when the defect detection model on the target production line needs to be migrated to the associated production line. For example, the target data set corresponding to the associated production line is obtained, and in response to the triggered model migration function, the target data set is used to update the defect detection model on the target production line to obtain a migration model; wherein, when the migration model is obtained, a variety of configuration options for model updates can also be output in different configuration windows for users to choose from.

[0156] Of course, in some other scenarios, the automated training can also be used in other scenarios different from the production lines in the industrial manufacturing field. For example, in the fields of smart banking, smart parks, or smart transportation, smart banks have many business lines, and smart parks and smart transportation include multiple transportation routes. If each line and business line is compared to a production line, and a corresponding business category on each line and business line is an object to be inspected on the production line, the method disclosed in this disclosure can be applied to collect information on objects on each line and business line, and use the collected information as training samples for the "production line", thereby obtaining an AI model for a class of objects under the production line.

[0157] For example, the automated training disclosed in this disclosure can be used to count customer flows for each business line in smart banks, or to find people in smart parks or smart transportation. This can also solve the aforementioned technical problems in these scenarios, such as a lack of training samples and the need to update or add models.

[0158] Similarly, AI models can be migrated between lines and between business lines. During the migration process, only a small number of training samples from the associated production lines are required, which can also solve the problem of difficulty in obtaining samples.

[0159] The quality inspection client includes functions such as manual re-inspection, initialization of the inspection machine, configuration of production material information, configuration of industrial control equipment parameters (i.e., configuration parameters), start of quality inspection, and data visualization. Production material information refers to the information of the objects to be inspected on the target production line. This function will also be explained in detail in the subsequent embodiment 2 in relation to the quality inspection window interface.

[0160] The data server includes functions such as personnel authority issuance, detection machine strategy, data management, and data analysis, which will be explained in detail in the subsequent embodiment 2.

[0161] It should be noted that the data annotated by manual review in the quality inspection client can be fed back to the automated training as a data set for model update (second data set); the model published by the service can be used for quality inspection;

[0162] The information generated by the quality inspection client in the above process is stored in the server and managed by the data server.

[0163] The quality management system disclosed in this disclosure can be deeply implemented in manufacturing plants, closely linked to the front-end production lines, and utilize the hardware support provided by the infrastructure layer to conduct quality inspections on the objects to be inspected produced on each production line through the quality control layer, and regulate the process equipment of the production line based on the quality inspection results. It is also possible to achieve separate management of different tasks in the quality inspection control process through the establishment of different modules, such as the information management module, the detection module, and the model management module. At the same time, it integrates core functions such as data annotation, model training, and model publishing in the development of quality inspection scenario models, lowering the technical threshold and enabling departments with weak AI technology accumulation to use AI technology, thereby improving R&D efficiency.

[0164] In combination with the manufacturing platform and quality control system described in the above embodiment 1, the quality control method of the production line disclosed in the present invention is described in the second embodiment.

[0165] Example 2

[0166] 3 , a flow chart of the steps of the quality control method for a production line of the present disclosure is shown. As shown in FIG3 , the method can be applied to a quality control system, more specifically, to a quality inspection module of the quality control system, and can specifically include the following steps:

[0167] Step S301: in response to a triggering of a first preset condition, quality inspection is performed on a plurality of objects to be inspected produced on a target production line.

[0168] In this embodiment, the first preset condition may be triggered by a preset operation on an interface of the quality control system. For example, a single click or double click on a quality inspection button on the interface indicates that the first preset condition has been triggered. Based on this trigger, quality inspection of multiple target machines produced on the target production line may be initiated.

[0169] The quality inspection includes capturing at least one image of the object to be inspected produced by the target production line and inputting the captured at least one image into the defect detection model to perform quality inspection on the object to be inspected. The quality inspection can be either a single inspection or a periodic inspection, without any specific limitation.

[0170] The objects to be inspected may refer to devices, appliances, textiles, etc. produced on the target production line, and are not limited here.

[0171] Specifically, the quality inspection process may be: each time an object to be inspected is produced, at least one image of the object to be inspected is captured, and then at least one captured image is input into a defect detection model for defect detection.

[0172] Capturing at least one image of the object to be inspected may refer to capturing at least one image of an area of ​​the object to be inspected. The area to be inspected may be determined based on the quality inspection objective. For example, if the target production line is a production line for cutting display substrates, the area to be inspected may be the edge of the cut display substrates. For another example, if the target production line is a production line for depositing film layers on display substrates, the area to be inspected may be the film deposition area.

[0173] The number of image acquisitions can be determined based on the accuracy requirements of quality inspection. Each image acquisition can obtain an image of the area to be inspected. In different image acquisitions, images can be acquired at different positions in the area to be inspected, or images with different resolutions can be acquired at the same position in the area to be inspected.

[0174] Image acquisition can be performed according to pre-configured parameters, which may include an acquisition frequency and an acquisition resolution. In some cases, the image acquisition may be performed at different locations of the area to be inspected while the image acquisition is moving. The pre-configured configuration parameters may also include a moving rate for image acquisition. The moving rate may be the moving rate of the image acquisition device when the object to be inspected is stationary and the image acquisition device is moving, or the moving rate of the object to be inspected when the image acquisition device is stationary and the object to be inspected is moving. For example, the image acquisition device moves relative to the area to be inspected of the object to be inspected. In this case, more image data may be obtained, or partial images that can be spliced ​​into a complete image may be obtained, or multiple separate samples that can be used to identify different defect areas may be obtained.

[0175] Among them, the defect detection model can be pre-configured into the quality inspection server according to the quality inspection requirements of the target production line. As described in the above embodiment, different quality inspection servers can be configured with defect detection models required by different production lines, or different quality inspection servers can be configured with defect detection models with different accuracies; when performing quality inspection, the image captured by the image acquisition device can be sent to the quality inspection server, and the quality inspection server calls the defect detection model to perform defect detection on the image, and outputs the defect detection results to the control object associated with the target production line.

[0176] Among them, the quality inspection result of the object to be inspected can be determined based on the defect detection results corresponding to multiple images of the object to be inspected. Since different images can reflect different positions in the area to be inspected, the defect detection results corresponding to different images can reflect the defect detection results at different positions. Therefore, the quality inspection result of the object to be inspected can be determined by combining the defect detection results at different positions.

[0177] In some examples, when the defect detection results of all images of the object to be inspected are defect-free, the object to be inspected is determined to have passed the quality inspection; otherwise, it is determined to have failed the quality inspection; in other examples, when the defect detection results of more than a preset number of images of the object to be inspected are defect-free, the object to be inspected is determined to have passed the quality inspection; if the number of images with defect-free results is less than the preset number, the object to be inspected is determined to have failed the quality inspection.

[0178] In other examples, at least one key position can be calibrated in the area to be inspected. The key position can refer to a position that has an important impact on the performance of the object to be inspected. Then, when the defect detection result at the key position passes, it can be determined that the object to be inspected has passed the quality inspection. When the defect detection result at the key position fails, even if there are no defects in other positions, it can be determined that the object to be inspected has failed the quality inspection. Based on this example, during the image acquisition process, the acquisition frequency and resolution of the image acquisition at the key position can be different from the acquisition frequency and resolution of the image acquisition at other positions. Specifically, the acquisition frequency of the image acquisition at the key position can be higher than the acquisition frequency of the image acquisition at other positions, or the resolution of the image acquisition at the key position can be different from the resolution of the image acquisition at other positions; or, the acquisition frequency and resolution of the image acquisition at the key position are both higher than the acquisition frequency and resolution of the image acquisition at other positions, thereby improving the pertinence of defect detection and helping to improve the efficiency of defect detection.

[0179] Of course, the calibration of the above-mentioned key positions can be achieved by setting configuration parameters, and the implementation process of the configuration parameters can be described in detail in the relevant parts of the following embodiments. In the example of determining the quality inspection result based on the defect detection result of the key position, an image acquisition method can be adopted in which the image acquisition device is stationary and the object to be inspected moves. In this way, when the key position moves into the camera range of the image acquisition device during the movement of the object to be inspected, it can be detected by the sensor, and then a trigger instruction is sent to the image acquisition device to trigger the image acquisition device to perform high-resolution and / or high-acquisition frequency image acquisition at the key position according to the configuration parameters corresponding to the key position; when the key position moves out of the camera range of the image acquisition device, it can also be detected by the sensor, and then a trigger instruction is sent to the image acquisition device to trigger the image acquisition device to perform low-resolution and / or low-acquisition frequency image acquisition at the remaining positions according to the configuration parameters corresponding to the positions outside the key position.

[0180] In some examples, the quality inspection results and images targeted by the quality inspection server can be sent to the data server. The data server can store the defect detection results of multiple images of the same object to be inspected, as well as the quality inspection results, so as to subsequently perform statistics and queries on the quality inspection results of the objects to be inspected on the target production line.

[0181] Step S302: Feedback the quality inspection result to the control object associated with the target production line, and the control object is used to control the process flow of the target production line.

[0182] During the quality inspection of the objects to be inspected on the target production line, the quality inspection results can be fed back to the control object every time a preset number of objects to be inspected are completed; or, the quality inspection results of the objects to be inspected completed in the current period can be fed back to the control object every preset time interval; or, the quality inspection results of each object to be inspected can be fed back to the control object every time the quality inspection of the object to be inspected is completed.

[0183] The control object is a control object associated with the target production line. In practice, it can be a client of the target production line. The client can be understood as the industrial control system of the target production line. As mentioned in the above background, the industrial control system can be used to control the process flow of the target production line and the process equipment on the target production line. For example, it can be a production information management system.

[0184] The control object may also be the equipment management unit in the information management module of the quality control system. Specifically, as shown in FIG2 , the equipment management unit may also communicate with the process equipment on the target production line (the second equipment referred to above) to configure the parameters of the process equipment. In this embodiment, the equipment management unit may be a window interface that provides control in the quality control system. The user may log in to the quality control system through an account, thereby entering a window interface related to control (production line control). In this window interface, the quality inspection results may be displayed according to the user's operation, and control may be triggered according to the user's operation.

[0185] In some embodiments, the control object can collect and analyze the quality inspection results of the objects to be inspected, determine the time period during which the objects to be inspected that failed the quality inspection were produced, or determine the distribution of defect locations of the objects to be inspected that failed the quality inspection, or both can be determined at the same time, and then the statistical results of the time period and the statistical results of the distribution of defect locations can be displayed in a chart, thereby helping users to intuitively understand the quality status of the objects to be inspected produced on the target production line, and thus conduct targeted debugging.

[0186] Step S303: In response to the second preset condition being triggered on the control object, the target production line is regulated based on the quality inspection result of at least one object to be inspected;

[0187] In this embodiment, the user can operate directly on the control object to promote the triggering of the second preset condition. The second preset condition can be triggered by the user clicking or double-clicking the "debug" button on the control object, such as in the above-mentioned control window interface. In this way, the system can automatically control the target production line according to the quality inspection results of the object to be inspected.

[0188] For example, in a specific solution, the browser front-end control button can be triggered, and the collected image (the image collected for the object to be inspected) stored in the database can be called in the background through a pre-set code or system library file or listening function, and transmitted to the defect detection model to output the result. In this case, the defect detection model can extract the image features through the pre-set code, and then output it as the quality inspection result. Of course, in some examples, the defect area image with defects can also be output, and then the above information (quality inspection results and defect area image) can be sorted and output to the front-end browser for presentation. It can be understood that after the AI ​​model (defect detection model) outputs the required information, the quality inspection results can be processed secondary by algorithms or pre-set codes, such as statistical classification, and the information obtained after the secondary processing can be presented to the user so that the user can observe the quality inspection results of the object to be inspected intuitively and in detail when the front-end control button is triggered.

[0189] Among them, the quality inspection results based on which the system is regulated can come from a batch of objects to be inspected. Of course, in some cases, they can also come from a certain object to be inspected. Preferably, the target production line can be regulated based on the quality inspection results of a batch of objects to be inspected.

[0190] Specifically, when the target production line is regulated based on the quality inspection results, as mentioned above, the quality inspection results can be statistically analyzed according to the output time period and defect location distribution. Generally speaking, when the quality inspection results are statistically analyzed according to the output time period, it can be determined which output time period has more objects to be inspected that fail the quality inspection and which output time period has fewer objects to be inspected that fail the quality inspection, so that it can be determined that the target production line is prone to process failures in certain time periods, so that the process flow of the time period can be regulated in a targeted manner; among them, when the statistical analysis is performed according to the defect location distribution, it can be determined which position of the objects to be inspected has more defects and which position has fewer defects. Generally speaking, the defect location is related to the output of the objects to be inspected by the target production line. For example, when cutting a display substrate, if the statistical analysis results of the defect location are displayed in the upper left corner of the display substrate, a large number of objects to be inspected will fail the defect detection, indicating that the cutting process equipment corresponding to the upper left corner needs to be debugged.

[0191] Therefore, the target production line can be regulated based on the statistical analysis results of the time period and / or defect location distribution of the objects to be inspected. During the regulation, the process flow corresponding to the output time period with a large number of objects to be inspected that fail the quality inspection can be regulated based on the statistical analysis results, and the process flow corresponding to the locations with a large number of defect inspections that fail can be regulated. In this way, the quality inspection of the objects to be inspected can be fed back to the target production line to regulate the target production line, so that the target production line can continue to maintain a higher quality process level, thereby improving the quality level of the objects it produces, and further improving the quality control level of the target production line.

[0192] It should be noted that the regulation referred to in this embodiment may include adjustment and control. Adjustment is the adjustment of the process flow, which may include improvement of the process flow and adjustment of the working parameters of the process equipment. Control may mean: controlling the target production line to prepare the object to be inspected based on the adjusted working parameters and the improved process flow.

[0193] By adopting the technical solution disclosed in the present invention, on the one hand, since it is possible to respond to the triggered quality inspection function, start quality inspection on the objects to be inspected on the target production line, and feed back the results of the quality inspection to the control object associated with the target production line, and respond to the debugging function triggered on the control object, the target production line is regulated based on the quality inspection results of at least one object to be inspected, so that the quality inspection of the objects to be inspected disclosed in the present invention can be deeply implemented in the production line for preparing the objects to be inspected, so as to regulate the process flow of the production line in combination with the quality inspection situation of the objects to be inspected, so as to solve the yield problem of the objects to be inspected from the source of preparing the objects to be inspected, such as process parameters, process equipment, etc., and reduce the production cost consumed by discarding the defective objects to be inspected. On the other hand, since the target production line can be regulated based on the statistical analysis results of the quality inspection results in the output period and defect location, the target production line can be regulated in a targeted manner to improve the regulation efficiency.

[0194] Among them, in order to enable quality inspection based on AI (Artificial Intelligence) technology to be deeply implemented in the production line to guide the quality control of the production line, the above-mentioned quality inspection can be triggered automatically based on the operation of the production line, thereby reducing labor costs, or it can be triggered manually by the user, so as to facilitate the user to control the progress of the quality inspection of the production line.

[0195] In order to fully explain the quality control method of the subsequent embodiments, the interface of the quality control system operated by the present method is described, as shown in FIG14 :

[0196] Among them, the quality control system may include a system interface, and multiple function buttons may be distributed on the system interface, such as a quality inspection function button, an adjustment function button, etc., wherein the quality inspection button can be clicked or double-clicked to enter the quality inspection window interface, and the adjustment button can be clicked or double-clicked to enter the adjustment window interface. In the quality inspection window interface, the user can configure the parameters based on quality inspection, trigger quality inspection, and adjust the progress of quality inspection in real time; through the adjustment window interface, the quality inspection results of the inspected object can be displayed, as well as the statistical analysis results of the quality inspection results, and the adjustment of the target production line can be triggered.

[0197] In a specific implementation, the infrastructure layer may also include a display server, which may refer to a web server that provides page display for the quality control system disclosed herein. The display server may communicate with a quality inspection server, a data server, a training server, and an industrial control server, and is used to transmit instruction information generated in response to a trigger in a front-end browser page to the quality inspection server, the data server, the training server, and the industrial control server to obtain corresponding information, and present the information on the front-end browser page after organizing it.

[0198] Among them, the above-mentioned system interface, quality inspection window interface, regulation window interface, etc. do not conflict with the architecture of the quality control system shown in Figure 2. The system interface, quality inspection window interface, regulation window interface can be reasonably arranged according to the architecture of the control system to facilitate user use.

[0199] Accordingly, in some examples, the first preset condition may include: the start of operation of the target production line, and / or a quality inspection function triggered on a preset interface.

[0200] In some embodiments, when the target production line begins to operate and prepare the object to be inspected, it can be determined that the first preset condition is triggered, thereby starting the quality inspection of the object to be inspected based on the operation of the target production line. In other embodiments, a quality inspection button can be set on the preset interface, and by clicking or double-clicking the quality inspection case, the first preset condition is triggered, thereby performing the quality inspection. The quality inspection button corresponds to the quality inspection function of the quality control system.

[0201] Among them, the preset interface can be the system interface of the quality control system, on which the following can be configured: quality inspection function button, control function button, the user can log in to the system through the account, thereby triggering the quality inspection function button, entering the quality inspection window interface, and triggering the quality inspection button in the window interface.

[0202] Accordingly, as described above, the triggering of the above-mentioned regulation can be an automatic regulation based on the results of quality inspection, thereby improving the level of automation of quality control of the production line, or it can be a regulation based on manual triggering by the user, thereby facilitating the user to manually debug the production line according to the results of quality inspection.

[0203] In some examples, the second preset condition includes: there are greater than or equal to a preset number of objects to be inspected that fail quality inspection, and / or a preset operation is triggered on the control object.

[0204] In this example, if the number of objects that fail quality inspection is greater than or equal to a preset number, the target production line can be regulated. In this case, it means that there are a large number of objects that fail quality inspection and the target production line needs to be regulated. As described above, the target production line can be regulated based on the statistical analysis results of the quality inspection results of the objects that pass quality inspection, such as based on the distribution of the production time period and defect locations of the objects.

[0205] Of course, in another example, the control of the target production line can also be triggered based on the user's preset operation on the control object. Among them, the preset operation can refer to a single click or double click of the control button on the control object. Among them, the control object here can be understood as the control window interface displayed when the control function button is triggered. The control window interface can display the feedback of the quality inspection results of the object to be inspected, and can also display the statistical analysis results of the feedback of the quality inspection results of the object to be inspected. Thus, the user can trigger the preset operation on the control window interface (control object) based on this statistical analysis result.

[0206] As described in the first embodiment above, the detection module may include an industrial control equipment configuration unit, an algorithm configuration unit, a material configuration unit, etc. These units can be used to configure and debug the parameters of the industrial control equipment, configure the parameters of the model algorithm, etc. Therefore, in some embodiments, when AI technology is implemented on the production line, configuration work can be performed first, and subsequent quality inspection can be carried out based on the configured parameters.

[0207] In a specific implementation, before step S301, in response to the triggered quality inspection configuration function, a first configuration interface may be displayed; and configuration parameters respectively inputted through the first configuration item and the second configuration item may be received;

[0208] The first configuration interface includes a first configuration item of a device for quality inspection and a second configuration item of a defect detection model;

[0209] Accordingly, when quality inspection is performed on a plurality of objects to be inspected produced on a target production line, quality inspection can be performed on the plurality of objects to be inspected produced on the target production line based on the configuration parameters.

[0210] In this embodiment, the quality inspection configuration function can be located in the quality inspection window interface. Specifically, a function button can be set to trigger the quality inspection configuration function, thereby displaying the first configuration interface. This function button can be related to the detection module in FIG. 2 above. For example, as shown in FIG. 14 , the quality inspection window interface can include a quality inspection configuration button for triggering the quality inspection configuration function. Its quality inspection configuration function can have the functions included in the industrial control equipment configuration unit, algorithm configuration unit, and material configuration unit in the detection module in FIG. 2 . Of course, after triggering the quality inspection configuration button, the window interface entered can provide the configuration items corresponding to the above-mentioned units to complete the setting of various parameters.

[0211] The first configuration interface may include a first configuration item and a second configuration item. The first configuration item may be used to configure parameters of a device used for quality inspection, and the second configuration item may be used to configure parameters of a defect detection model.

[0212] The number of devices used for quality inspection may be one or more, and may include devices required in the quality inspection process, such as image acquisition devices and controllers related to the image acquisition devices. In practice, configuration parameters of the devices used for quality inspection need to be input through the first configuration item. For example, in a specific implementation, the input configuration parameters can be collected in the background through the triggering of the quality inspection configuration button on the browser front end, through pre-set code or system library files or listening functions, and then sent to the background server, such as the industrial control server in the infrastructure layer, through the message queue in the form of a data packet, so that the industrial control server can parse the data instruction information and control the device or program to be controlled.

[0213] Specifically, taking the device for quality inspection including an image acquisition device and a controller as an example, as shown in FIG4 , a schematic diagram of the interface of the first configuration interface is shown. As shown in FIG4 , the first configuration item may include multiple sub-configuration items, and different sub-configuration items are used to configure different configuration parameters. As shown in FIG4 , the sub-configuration items include the sub-configuration items of the camera (image acquisition device), the PLC (controller), and the sub-configuration items of the trigger;

[0214] In some embodiments, the equipment for quality inspection includes an image acquisition device and a controller, and the first configuration item may be used to input working parameters corresponding to the image acquisition device and the controller respectively;

[0215] The operating parameters corresponding to the image acquisition device include acquisition frequency and / or acquisition resolution, and the operating parameters corresponding to the controller include a starting position, an ending position, and a movement rate. The controller is used to control the movement of the image acquisition device, or to control the movement of the object to be inspected, or to control the movement of both the image acquisition device and the object to be inspected. The movement rate here can be described in the relevant sections above.

[0216] Among them, the image acquisition device may include a camera, and the controller may be a PLC; as shown in Figure 4, for the camera configuration, the working parameters that need to be configured may include: acquisition frequency, acquisition resolution, etc.; for the controller, the working parameters that need to be configured may include: starting position, ending position, moving speed, etc. The controller here needs to control the starting position, ending position, and moving rate of image acquisition, wherein the starting position refers to: the starting position for starting image acquisition of the object to be inspected, and the ending position refers to: the starting position for ending image acquisition of the object to be inspected. The moving rate refers to: the moving rate of the object to be inspected from the starting position to the ending position, or refers to: the moving rate of the image acquisition device from the starting position to the ending position.

[0217] In this embodiment, since the starting position, ending position, and moving speed are configured, the command information can be directly generated and sent to the controller by defining the position on the configuration interface. Therefore, more convenient and rapid command information formation is achieved through the visual operation button.

[0218] For the trigger, the operating parameters that need to be configured may include: IP address, channel number, frequency setting, etc. The trigger here is used to detect the arrival of the object to be inspected and send a trigger signal to the corresponding controller to instruct the controller to control the camera or the movement of the object to be inspected for image acquisition. Among them, the trigger and the controller can communicate through the network, so parameters such as IP address, channel number, and frequency need to be configured.

[0219] Of course, the first configuration item may also include sub-configuration items of basic information, and the sub-configuration items of basic information may include the name of the associated target production line, configuration description, etc.

[0220] In still other embodiments, the second configuration item may be used to input a threshold parameter corresponding to the defect detection model, and / or to input an operating parameter of a server that configures the defect detection model.

[0221] As shown in Figure 4, the second configuration item is used to configure the parameters of the defect detection model. The parameters of the defect detection model here can be parameters used to control the defect pass rate of the model for the object to be inspected, such as threshold parameters. Therefore, the hyperparameters and operating environment parameters in the model do not need to be configured, but can be configured by professionals. In this way, the technical threshold for technicians to use AI to perform quality inspection of objects to be inspected can be lowered.

[0222] Among them, the server that is configured with the defect detection model is the above-mentioned quality inspection server. The working parameters of the quality inspection server may refer to the software and hardware configuration of the quality inspection server used for defect detection, or the ID configuration of the quality inspection server, so as to select a quality inspection server that is suitable for the current output status of the object to be inspected, such as the first quality inspection server (corresponding to rough inspection) or the second quality inspection server (corresponding to fine inspection).

[0223] Among them, the user can complete the configuration of the quality inspection equipment and the defect detection model by entering the corresponding configuration parameters in the first configuration item and the second configuration item on the first configuration interface. In this case, the quality control system can associate the configuration parameters configured by the user with the target production line, so that when the first preset condition of the target production line is triggered, quality inspection can be performed directly based on the configuration parameters.

[0224] Specifically, when quality inspection is performed on multiple objects to be inspected produced on the target production line based on configuration parameters, images of the objects to be inspected can be captured based on the configuration parameters of the quality inspection equipment, and when the defect detection model performs defect detection on the images of the objects to be inspected, it can be performed based on the configuration parameters of the defect detection model, such as based on the parameters of the defect pass rate, to adjust the probability threshold that characterizes the defect probability, thereby regulating the accuracy and pass rate of the quality inspection.

[0225] Of course, before this, the user can use the test tools, real machine debugging and other sub-functions shown in Figure 2 to pre-set the configuration parameters, that is, enter the test tools, real machine debugging, and enter a real machine debugging interface. The first configuration item and the second configuration item are also displayed on this interface, but the first configuration item and the second configuration item here are used to enter the configuration parameters used by the user for testing.

[0226] If the user inputs the configuration parameters of the quality inspection equipment and the configuration parameters of the defect detection model, the quality inspection equipment can collect information about the test device according to the configuration parameters input in the first configuration item, and the collected image of the test device can be detected according to the configuration parameters input in the first configuration item. The image of the test device and the detection result can be fed back to the real machine debugging interface so that the user can observe the image acquisition effect and the detection result, and thus set the configuration parameters in the first configuration item and the second configuration item to obtain the adapted configuration parameters. The adapted configuration parameters can be automatically migrated to the first configuration interface when the quality inspection configuration function is triggered, and automatically input into the first configuration item and the second configuration item in the first configuration interface, so that workers with low technical thresholds do not need to input the corresponding configuration parameters, but can be used directly. Among them, the initial device is a device that is consistent in shape and size with the object to be tested on the target production line.

[0227] In some embodiments, quality inspection can be real-time quality inspection, which means that the target production line is simultaneously inspecting the objects to be inspected while the objects have already been inspected. In this case, during the real-time quality inspection, the configuration parameters can be adjusted and updated in a timely manner through the quality inspection window interface based on the efficiency and results of the quality inspection to adapt to the output of the objects to be inspected by the target production line.

[0228] In a specific implementation, a second configuration interface may be displayed in response to a triggered quality inspection debugging function; and on the second configuration interface, statistical results of the quality inspection results of the inspected objects to be inspected are displayed, and configuration parameters after the current configuration parameters of the target configuration items are modified are recorded;

[0229] The second configuration interface includes target configuration items, and the target configuration items include first configuration items of equipment used for quality inspection and / or second configuration items of a defect detection model;

[0230] Accordingly, when quality inspection is performed on multiple objects produced on the target production line, quality inspection can be performed on multiple objects produced on the target production line based on the modified configuration parameters after the current moment. In particular, for real-time quality inspection, the configuration parameters can be adjusted at any time to adapt to the current quality inspection situation.

[0231] In this embodiment, configuration parameters can be adjusted during the quality inspection process, wherein the quality inspection debugging function can be triggered in the quality inspection window interface, for example, by triggering the "Quality Inspection Debug" button in the window interface. When this function is triggered, a second configuration interface can be displayed, wherein the second configuration interface can refer to the interface layout schematic of the first configuration interface, and the second configuration interface can include the first configuration item, the second configuration item, or both the first configuration item and the second configuration item.

[0232] As described above, the first configuration item is used to set configuration parameters for multiple quality inspection devices, and the second configuration item is used to set configuration parameters for the defect detection model. In the second configuration interface, the first configuration item and the second configuration item may have already set configuration parameters, that is, the original configuration parameters, and the original configuration parameters may be set in the first configuration interface as described in the above embodiment.

[0233] Among them, the second configuration interface can also display the statistical results of the quality inspection results of the inspected objects to be inspected, and the statistical results may include at least one of the following: the number of objects to be inspected that have been inspected per unit time, the number of objects to be inspected that have passed the quality inspection, the number of objects to be inspected that have failed the quality inspection, and the number of objects to be inspected that have been missed; among them, the number of objects to be inspected that have been inspected per unit time can reflect the speed of quality inspection, and the number of objects to be inspected that have passed / failed the quality inspection / missed the quality inspection can reflect the quality inspection accuracy and efficiency of the defect detection model on the objects to be inspected.

[0234] Among them, the above statistical results can be displayed in the form of numbers, charts, etc. on the second configuration interface, which can help users understand the progress and accuracy of quality inspection, so as to adaptively adjust the configuration parameters of the collected equipment and the configuration parameters of the defect detection model.

[0235] Specifically, when the number of objects to be inspected per unit time is less than the number of objects to be inspected produced by the target production line per unit time, the speed of quality inspection can be increased. For example, the moving rate of image acquisition (moving rate of the camera or controller), acquisition frequency, etc. can be adjusted so that more objects to be inspected can be inspected per unit time. At the same time, new quality inspection servers can be configured to implement distributed defect detection through distributed servers to improve quality inspection efficiency, or the software and hardware parameters provided by the server for the defect detection model can be configured so that the quality inspection server can call more computing resources for the defect detection model to improve the quality inspection efficiency of the defect detection model.

[0236] Among them, the user can adjust the existing configuration parameters in the first configuration item and / or the second configuration item, and in response to the user's save, the configuration parameters after the configuration items (first configuration item and second configuration item) are modified under the current configuration parameters can be recorded, thereby associating the modified configuration parameters with the target production line.

[0237] The association may include sending the modified configuration parameters to the corresponding device so that the corresponding device performs the corresponding task in quality inspection according to the modified configuration parameters. For example, if the modified configuration parameters are those of a camera, the modified configuration parameters will be sent to the camera so that the camera will re-acquire images according to the modified configuration parameters. For example, if the modified configuration parameters are those of a defect detection model, the modified configuration parameters may be sent to the quality inspection server so that the quality inspection server will perform defect detection according to the modified configuration parameters when using the defect detection model.

[0238] Since the speed, accuracy and precision of quality inspection can be adjusted by entering the corresponding parameters in the corresponding interface of the quality control system during quality inspection, it is convenient for users to manage and control quality inspection. In the process of quality inspection, the configuration parameters can be adjusted by entering the configuration interface. Therefore, when the target production line produces the objects to be inspected, the speed, accuracy and precision of quality inspection can be adjusted according to the output of the objects to be inspected, so as to deeply adapt to the front-end production line.

[0239] In yet other embodiments, the transmission speed of the objects to be inspected on the target production line can be adjusted based on the efficiency of the quality inspection of the objects to be inspected. Specifically, during the quality inspection of multiple objects to be inspected produced on the target production line, the number of objects to be inspected that have completed quality inspection per unit time can be determined; and based on this number, the transmission speed of the objects to be inspected on the target production line can be adjusted. This approach can also be applied to real-time quality inspection processes, and when applied to real-time quality inspection processes, the efficiency of quality inspection can be balanced with the production efficiency of the objects to be inspected.

[0240] As mentioned above, the number of objects to be inspected that have completed quality inspection per unit time can reflect the speed of quality inspection. If the speed of quality inspection is too fast, the output of objects to be inspected cannot keep up with the efficiency of quality inspection. If the speed of quality inspection is too slow, the speed of quality inspection cannot keep up with the output speed of objects to be inspected, which may cause the output of objects to be inspected to pile up at the quality inspection site, affecting the speed at which the objects to be inspected leave the factory.

[0241] In this embodiment, the transmission speed of the objects to be inspected produced by the target production line can be reversely adjusted according to the speed of quality inspection. The transmission speed can refer to: the transmission speed of the conveyor belt that transports the objects to be inspected to the quality inspection site after they are produced from the target production line. That is to say, in order to ensure the efficiency of producing the objects to be inspected, the transmission speed of the objects to be inspected from the target production line to the quality inspection site can be adjusted to achieve the adaptation of the quality inspection and the speed at which the objects to be inspected are transmitted, thereby avoiding stacking of objects to be inspected.

[0242] In a specific implementation, the transmission speed of the objects to be inspected on the target production line can be adjusted to be consistent with the quality inspection speed, or at least to have a small difference between the two. When adjusting the transmission speed, the quality control system can send instructions to the conveyor belt control device, causing the conveyor belt control device to adjust the conveyor belt speed according to the instructions. The conveyor belt control device can be a motor, pulley, etc., and is not limited here.

[0243] By adopting this embodiment, the transmission speed of the objects to be inspected produced on the target production line can be controlled according to the speed of quality inspection, so as to achieve a match between the quality inspection speed and the speed of transporting the objects to be inspected to the quality inspection site, thereby reducing the accumulation of objects to be inspected at the quality inspection site, so that when quality inspection is carried out in depth on the production line, it can be closely linked to the production line.

[0244] Next, the quality inspection shown in this disclosure will be described.

[0245] As described above, quality inspection may refer to the detection of defects in the area to be inspected of the object to be inspected, wherein the detection may be performed using a defect detection model. In some embodiments, quality inspection may be performed by combining coarse inspection and fine inspection, wherein the accuracy of coarse inspection is lower than that of fine inspection. Thus, the efficiency of quality inspection may be ensured through coarse inspection, while the accuracy of quality inspection may be improved through fine inspection.

[0246] Referring to Figure 5, a schematic diagram of the strategy architecture for conducting quality inspection under the quality control system shown in Figure 2 is shown. Specifically, as described in the above-mentioned embodiment 1, different quality inspection servers can be configured with defect detection models with different detection accuracy, and the quality inspection strategy can be set by the material configuration unit in the detection module shown in Figure 2. The material configuration unit can provide strategy configuration options in the visual interface, such as three configuration options: fine inspection, coarse inspection, and fine inspection + coarse inspection. The user can trigger the quality inspection function on the front-end browser interface and enter the quality inspection window interface. Then, the quality inspection strategy setting button is triggered to use the strategy configuration options provided by the material configuration unit. Among them, a strategy configuration option can be associated with a corresponding quality inspection call logic program in the background. The quality inspection call logic program can use computer instructions to build the software and hardware environment under each quality inspection strategy, as well as the communication of hardware quality inspection, thereby guiding the flow of images collected from the objects to be inspected on the target production line, as well as the processing accuracy.

[0247] In a specific implementation, when performing quality inspection, at least one image may be input into a first model to perform a first quality inspection on the object to be inspected; and when the result of the first quality inspection indicates that the quality inspection has failed, at least one image capture is performed again on the object to be inspected, and the recaptured image is input into a second model to perform a second quality inspection on the object to be inspected;

[0248] The quality inspection accuracy of the first quality inspection is lower than that of the second quality inspection.

[0249] In this embodiment, in order to improve the efficiency of quality inspection, when using the first model for rough inspection, the image acquisition device can be instructed to capture images of the object to be inspected according to the preset first resolution and first acquisition accuracy, wherein the preset first resolution can be a lower resolution, and the first acquisition accuracy can be a lower accuracy. The acquisition accuracy can refer to the focal length corresponding to the camera's image acquisition at the position of the test area, so as to control the degree of detail of the image acquisition. Since the acquisition resolution is low and the acquisition accuracy is also low, fast acquisition can be achieved. At the same time, images with lower resolution and lower acquisition accuracy can reduce the computational complexity of the first model, thereby improving the efficiency of defect detection and then improving the quality inspection efficiency.

[0250] Among them, if the first model detects that an object to be inspected fails the quality inspection, at this time, the image of the object to be inspected can be re-captured. When re-capturing the image, the image acquisition device can be instructed to capture the image of the position where the defect exists on the object to be inspected, or the image acquisition device can be instructed to capture the image of the entire area to be inspected of the object to be inspected again. Of course, in order to improve efficiency, the former can be preferred.

[0251] Among them, when re-acquisition of images, it can be carried out according to the preset second resolution and second acquisition accuracy. The preset second resolution is greater than the preset first resolution, and the second acquisition accuracy is higher than the first acquisition accuracy. This can improve the resolution and detail of the image in the fine inspection, and send the acquired image to the second model. Since the resolution and accuracy of the image sent to the second model are higher, the second model can perform higher-precision defect detection on the object to be inspected, thereby ensuring the accuracy of quality inspection.

[0252] Among them, the image acquisition device used for image acquisition during rough inspection can be different from the image acquisition device used for image acquisition during fine inspection; the first model and the second model can be respectively configured in different quality inspection servers.

[0253] Among them, as shown in Figure 5, the first model can be configured in the first quality inspection server ① (coarse inspection server), and the second model can be configured in the second quality inspection server ② (fine inspection server). The coarse inspection server can be connected to the coarse inspection camera, and the fine inspection server can be connected to the fine inspection camera. When the coarse inspection server outputs that the quality inspection of the object to be inspected fails, it can send instructions to the fine inspection server and the fine inspection camera to instruct the fine inspection camera to re-capture images of the object to be inspected according to the second resolution and the second accuracy, and send the captured images to the fine inspection server for fine inspection.

[0254] In some examples, both the rough inspection server and the fine inspection server can save the quality inspection results of the object to be inspected and feed them back to the control object and the data server, so that the control object and the data server can perform statistical analysis on the quality inspection results.

[0255] Below, the construction of the defect detection model used in the quality inspection of the present disclosure is described, with reference to Figures 6 and 7. Figure 6 shows a flowchart of the construction steps of the defect detection model of the present disclosure, and Figure 7 shows a schematic diagram of the third configuration interface when constructing the defect detection model. As shown in Figure 6, it specifically includes the following steps:

[0256] Step S601: In response to a triggered model building function, displaying a third configuration interface; wherein the third configuration interface includes at least one trigger control, and different trigger controls are used to trigger different configuration windows in the model building process;

[0257] Step S602: Determine the configuration options selected in different configuration windows;

[0258] Step S603: Based on the preset configuration parameters corresponding to the selected configuration option, a defect detection model is trained using the pre-stored first data set as a training sample.

[0259] In this embodiment, in addition to the above-mentioned quality inspection function button and debugging function button, the system interface may also include a model setting button. By triggering the model setting button, you can enter the model setting window interface, wherein the model setting window interface can correspond to the model management module shown in Figure 2. The model building function is triggered in this window interface to enter the third configuration interface. At least one trigger control can be controlled in the third configuration interface. When the trigger control is triggered, for example, by clicking, double-clicking, or touching, you can enter the corresponding configuration window to enter the corresponding construction stage in the model building process and set parameters for the construction stage.

[0260] Generally speaking, the model building process can be divided into: model creation stage, data upload stage, data labeling stage, training stage and publishing stage; among them, the model creation stage is used to create an initial neural network model, such as selecting a neural network model with a certain network structure; in the data upload stage, it is generally necessary to upload data samples for model training. The data samples are labeled samples or unlabeled samples, or they can include both labeled and unlabeled samples. The data labeling stage is used to label the uploaded unlabeled data; the training stage is to train the created neural network model based on the labeled data samples to update the parameters in the neural network; the publishing stage is used to publish the trained model, such as deploying it to the cloud or deploying it to a server in a local area network.

[0261] Among them, in the configuration window, multiple configuration options can be preset for users to choose. Different configuration options can correspond to different preset configuration parameters. In this way, users only need to select a configuration option according to their needs to start training the model without the need for users to input model parameters that require a high level of professionalism. For example, in the model creation stage, the configuration window it enters can provide the following configuration options: lightweight model, high-precision model, multi-task model, etc. Each configuration option can be associated with a preset neural network, wherein the associated neural network can be designed in advance by technical personnel and stored in the quality control system. In this way, users only need to select a model according to quality inspection requirements, without the need for users to build the model themselves, thereby reducing the technical threshold for users to use this function.

[0262] As shown in Figure 7, four trigger controls are shown: model creation, data upload and annotation, automated training, and model verification and release. When automated training is triggered, the configuration window it enters contains multiple configuration options. For example, in the deployment mode, it can include: open platform deployment, open platform local deployment;

[0263] The training machine includes options such as CPU servers, lightweight GPU servers, and high-performance GPU servers. As shown in Figure 2, the infrastructure layer includes multiple servers, which may include quality inspection servers, data servers, and industrial control servers. In this example, training servers with various hardware configurations may be included, such as CPU servers, lightweight GPU servers, and high-performance GPU servers. By selecting a certain type of configuration in the configuration options of the training machine, the corresponding training server in the infrastructure layer may be called, thereby completing the model training through the called training server, and then publishing it to the selected quality inspection server.

[0264] The algorithm configuration can include configuration options such as high precision, balance, and high performance. Each configuration option corresponds to preset configuration parameters, so users only need to select according to their needs. In subsequent training and release, data sets can be obtained, neural networks can be trained to obtain defect detection models, and defect detection models can be automatically released based on the preset configuration parameters corresponding to the configuration options selected by the user.

[0265] Among them, the first data set obtained can also be obtained based on the configuration options selected by the user in the window interface for data upload and annotation. Generally speaking, the data sets for training the defect detection model can be pre-stored in the quality control system. These data sets can be obtained by pre-annotation by the user, which can be called the initial data sets for training the defect detection model. In some embodiments, different data sets can be constructed for different production lines, and the first data set can be the data set selected by the user according to the production line.

[0266] In practice, users who build models only need to select a dataset and do not need to label the dataset. Of course, in some examples, users who build models can also simply label the data, which is not limited here.

[0267] Accordingly, in the process of quality inspection, as the quality inspection progresses, the more data is accumulated, that is, a larger number of images of the objects to be inspected can be obtained, so that a large amount of labeled data can be obtained based on the quality inspection results of these objects to be inspected, so as to update the defect detection model. Of course, it can also be used to construct defect detection models for detecting other types of defects.

[0268] In some embodiments, after quality inspection is performed on multiple objects to be inspected produced on the target production line, a second data set can be obtained in response to a triggered model update function; and the defect detection model can be updated using the second data set as a training sample; then, the defect detection model performing the quality inspection at the current moment can be replaced with the updated defect detection model.

[0269] The second data set includes a plurality of target image samples, and the target image samples include areas where defective areas of the inspected object to be inspected are re-marked.

[0270] In this embodiment, the second data set may include target image samples of objects to be inspected that have been quality-inspected on the target production line, so that the generalization performance of the updated defect detection model can be improved; specifically, it may also be image samples of objects to be inspected that have been misdetected, so that difficult samples can be constructed to improve the detection accuracy of the updated defect detection model.

[0271] The target image sample may include a marking box for marking the defect position. The marking box may be marked by the defect detection model during defect detection or may be marked manually, which is not limited here.

[0272] As shown in FIG14 , the triggered model update function may be triggered in the model setting window interface, for example, after the “model update” button is triggered, this function is triggered; in some examples, in response to the triggering of the model update function, the defect detection model may be updated using the second data set using the configuration options selected by the user in the third configuration interface for the automated training phase.

[0273] Of course, in some other examples, a corresponding configuration interface can also be displayed in response to the triggering of the model update function. The configuration interface can include multiple configuration options related to the model update. The user can select the corresponding configuration option according to the needs. The quality control system can then start using the second data set to update the defect detection model according to the preset configuration parameters corresponding to the selected configuration option.

[0274] After updating the defect detection model, an updated defect detection model can be obtained. In practice, the updated defect detection model can be automatically deployed to the target production line according to the configuration option selected by the user in the model deployment method in the third configuration interface, so as to replace the defect detection model currently being inspected with the updated defect detection model.

[0275] In some embodiments, if the target image sample in the second data set is an image sample of the object to be inspected that was falsely detected, it is necessary to mark the defect location of the falsely detected image to obtain the second data set; in practice, an interface can be provided for the user to mark the second data set. At the same time, the user can also verify, correct and mark the quality inspection results in this interface.

[0276] 8 and 9 , FIG8 shows an interface for obtaining the second data set, and FIG9 shows a schematic flow chart of steps for obtaining the second data set. As shown in FIG9 , the steps may include:

[0277] Step S901: in response to a triggered data annotation function, quality inspection results of a plurality of objects to be inspected that have undergone quality inspection are obtained, and the plurality of quality inspection results are displayed.

[0278] In this embodiment, the triggered data annotation function can be triggered in the quality inspection window interface or in the model setting window interface. That is to say, in practice, the data annotation function can be set in both interfaces to correct, annotate and verify the quality inspection results.

[0279] Among them, after triggering the data labeling function, the quality control system can obtain the quality inspection results of multiple objects to be inspected that have undergone quality inspection from the quality inspection server or data server corresponding to the target production line, and display multiple quality inspection results. When displaying multiple quality inspection results, they can be displayed in the quality inspection result display interface. The quality inspection result display interface can be understood as a window interface corresponding to the data labeling function.

[0280] When displaying multiple quality inspection results, the quality inspection results of each object to be inspected may be displayed in sequence according to the time sequence of the inspection of the object to be inspected; the displayed quality inspection results may be the results of whether the quality inspection is passed.

[0281] In some examples, as shown in FIG8 , the displayed quality inspection results may also include each image of the inspected object and whether the image passed defect detection. In still other examples, the displayed quality inspection results may also include a schematic diagram marking all defective areas detected in the inspected object that failed quality inspection, such as the one shown in the lower right corner of FIG8 , which is a schematic diagram of the locations where defects were detected on the inspected object.

[0282] Specifically, the visual interface of Figure 8 is introduced: it mainly includes the following functions:

[0283] Image large-scale browsing: The main space on the page is the large-scale image of the current image, which can be zoomed in and out using the mouse wheel. The diagram in the lower right corner represents the current zoom ratio of the original image. In specific implementation, when the large-scale browsing of an image is detected on the front-end browser page, the back-end generates a large-scale browsing instruction message, transmits the data packet containing the instruction message via the message queue to the display server, and the display server converts it into a large image using pre-set protocols and codes and presents it on the front-end.

[0284] Whether to display defect marks: Use the toggle switch in the upper left corner of the large image to choose whether to mark the defect location on the image. If this switch is turned on, the system will display an identification frame at the defect location. In specific implementation, when the display defect mark is detected to be triggered in the front-end browser page, the background will generate instruction information for displaying the defect mark, and the data packet containing the instruction information will be transmitted to the display server via the message queue. The display server will obtain the identification frame coordinates of the defect location of each image through the pre-set protocol and code, and render the identification frame in the image presented on the front end according to the identification frame coordinates.

[0285] Mouse coordinates display, the upper right corner of the picture shows the current mouse coordinates;

[0286] Image list: For each product material, the inspection machine will take a series of pictures, and the image list will be displayed at the bottom of the page, linked to the large image and the schematic diagram on the lower right shelf; in the specific implementation, when the image list display is detected to be triggered in the front-end browser page, the background will generate instruction information for displaying defect marks, and the data packet containing the instruction information will be transmitted to the display server via the message queue. The display server will obtain the requested multiple images from the data server or quality inspection server, and the obtained multiple images will be presented in the form of a list in the front-end according to the preset protocol and code.

[0287] The list order corresponds to the shooting order and also reflects the relative position at the edge of the material. The list order and shooting order can be pre-configured into the backend protocol and code so that when multiple images are acquired, they are displayed in the list order and shooting order.

[0288] Among them, each image will be accompanied by NG / OK status distinction for users to distinguish; when the function with status distinction is detected to be triggered in the front-end browser page, the background will generate status acquisition instruction information, and the data packet containing the instruction information will be transmitted to the display server via the message queue. The display server will obtain the status information of each of the multiple images (whether the quality inspection passed or failed) from the data server or the quality inspection server, and the obtained status information will be rendered in the multiple images on the front end according to the preset protocol and code.

[0289] The input box and drop-down menu in the upper left corner of the image list support image retrieval and classification retrieval;

[0290] Manual judgment: Manually judge whether there are defects in the image using the OK and NG buttons. If the machine detection result is found to be incorrect, the image data can be re-labeled. The labeled image can be used in the next version of the model iteration by the user, that is, as the second data set.

[0291] Step S902: In response to an error correction operation triggered for a target quality inspection result among the multiple quality inspection results, at least one target image corresponding to the target quality inspection result is displayed, where defective areas that fail quality inspection are marked in the target image.

[0292] In this embodiment, the target quality inspection result corresponds to an object to be inspected. Since multiple images can be captured for an object to be inspected, multiple target images corresponding to the target quality inspection result and the result of whether the target image passes the defect detection can be displayed in the quality inspection result display interface.

[0293] Among them, whether the target image passes the defect detection result can be marked by an identifier so that the user can understand it intuitively.

[0294] In some embodiments, in response to a target image being selected, the selected target image and the defective areas marked in the target image can be magnified and displayed in the annotation window. For example, as shown in FIG8 , when the leftmost image is selected, the leftmost image can be magnified and displayed, and the defective areas detected during the quality inspection can be displayed.

[0295] Step S903: In response to the modification operation on the defective area, the modified defective area is marked on the target image.

[0296] In this embodiment, the defective area of ​​the selected target image can be modified. Specifically, the quality inspection result display interface can provide a marking tool, such as a rectangular frame, and the user can use the marking tool to re-mark the correct defective area. Then, the defective area detected during the quality inspection in the selected target image can be removed, thereby realizing verification and error correction of the quality inspection.

[0297] Step S904: adding the target image marked with the modified defect area as a target image sample to the second data set.

[0298] Accordingly, in response to the user saving the modified defect area, the target image marked with the modified defect area can be saved in the second data set.

[0299] When this implementation method is adopted, during the process of the target production line producing the objects to be inspected and conducting quality inspection on the objects to be inspected, images and quality inspection results of the objects to be inspected can be continuously collected as a second data set to update the defect detection model, thereby improving the accuracy of the defect detection model.

[0300] As shown in the above embodiment, the defect detection model may include a first model and a second model. When the defect detection model is updated, the first model may be updated using the second data set, or the second model may be updated, or both models may be updated.

[0301] In this case, the second data set may include a data set from the coarse inspection server and a data set from the fine inspection server. When updating the first model, the first model may be updated using the data set obtained by performing defect inspection on the object under inspection using the first model and the results, such as the data set from the coarse inspection server. Similarly, when updating the second model, the second model may be updated using the data set obtained by performing defect inspection on the object under inspection using the second model and the results, such as the data set from the fine inspection server.

[0302] Therefore, in the quality inspection combining rough inspection and fine inspection, the first model of rough inspection and the second model of fine inspection can be updated in a targeted manner, so as to achieve the goal of targeted updating and improving updating efficiency.

[0303] In some embodiments, quality inspection may include quality inspection of multiple defect types of the object to be inspected, such as the multi-task model mentioned above, wherein the defect detection model includes detection branches corresponding to each of the multiple defect types, and different detection branches correspond to different defect types; accordingly, when updating the defect detection model, the parameters of the detection branch corresponding to the defect type in the defect detection model can be updated according to the defect type in which an error occurs in the quality inspection, while the parameters of other detection branches are retained. In this way, the updating efficiency of the defect detection model can be improved, thereby matching the output efficiency of the objects to be inspected of the target production line.

[0304] In a specific implementation, in response to a selection operation of at least one defect type from a plurality of defect types, a labeled subset corresponding to the selected defect type is obtained from the second data set, wherein the defect area labeled on the target image sample in the labeled subset corresponds to the defect type;

[0305] Then, the detection branch corresponding to the selected defect type in the defect detection model is updated using the labeled subset.

[0306] In this embodiment, multiple defect types of the object to be inspected can be detected. It should be noted that the multiple defect types may refer to the same type of defects that may appear in the same area. For example, taking a display substrate as an example, when cutting the display substrate, the types of defects that may appear on the cutting edge include: uneven cutting, defects in cutting, and edge cracks caused by cutting. In practice, the above-mentioned defects need to be detected during quality inspection.

[0307] In this embodiment, a defect detection model can be used to detect multiple defect types in the test area of ​​the object to be inspected. Specifically, the image collected from the object to be inspected can be input into multiple detection branches after feature extraction, and the extracted features are processed by different detection branches to determine whether the object to be inspected has defects of the corresponding type.

[0308] In practice, if it is determined that the detection accuracy of a certain defect type is not high, the detection branch for this defect type can be updated. In practice, a labeling subset corresponding to the defect type can be collected, and the defect area marked in the target image sample included in the labeling subset corresponds to the defect type. For example, if the defect type required to be updated is a defect of the defect type, image samples of the object to be inspected with the defect defect can be collected, and the defect position and defect label can be marked to obtain the labeling subset of the defect type.

[0309] Then, the labeled subset can be used to update the parameters of the detection branches corresponding to at least one defect type in the defect detection model. In practice, when updating, as described above, the "Model Update" button can be triggered in the model settings window interface to trigger the configuration option to update one or several detection branches of the defect detection model. Then, according to the update logic corresponding to the configuration option, only some of the detection branches of the defect detection model can be updated. This can reduce the amount of calculation during the update, save computer resources, and improve the efficiency and targetedness of the update of the defect detection model.

[0310] In some further embodiments, the defect detection model of the target production line can be migrated to other production lines related to the target production line, so as to use the defect detection model of the target production line and the corresponding configuration parameters to detect defects in the objects to be inspected on the other production lines, wherein the defect types of the objects to be inspected to be detected on the other production lines are the same or similar to the defect types of the objects to be inspected to be detected on the target production line.

[0311] For example, if production line 1 performs defect detection on the cut edges of array substrates in display panels, while production line 2 performs defect detection on the cut edges of color filter substrates, the defect detection model for production line 1 can be migrated to production line 2 for defect detection. The array substrate and color filter substrate are the two cell-aligned panels in a liquid crystal display (LCD).

[0312] In a specific implementation, when it is detected that an associated production line associated with the target production line is running, the configuration parameters can be associated with the associated production line in response to the triggered parameter migration function; then, based on the configuration parameters, the defect detection model is used to perform quality inspection on multiple objects to be inspected produced on the associated production line;

[0313] The defects of the objects to be inspected during quality inspection on the associated production line are identical or similar to the defects of the objects to be inspected during quality inspection on the target production line. Similarity means that the degree of similarity between the defects of the objects to be inspected on the associated production line and the defects of the objects to be inspected on the target production line is less than a preset similarity, such as a chip defect or a chipped corner defect.

[0314] In this embodiment, the associated production line is related to the target production line, and the defect types targeted by the defect detection to be performed on the object to be inspected are the same or similar. Therefore, the configuration parameters and defect detection model used by the target production line when performing quality inspection on the object to be inspected can be applied to the associated production line to perform quality inspection on the object to be inspected.

[0315] The parameter migration function can be triggered by the user or automatically when the associated production line is detected to be online. After the parameter migration function is triggered, the configuration parameters of the target production line can be obtained. It is understood that these configuration parameters may include the operating parameters of the equipment used for quality inspection in the associated production line, as well as the threshold parameters of the defect detection model or the operating parameters of the server.

[0316] Then, the working parameters of the quality inspection equipment can be sent to the corresponding equipment, so that the equipment starts to collect images of the objects to be inspected produced on the associated production line based on the working parameters, and sends the collected images to the server corresponding to the associated production line. The server starts to perform defect detection on the objects to be inspected based on the threshold parameters of the defect detection model and its own working parameters, and determines the quality inspection results according to the results of the defect detection.

[0317] When adopting this implementation method, the parameter migration function of the quality control system can be used to reuse the quality inspection logic of the existing production line without the user having to reset the quality inspection logic for the associated production line. Therefore, through parameter migration, intercommunication and reuse between production lines can be achieved, thereby making full use of resources and reducing workers' labor expenditure costs.

[0318] Accordingly, in some embodiments, if the defects of the objects to be inspected targeted by quality inspection in the associated production line are similar to the defects of the objects to be inspected targeted by quality inspection in the target production line, the representations are not exactly the same defect types. In practice, the defect detection model of the target production line can be updated according to the image samples of the objects to be inspected produced on the associated production line, thereby obtaining a defect detection model adapted to the associated production line to improve the quality inspection accuracy of the objects to be inspected produced on the associated production line.

[0319] In a specific implementation, a target data set corresponding to an associated production line may be obtained, the target data set including image samples obtained by image acquisition and annotation of multiple objects to be inspected on the associated production line; and in response to a triggered model migration function, the target data set is used to update the defect detection model to obtain a migration model;

[0320] Accordingly, when quality inspection is performed on multiple objects to be inspected produced on the associated production line using the defect detection model based on the configuration parameters, quality inspection can be performed on multiple objects to be inspected produced on the associated production line using the migration model based on the configuration parameters.

[0321] In this embodiment, the image samples in the target data set can be obtained by collecting images of multiple objects to be inspected on the associated production line, and the defect locations and defect types are marked in the image samples; therefore, after triggering the "Model Update" button in the model setting window interface, the target data set can be used to automatically update the defect detection model. At this time, since the image samples in the target data set used are marked with defects on the associated production line, after the defect detection model is updated, the updated model can be called a migration model, and the migration model can perform defect detection on the objects to be inspected produced on the associated production line.

[0322] For example, the defect of the object to be inspected on the associated production line is a missing defect, and the defect of the object to be inspected on the target production line is a missing corner defect; then the defect marked by the image sample in the target data set is a missing corner defect. After the defect detection model is trained using the target data set, the defect detection model can improve the detection accuracy of the object to be inspected with missing corner defects, so there is no need to rebuild the detection model for missing corner defects, reducing the quality inspection configuration work for the associated production line.

[0323] Next, how to debug the target production line is described.

[0324] Among them, when making adjustments, it is necessary to base the adjustment on the quality inspection results of the objects to be inspected. In some embodiments, the quality inspection results of the objects to be inspected that have been quality inspected can be automatically counted. The statistics may include the statistics of the number of objects to be inspected that have passed the quality inspection, the statistics of the time periods during which the objects to be inspected that have failed the quality inspection have been quality inspected, the statistics of the locations of defects in the objects to be inspected that have failed the quality inspection, the statistics of the types of defects, etc., thereby providing a basis for debugging the target production line based on the distribution of quantity, time period distribution, defect location and defect type.

[0325] 10a to 10c and 11, FIG10a to 10c show schematic diagrams of interfaces for performing statistics on quality inspection results, and FIG11 shows a flow chart of steps for performing statistics on quality inspection results. As shown in FIG11, the steps may include:

[0326] Step S1101: In response to the triggered quality inspection statistics function, statistics are collected on the quality inspection results of multiple objects to be inspected to obtain statistical results;

[0327] Step S1102: generating a statistical chart based on the statistical results;

[0328] Step S1103: Send the statistical chart to the control object; wherein the statistical chart includes at least one of a pie chart, a bar chart, a line chart and a heat map.

[0329] The quality inspection statistics function can also be triggered from within the quality inspection window interface. Once this function is triggered, the statistical results are displayed within the control object, which can be in the form of statistical charts. As mentioned above, the control object is the window interface within the industrial control system, inspection module, or quality control system where controls are set.

[0330] Among them, the statistical chart can be fed back to the control object corresponding to the target production line for display. The display interface is shown in Figure 10a. In this interface, it can include a main interface and an auxiliary interface for displaying statistical charts. The main interface can be used to display the quality inspection results of a specific object to be inspected, including an image of the location of defects in the object to be inspected; the auxiliary interface is used to display statistical charts, which display a line graph. As shown in Figure 10b, a schematic diagram of the auxiliary interface in Figure 10a is shown. In addition to displaying statistical charts, it can also display statistical dimensions, categories and other information related to the object to be inspected. As shown in Figure 10c, a schematic diagram of the auxiliary interface in Figure 10a is shown, which shows a heat map of the quality inspection results. Through the heat map, you can intuitively understand the location distribution of more defects in a batch of objects to be inspected.

[0331] In Figure 10a, the left side shows the real-time image captured by the current camera, and the right side shows the current recipe information. Clicking the parameter configuration button takes you to the system parameter configuration category strategy page, which is the second configuration interface. The defective list for the day shows the defective (NG) list of this inspection machine for the day. Clicking the Panel ID displays the image on the right. The data statistics show the number of images taken and completed today. The defective data statistics support the statistics of input volume, defective ratio and detection rate. By selecting different time ranges and periods, it can be displayed as a pie chart or a line chart (detection rate: number of NG points during this period / total number of images).

[0332] Among them, for heat maps, what is provided is a heat map of defect locations. Of course, it can also provide data analysis visualization charts of defects over time, product models and other dimensions, allowing users to extend the analysis from the regular phenomena discovered by single-point defect detection to the upstream and downstream of the entire process, discover systematic problems in the production process, optimize the process, and create greater business value.

[0333] In some embodiments, when performing statistics on the quality inspection results, at least one of the following statistical methods may be used:

[0334] Dimension 1: Statistics on the quality inspection results of multiple objects to be inspected according to the location of defects;

[0335] Dimension 2: Statistics on the quality inspection results of multiple objects to be inspected according to the time period in which the objects to be inspected were produced;

[0336] Dimension 3: Statistics are collected on the quality inspection results of multiple objects to be inspected according to the defect type; quality inspection is used to detect multiple defect types of the objects to be inspected.

[0337] Among them, when statistics are performed according to the defect location, the objects to be inspected with defects at the same location can be classified to obtain the number of objects to be inspected, and the number of objects to be inspected is used as the statistical result of the location, so that statistical results of multiple locations can be obtained;

[0338] Among them, when statistics are performed according to the time period when the objects to be inspected are produced, the number of objects to be inspected that passed the quality inspection and those that failed the quality inspection in the same production time period can be counted to obtain the quality inspection statistical results related to the time period;

[0339] Among them, when statistics are performed according to the defect type, the objects to be inspected with the same defect can be classified to obtain the number of objects to be inspected, and the number of objects to be inspected is used as the statistical result of the defect of this type, so that statistical results of multiple defect types can be obtained;

[0340] In practice, the above three dimensions can be combined with each other to obtain statistical results related to quality inspection and defect location, defect type and time period. In this way, the process quality of the target production line can be comprehensively reflected from multiple dimensions.

[0341] Among them, in response to the second preset condition triggered on the control object, when the target production line is controlled based on the quality inspection results of at least one object to be inspected, since the statistical chart can feedback the distribution of the quality inspection results in quantity, time period distribution, defect location, and defect type, in some examples, it is possible to directly respond to the preset operation triggered on a certain area of ​​the statistical chart, and display the quality inspection information corresponding to the area to indicate the distribution of the quantity, time period distribution and defect location represented by the statistical area, so that the user can input the corresponding debugging parameters according to the distribution of the quantity, time period distribution and defect location represented by the statistical area to debug the target production line.

[0342] 12 shows a schematic flow chart of steps for regulating a target production line. As shown in FIG12 , in some examples, the debugging may include the following steps:

[0343] Step S1201: when the second preset condition is a preset operation triggered on the control object, displaying quality inspection information in response to the preset operation;

[0344] The quality inspection information includes at least one of a target defect location, a target time period, and a target defect type; wherein the number of objects to be inspected having defects at the target defect location exceeds a first preset number, the number of objects to be inspected that fail quality inspection during the target time period exceeds a second preset number, and the number of objects to be inspected having defects of the target defect type exceeds a third preset number;

[0345] Step S1202: receiving debugging parameters entered for quality inspection information, where the debugging parameters include operating parameters of process equipment on the target production line;

[0346] Step S1203: Regulating the process equipment based on the debugging parameters.

[0347] In this embodiment, the triggering of the preset operation may indicate that the user wants to know the statistical results of the quality inspection problems. Therefore, the quality control system may analyze the statistical results of the quality inspection results and display the quality inspection information.

[0348] The quality inspection information may include at least one of a target defect location, a target time period, and a target defect type. As described above, the target defect location refers to a location where the number of objects to be inspected that have defects exceeds a first preset number, indicating that a process failure has occurred at that location, resulting in a low quality inspection pass rate.

[0349] The target period refers to the period during which the objects to be inspected are more likely to fail the quality inspection;

[0350] The target defect type means that a large number of objects to be inspected have defects of this defect type, indicating that the corresponding process equipment has failed.

[0351] Among them, the above-mentioned first preset quantity, second preset quantity and third preset quantity can be set according to the product yield corresponding to the target production line, and will not be repeated here.

[0352] Among them, the quality inspection information may include any one, any two or all three of the above. Then, the user can enter the debugging parameters according to the displayed quality inspection information. The quality control system can send the debugging parameters to the corresponding process equipment to regulate the process equipment; or send them to the industrial control system of the target production line to regulate the process flow.

[0353] 13 and 14 , FIG13 shows a complete flow chart of quality control according to the present disclosure, and FIG14 shows a schematic diagram of the interface settings of the quality control system according to the present disclosure. In conjunction with FIG13 and FIG14 , the quality control method according to the present disclosure is exemplarily described by taking a target production line as a production line for cutting display panels as an example:

[0354] S1: First, the user logs in to the quality control system, enters the system interface, clicks the quality inspection function button, enters the quality inspection window interface, and triggers the quality inspection configuration in the window interface, thereby entering the first configuration interface, which is shown in Figure 4;

[0355] S2: The quality control system receives configuration parameters entered by the user for multiple configuration items (first configuration item and second configuration item) on the first configuration interface, including operating parameters of the camera, PLC, trigger, and server, and threshold parameters of the model; and associates these configuration parameters with the target production line;

[0356] S3: Next, the user can trigger the model setting to enter the model setting window interface, which may include model building and model update functions. Specifically, the model building is first triggered to enter the window interface shown in FIG7 ;

[0357] S4: In the window interface shown in FIG7 , different trigger controls are displayed. When different trigger controls are triggered, corresponding configuration options are displayed. After the user selects the corresponding configuration option according to the requirements, the quality control system begins to automatically train the defect detection model used in the target production line.

[0358] S5: After the model is published and the configuration parameters are set, return to the quality inspection window interface and trigger the quality inspection in the window interface. Then, the quality mask can enable the system to start quality inspection of the devices produced on the target production line according to the associated configuration parameters and the constructed defect detection model;

[0359] S6: During the quality inspection process, if the user clicks on the quality inspection debug button in the quality inspection window, the quality inspection debug function is triggered, and the target configuration items, the quality inspection results of the currently inspected devices, the quality inspection quantity, etc. are displayed on the second configuration interface. Statistical analysis is also performed on the devices that have passed and failed the quality inspection.

[0360] S7: Based on the displayed quality inspection statistical analysis results, the configuration parameters in the target configuration item are modified, and quality inspection after the current moment is performed based on the modified configuration parameters;

[0361] S8: After a period of quality inspection, the control function is triggered on the system interface, and the control window interface is entered. In this interface, click on the quality inspection statistics, and the interface shown in Figure 10a will be entered; the statistical results of the quality inspection are intuitively displayed;

[0362] S9: Next, when production line control is triggered in the control window interface, that is, the second preset condition is triggered, quality inspection information begins to be displayed. This quality inspection information may be obtained based on further analysis of quality inspection statistics, and includes at least one of a target defect location, a target time period, and a target defect type.

[0363] S10: Enter debugging parameters in the production line control interface, and the quality control system sends the debugging parameters to the process equipment on the target production line, thereby realizing the control of the process equipment;

[0364] S11: When the new associated production line associated with the target production line is put into operation, the production line migration function is triggered on the system interface to enter the migration window interface;

[0365] S12; The migration window interface may include two functions: model migration and parameter migration. In practice, after selecting the two functions, and selecting the target production line to be migrated and the associated production line to be migrated, the configuration parameters and defect detection model of the target production line are associated with the associated production line, so that the associated production line directly uses the configuration parameters and defect detection model of the target production line to perform quality inspection of the device.

[0366] Of course, the above process, as shown in Figure 13, requires corresponding permissions to operate. The determination of permissions can be performed by the user information management unit. For example, each corresponding function button in Figure 14 can be verified to have the corresponding permission before entering the corresponding window interface. Please see the description of the above embodiment for details, which will not be repeated here.

[0367] Based on the same inventive concept, the present disclosure further provides a quality control method. The quality control method can be solidified into a computer-readable storage medium and configured on a computer device to perform quality control. Specifically, referring to FIG15 , a flow chart of the steps of a quality control method for a production line of the present disclosure is shown. As shown in FIG15 , the method can specifically include the following steps:

[0368] Step S1501: performing at least one image acquisition on the object to be inspected produced on the target production line;

[0369] Step S1502: inputting the collected at least one image into the defect detection model to perform quality inspection on the object to be inspected through the defect detection model;

[0370] Step S1503: Based on the quality inspection result of at least one object to be inspected on the target production line, the target production line is regulated.

[0371] In this embodiment, in response to the target production line being put into operation, at least one image acquisition of the objects to be inspected prepared on the target production line can be started. During the image acquisition process, the image acquisition device can acquire images according to a pre-configured acquisition frequency and resolution.

[0372] The quality inspection of the object to be inspected by the defect detection model can refer to the description of the above embodiment and will not be repeated here.

[0373] The target production line can be regulated at preset intervals based on the quality inspection results of the inspected objects, or the target production line can be automatically regulated when the number of inspected objects that fail the quality inspection exceeds a preset number.

[0374] Among them, during regulation, the quality inspection results can be statistically analyzed according to the output time period and defect location distribution. Generally speaking, when the quality inspection results are statistically analyzed according to the output time period, it can be determined which output time period has more objects to be inspected that fail the quality inspection, and which output time period has fewer objects to be inspected that fail the quality inspection, so that it can be determined that the target production line is prone to process failures in certain time periods, so that the process flow of this time period can be regulated in a targeted manner; Among them, when the statistical analysis is performed according to the defect location distribution, it can be determined which position of the objects to be inspected has more defects and which position has fewer defects. Generally speaking, the defect location is related to the output of the objects to be inspected by the target production line. For example, when cutting a display substrate, if the statistical analysis results of the defect location are displayed in the upper left corner of the display substrate, a large number of objects to be inspected will fail the defect detection, indicating that the cutting process equipment corresponding to the upper left corner needs to be debugged.

[0375] Therefore, the target production line can be regulated based on the statistical analysis results of the time period and / or defect location distribution of the objects to be inspected. During debugging, the process flow corresponding to the output time period with a large number of objects to be inspected that failed the quality inspection can be regulated based on the statistical analysis results, and the process flow corresponding to the locations with more defect detection failures can be regulated. In this way, the quality inspection of the objects to be inspected can be fed back to the target production line to regulate the target production line, so that the target production line can continue to maintain a higher quality process, thereby improving the quality level of the objects to be inspected it produces, and then improving the quality control level of the target production line.

[0376] By adopting this embodiment, since the quality inspection of the objects to be inspected on the target production line is carried out, and the target production line is regulated based on the quality inspection results, the quality inspection of the objects to be inspected disclosed in this disclosure can be deeply implemented in the production line for preparing the objects to be inspected, so as to regulate the process flow of the production line in combination with the quality inspection results of the objects to be inspected, thereby solving the yield problem of the objects to be inspected from the source of preparing the objects to be inspected, such as process parameters, process equipment, etc., and reducing the production costs consumed by discarding the defective objects to be inspected. On the other hand, since the target production line can be regulated based on the statistical analysis results of the quality inspection results in the output period and defect location, the target production line can be regulated in a targeted manner, thereby improving the regulation efficiency.

[0377] In some embodiments, when performing image acquisition, image acquisition can be performed according to pre-set configuration parameters, and quality inspection can be performed on the object to be inspected through the defect detection model according to pre-set threshold parameters corresponding to the defect detection model.

[0378] Accordingly, the configuration parameters corresponding to the target production line can be obtained, including the operating parameters of the equipment used for quality inspection and the threshold parameters of the defect detection model;

[0379] When capturing an image of an object to be inspected prepared on a target production line at least once, the image of the object to be inspected may be captured at least once based on the working parameters;

[0380] When inputting at least one collected image into a defect detection model to perform quality inspection on the object to be inspected through the defect detection model, at least one collected image can be input into the defect detection model, and the defect detection model can be instructed to perform quality inspection on the object to be inspected based on a threshold parameter.

[0381] In this embodiment, the configuration parameters may be migrated from other production lines associated with the target production line, or they may be pre-stored and associated with the target production line. In this case, the process of image acquisition based on the configuration parameters and defect detection based on the threshold parameters of the defect detection model can be described in the above embodiment and will not be repeated here.

[0382] In some embodiments, as described above, during the quality inspection process, the working parameters of the industrial control equipment can be adjusted according to the efficiency of the quality inspection and the location of the defects, and the image acquisition parameters can be adjusted in time, so as to achieve real-time adjustment of the information collection required for quality inspection.

[0383] Specifically, during the quality inspection process, a first position and a second position may be determined based on the defect position distribution of a plurality of objects to be inspected, wherein the number of objects to be inspected having defects at the first position exceeds a first preset number; the number of objects to be inspected having defects at the second position does not exceed a second preset number, and the first preset number is greater than the second preset number;

[0384] The image acquisition parameters include acquisition frequency and / or acquisition resolution.

[0385] In this embodiment, the configuration parameters may be automatically updated based on the quality inspection results of the objects to be inspected. Specifically, the first position and the second position may be determined based on the defect position distribution of multiple objects to be inspected.

[0386] Among them, the first position can be understood as: at this position, more than a first preset number of objects to be inspected pass the defect detection within a unit time, and the second position can be understood as: at this position, more than a second preset number of objects to be inspected fail the defect detection within a unit time; in this way, the first position can refer to a location with low defect incidence, and the second position can refer to a location with high defect incidence, wherein the first position can be different from the second position.

[0387] It should be noted that there can be more than one first position and more than one second position.

[0388] Thus, the quality control system can target the image acquisition parameters of the device; as described in the above embodiment, the operating parameters may include the operating parameters of the image acquisition device and the operating parameters of the controller. The operating parameters of the controller are used to indicate the starting position, the ending position, and the moving rate between the starting position and the ending position when the image acquisition device performs image acquisition. The operating parameters of the image acquisition device are used to indicate the acquisition frequency and resolution of the image acquisition. Therefore, when the operating parameters are adjusted based on the first position and the second position, the adjustment can be performed in the following manner:

[0389] First, for the first position, since it is a location with a low incidence of defects, the camera's movement rate can be adjusted to be higher at this location. In other words, the camera does not need to perform detailed acquisition at this location. Specifically, the preset value can be reduced at the current movement rate of the image acquisition device. The image acquisition device's acquisition frequency at this location can also be lowered to reduce the number of images captured by the image acquisition device at this location. Specifically, the preset value can be reduced at the current acquisition frequency of the image acquisition device. Of course, the acquisition resolution of the image acquisition device at this location can also be lowered, for example, by reducing the current acquisition resolution by a preset resolution level.

[0390] At the second location, since it is a location with a high incidence of defects, the camera's movement rate can be adjusted to be slower at this location. In other words, the camera can perform refined acquisition at this location. Specifically, a preset value can be increased under the current movement rate of the image acquisition device. The acquisition frequency of the image acquisition device at this location can also be increased, thereby increasing the number of images captured by the image acquisition device at this location. Specifically, a preset value can be increased under the current acquisition frequency of the image acquisition device. Of course, the acquisition resolution of the image acquisition device at this location can also be increased. For example, a preset resolution level can be increased under the current acquisition resolution, thereby achieving a rough inspection at the first location and a fine inspection at the second location, thereby saving resources.

[0391] The above adjustments to the working parameters at the first position and the second position may be performed selectively or in combination, which is not limited here.

[0392] Accordingly, when the quality inspection process is adjusted according to the quality inspection results, the quality inspection control system can automatically adjust the working parameters of the industrial control equipment according to the quality inspection results.

[0393] In some other embodiments, referring to FIG. 16 , a flow chart of steps for adjusting the quality inspection process in real time is shown. As shown in FIG. 16 , the steps may include:

[0394] Step S1601: determining the quality inspection results of multiple objects to be inspected in different production periods;

[0395] Step S1602: determining a first time period and a second time period based on quality inspection results of different output time periods, wherein the quality inspection pass rate of the objects to be inspected produced in the first time period is lower than the quality inspection pass rate of the objects to be inspected produced in the second time period;

[0396] In this embodiment, the quality inspection pass rate of the objects to be inspected produced in the first time period is lower than the quality inspection pass rate of the objects to be inspected produced in the second time period, which can mean: the number of objects to be inspected produced in the first time period that failed the quality inspection is higher than the third preset number, and the number of objects to be inspected produced in the second time period that failed the quality inspection is lower than the fourth preset number; wherein the third preset number is greater than the fourth preset number.

[0397] In practice, the output time periods can be divided according to demand, which will not be elaborated here. At the end of a statistical cycle, such as a week or a day, the quality inspection results of multiple objects to be inspected in different output time periods can be counted to obtain the quality inspection pass rate of the objects to be inspected in each output time period.

[0398] Then, the different time periods can be sorted in order of quality inspection pass rate from high to low or from low to high, thereby obtaining a first time period with a lower quality inspection pass rate and a second time period with a higher quality inspection pass rate;

[0399] Of course, the first and second time periods can be determined based on the quality inspection pass rates of multiple statistical periods. For example, if multiple statistical periods show that the pass rate of time period 1 is low, time period 1 will be used as the first time period. If only one statistical period shows a low pass rate, it may not be used as the first time period. The second time period can also be determined based on the determination of the first time period.

[0400] Step S1603: Based on the first time period and the second time period, perform at least one of the following adjustments:

[0401] Adjusting the operating parameters of the first time period and / or the operating parameters of the second time period so that the image acquisition frequency of the device in the first time period is higher than the image acquisition frequency in the second time period;

[0402] Increase the quality inspection accuracy during the first period of time, and / or decrease the quality inspection accuracy during the second period of time.

[0403] In practice, the first time period may refer to a time period when defects of the object to be inspected occur frequently. In practice, problems may occur in the process flow of this time period or in the workers' control of their target production line. Therefore, in this time period, the working parameters may be adjusted in a targeted manner. Specifically, similar to the above-mentioned control method for the defect position, in this time period, the image acquisition frequency of the equipment may be appropriately increased. Specifically, a preset value may be added to the current image acquisition frequency. In the second time period, the image acquisition frequency of the equipment may be lowered. Specifically, a preset value may be reduced from the current image acquisition frequency, thereby achieving precise inspection in the first time period and rough inspection in the second time period to save resources.

[0404] At the same time, the quality inspection accuracy of the first time period and the quality inspection accuracy of the second time period can also be adjusted. The quality inspection accuracy can be targeted at the defect detection model. In practice, the quality inspection accuracy of the first time period can be increased and the quality inspection accuracy of the second time period can be lowered, thereby achieving strict quality inspection of the first time period and rough inspection of the second time period, thereby saving resources and improving efficiency.

[0405] In some embodiments, the target production line can be regulated based on the quality inspection results. Specifically, during debugging, the quality inspection results can be first counted according to the defect locations of the objects to be inspected, the number distribution of objects to be inspected that failed the quality inspection in different time periods, and the defect types, and then the target production line can be regulated based on the statistical results.

[0406] During specific implementation, the quality inspection results of multiple objects to be inspected can be counted according to the defect location, and / or the quality inspection results of multiple objects to be inspected can be counted according to the output time period; then, based on the statistical results of the defect location, and / or the statistical results of the detection time period when the defect occurs, the process equipment corresponding to the defect location on the target production line is regulated.

[0407] The regulation in this embodiment can refer to the description of the embodiment of the above-mentioned quality control method, and will not be repeated here.

[0408] Of course, in some embodiments, when statistics are collected on the quality inspection results, the quality inspection results of multiple objects to be inspected can be collected according to the defect location and / or inspection period; and the statistical results can be generated into statistical charts according to preset icon methods, and the statistical charts can be displayed; wherein, the preset icon methods include pie charts, bar charts, line charts, and heat maps.

[0409] The statistical and chart generation method in this embodiment can refer to the description of the embodiment of the above-mentioned quality control method, and will not be repeated here.

[0410] Accordingly, when regulating the target production line, it can be done based on statistical charts. Specifically, in response to triggering a chart area in the statistical chart, the quality inspection information corresponding to the chart area can be determined; and the debugging parameters corresponding to the quality inspection information can be obtained from the database, and based on the debugging parameters, the process equipment on the target production line can be regulated.

[0411] The quality inspection information includes at least one of a target defect location, a target time period, and a target defect type; wherein the number of objects to be inspected having defects at the target defect location exceeds a first preset number, the number of objects to be inspected that fail quality inspection during the target time period exceeds a second preset number, and the number of objects to be inspected having defects of the target defect type exceeds a third preset number;

[0412] The database stores debugging parameters corresponding to the plurality of quality inspection information.

[0413] Regarding the regulation of quality inspection information in this embodiment, reference may be made to the description of the above-mentioned embodiment of the quality control method, which will not be repeated here.

[0414] Of course, in some other embodiments, the defect detection model can be updated, wherein the condition for triggering the update of the defect detection model may be: the number of objects to be inspected that have undergone quality inspection reaches a preset number, for example, ten thousand pieces, then the update of the defect detection model can be triggered when ten thousand objects to be inspected have been quality inspected.

[0415] During specific implementation, when updating the defect detection model, a second data set can be obtained, and the defect detection model can be updated using the second data set according to the description of the embodiment of the above-mentioned quality control method; then, the defect detection model being quality inspected at the current moment can be replaced with the updated defect detection model.

[0416] The technical solution of this embodiment has the following advantages:

[0417] On the one hand, since quality inspection is performed on the objects to be inspected on the target production line and the target production line is regulated based on the quality inspection, the quality inspection of the objects to be inspected disclosed in the present invention can be deeply implemented in the production line for preparing the objects to be inspected, so as to regulate the process flow of the production line in combination with the quality inspection situation of the objects to be inspected, so that the yield problem of the objects to be inspected can be solved from the source of preparing the objects to be inspected, such as process parameters, process equipment, etc., and the production cost consumed by discarding the defective objects to be inspected can be reduced.

[0418] On the other hand, since the target production line can be regulated based on the statistical analysis results of the quality inspection results in the output period and defect location, the target production line can be regulated in a targeted manner to improve the regulation efficiency.

[0419] On the other hand, since the locations and production time periods with high defect incidence, as well as the locations and production time periods with low defect incidence, can be determined based on the quality inspection results, and then fed back to the quality inspection equipment, the accuracy of image acquisition and defect detection in quality inspection can be adjusted according to the location and production time period, thereby fully saving resources and allowing the hardware resources of the production line to be fully focused on the locations and time periods of the objects to be inspected that require attention, achieving rational resource utilization and improving efficiency.

[0420] In summary, the quality control method of the second embodiment has the following advantages:

[0421] 1. Provides an industrial quality inspection AI model training solution with a visual interface and low technical threshold. It integrates core functions such as data labeling, model training, and model publishing in the development of quality inspection scenario models. This greatly reduces the technical threshold and R&D efficiency of using AI technology to train quality inspection models in departments with weak AI technology accumulation, such as factories and production lines.

[0422] 2. Provide unified management of employees, equipment, and data. In terms of employee management, it provides permission management for employees in different positions to assist in the reliable implementation of production line management systems. In terms of equipment management, this disclosure supports integrated management of PLCs, industrial control servers, data servers, industrial cameras, triggers, and other equipment, mainly including: equipment asset management, equipment status monitoring, policy issuance, etc.; it can also perform data management, including data statistics, data analysis, data retrieval, data storage, data labeling, multi-terminal display, etc.

[0423] 3. Supports training iteration of algorithm models (defect detection models) and real-machine debugging environment for industrial control equipment, facilitating real-machine debugging for R&D and maintenance personnel, thereby improving R&D efficiency;

[0424] 4. According to the regular business needs of industrial production lines, it is divided into three parts: quality inspection, control, and automatic voice training. By breaking down the overall system into multiple modules, the relative decoupling of production line quality inspection, line start-up, and R&D is achieved, which increases the space and flexibility for future functional expansion.

[0425] 5. On the automated training platform, various configuration options can be used to lower the barrier to entry. On the quality inspection side, the system integrates industrial control equipment configuration, algorithm configuration, material configuration, inspection workbench, equipment monitoring, and other core functions directly used in frontline quality inspection scenarios for the inspection server on the production line. It also communicates with the digital server to transmit production and operation data. On the control side, as the central management node of the entire system, it is equipped with big data processing, storage, analysis, and equipment management functions, covering data management, inspection equipment management, user management, data analysis, and other functions.

[0426] 6. In addition, this quality control system supports data integration with industrial production systems to achieve data sharing and promote the digitalization of the entire production process.

[0427] Based on the same inventive concept, the present disclosure also provides a quality control device for a production line, which may include the following modules:

[0428] A quality inspection module, configured to perform quality inspection on a plurality of objects to be inspected produced on a target production line in response to a triggering of a first preset condition;

[0429] A feedback module, configured to feed back the quality inspection result to a control object associated with the target production line, wherein the control object is used to control the process flow of the target production line;

[0430] a control module, configured to control the target production line based on the quality inspection result of at least one of the objects to be inspected, in response to a second preset condition triggered on the control object;

[0431] The quality inspection includes: performing at least one image acquisition on the object to be inspected produced by the target production line, and inputting at least one acquired image into a defect detection model to perform quality inspection on the object to be inspected.

[0432] Optionally, the first preset condition includes: the start of operation of the target production line, and / or a quality inspection function triggered on a preset interface.

[0433] Optionally, the second preset condition includes: there are greater than or equal to a preset number of objects to be inspected that fail the quality inspection, and / or a preset operation is triggered on the control object.

[0434] Optionally, the device further comprises:

[0435] a first interface display module configured to display a first configuration interface in response to a triggered quality inspection configuration function before quality inspection of the plurality of objects to be inspected produced on the target production line; wherein the first configuration interface includes a first configuration item of a device for quality inspection and a second configuration item of the defect detection model;

[0436] a parameter receiving module, configured to receive configuration parameters inputted through the first configuration item and the second configuration item respectively;

[0437] The performing quality inspection on the multiple objects to be inspected produced on the target production line specifically includes: performing quality inspection on the multiple objects to be inspected produced on the target production line based on the configuration parameters.

[0438] Optionally, the device further comprises:

[0439] a second interface display module, configured to display a second configuration interface in response to a triggered quality inspection and debugging function; wherein the second configuration interface includes target configuration items, the target configuration items including a first configuration item of a device for quality inspection and / or a second configuration item of the defect detection model;

[0440] A result display module, configured to display, on the second configuration interface, statistical results of the quality inspection results of the inspected objects to be inspected, and record configuration parameters after the current configuration parameters of the target configuration item are modified;

[0441] The step of performing quality inspection on the multiple objects to be inspected produced on the target production line includes: after the current moment, performing quality inspection on the multiple objects to be inspected produced on the target production line based on the modified configuration parameters.

[0442] Optionally, the equipment for quality inspection includes an image acquisition device and a controller for controlling movement of the image acquisition device, and the first configuration item is used to input working parameters corresponding to the image acquisition device and the controller respectively;

[0443] Among them, the working parameters corresponding to the image acquisition device include acquisition frequency and / or acquisition resolution, and the working parameters corresponding to the controller include at least one of the starting position, ending position and moving speed of the image acquisition.

[0444] Optionally, the second configuration item is used to input a threshold parameter corresponding to the defect detection model and / or to input an operating parameter of a server where the defect detection model is located.

[0445] Optionally, the device further comprises:

[0446] an associating module, configured to, upon detecting that an associated production line associated with the target production line is operating, associate the configuration parameters with the associated production line in response to a triggered parameter migration function; wherein the defects of the objects to be inspected targeted by the quality inspection in the associated production line are the same as or similar to the defects of the objects to be inspected targeted by the quality inspection in the target production line;

[0447] An associated quality inspection module is used to perform quality inspection on a plurality of objects to be inspected produced on the associated production line using the defect detection model based on the configuration parameters.

[0448] Optionally, the device further comprises:

[0449] a target parameter set acquisition module, configured to acquire a target data set corresponding to the associated production line, wherein the target data set includes image samples obtained by capturing and annotating a plurality of objects to be inspected on the associated production line;

[0450] An updating module, configured to update the defect detection model using the target data set in response to a triggered model migration function to obtain a migration model;

[0451] The associated quality inspection module is specifically configured to perform quality inspection on a plurality of objects to be inspected produced on the associated production line using the migration model based on the configuration parameters.

[0452] Optionally, the device further comprises:

[0453] a third interface display module, configured to display a third configuration interface in response to a triggered model building function; wherein the third configuration interface includes at least one trigger control, and different trigger controls are used to trigger different configuration windows in the model building process;

[0454] An option determination module, configured to determine a configuration option selected in a different configuration window; wherein the configuration window includes a plurality of configuration options;

[0455] The automatic training module is used to train the defect detection model based on the preset configuration parameters corresponding to the selected configuration option and using the pre-stored first data set as a training sample.

[0456] Optionally, the device further comprises:

[0457] a data set acquisition module, configured to acquire a second data set in response to a triggered model update function; wherein the second data set includes a plurality of target image samples, and the target image samples include areas where defective areas of an inspected object to be inspected are re-labeled;

[0458] The model updating module is used to update the defect detection model using the second data set as a training sample, and replace the defect detection model currently performing the quality inspection with the updated defect detection model.

[0459] Optionally, the second data set is obtained by the following steps:

[0460] In response to the triggered data annotation function, obtaining quality inspection results of a plurality of objects to be inspected that have undergone the quality inspection, and displaying the plurality of quality inspection results;

[0461] In response to an error correction operation triggered for a target quality inspection result among the plurality of quality inspection results, displaying at least one target image corresponding to the target quality inspection result, wherein a defective area is marked in the target image;

[0462] In response to the modification operation on the defect area, marking the modified defect area on the target image;

[0463] The target image marked with the modified defect area is added to the second data set as the target image sample.

[0464] Optionally, the quality inspection includes quality inspection of multiple defect types of the object to be inspected; the defect detection model includes detection branches corresponding to each of the multiple defect types, and the model updating module includes:

[0465] a type determination unit, configured to, in response to a selection operation on at least one defect type from the plurality of defect types, obtain, from the second data set, a labeled subset corresponding to the selected defect type, wherein the defect region labeled on the target image sample in the labeled subset corresponds to the defect type;

[0466] An updating unit is configured to update a detection branch corresponding to a selected defect type in the defect detection model using the annotation subset.

[0467] Optionally, the feedback module includes:

[0468] a statistical unit, configured to, in response to a triggered quality inspection statistical function, perform statistics on the quality inspection results of the plurality of objects to be inspected to obtain statistical results;

[0469] A chart generating unit, configured to generate a statistical chart based on the statistical results;

[0470] A chart sending unit is used to send the statistical chart to the control object; wherein the statistical chart includes at least one of a pie chart, a bar chart, a line chart and a heat map.

[0471] Optionally, the statistical unit includes at least one of the following:

[0472] a first statistical subunit, configured to collect statistics on the quality inspection results of the plurality of objects to be inspected according to the locations where the defects occur;

[0473] A second statistical subunit is used to collect statistics on the quality inspection results of multiple objects to be inspected according to the time period in which the objects to be inspected are produced;

[0474] The third statistical subunit is used to collect statistics on the quality inspection results of the plurality of objects to be inspected according to the types of defects; wherein the quality inspection is used to detect multiple defect types of the objects to be inspected.

[0475] Optionally, the control module includes:

[0476] an information display unit, configured to, when the second preset condition is a preset operation triggered on the control object, display quality inspection information in response to the preset operation, the quality inspection information including at least one of a target defect location, a target time period, and a target defect type; wherein the number of objects to be inspected having defects at the target defect location exceeds a first preset number, the number of objects to be inspected that fail quality inspection at the target time period exceeds a second preset number, and the number of objects to be inspected having defects of the target defect type exceeds a third preset number;

[0477] a debugging parameter receiving unit, configured to receive debugging parameters entered for the quality inspection information, wherein the debugging parameters include operating parameters of process equipment on the target production line;

[0478] A control unit is used to control the process equipment based on the debugging parameters.

[0479] Optionally, the defect detection model includes a first model and a second model, and the quality inspection module includes:

[0480] a first quality inspection unit, configured to input the at least one image into the first model to perform a first quality inspection on the object to be inspected;

[0481] a second quality inspection unit, configured to, when the result of the first quality inspection indicates that the quality inspection has failed, re-capture images of the object to be inspected at least once, and input the re-captured images into the second model to perform a second quality inspection on the object to be inspected;

[0482] The quality inspection accuracy of the first quality inspection is lower than the quality inspection accuracy of the second quality inspection.

[0483] Optionally, the device further comprises:

[0484] a quantity determination module, configured to determine the number of objects to be inspected whose quality inspections have been completed within a unit time during the process of performing the quality inspection on the plurality of objects to be inspected produced on the target production line;

[0485] A speed adjustment module is used to adjust the transmission speed of the objects to be inspected on the target production line based on the quantity.

[0486] Based on the same inventive concept, the present disclosure also provides a quality control device for a production line, which may include the following modules:

[0487] An image acquisition module is used to acquire at least one image of the object to be inspected produced on the target production line;

[0488] A quality inspection module, configured to input the collected at least one image into a defect detection model, so as to perform quality inspection on the object to be inspected through the defect detection model;

[0489] The control module is used to control the target production line based on the quality inspection result of at least one of the objects to be inspected on the target production line.

[0490] Optionally, the device further comprises:

[0491] A configuration parameter acquisition module, configured to acquire configuration parameters corresponding to the target production line, wherein the configuration parameters include operating parameters of equipment used for quality inspection and threshold parameters of the defect detection model;

[0492] The image acquisition module includes:

[0493] an acquisition unit, configured to acquire an image of the object to be inspected at least once based on the working parameters;

[0494] Among them, the step of inputting at least one collected image into the defect detection model to perform quality inspection on the object to be inspected through the defect detection model includes: inputting at least one collected image into the defect detection model, and instructing the defect detection model to perform quality inspection on the object to be inspected based on the threshold parameters.

[0495] Optionally, the quality inspection result includes a defect location of the object to be inspected, and the device further includes:

[0496] a position determination module configured to, after inputting the at least one captured image into the defect detection model to perform quality inspection on the object to be inspected using the defect detection model, determine a first position and a second position based on a defect position distribution of a plurality of the objects to be inspected, wherein the number of objects to be inspected having defects at the first position exceeds a first preset number; the number of objects to be inspected having defects at the second position does not exceed a second preset number, and the first preset number is greater than the second preset number;

[0497] A parameter updating module is used to update the operating parameters based on the first position and the second position so that the image acquisition parameters of the device at the first position are lower than the image acquisition parameters at the second position; wherein the image acquisition parameters include acquisition frequency and / or acquisition resolution.

[0498] Optionally, the device further comprises:

[0499] The quality inspection result determination module is used to determine the quality inspection results of multiple objects to be inspected in different output periods;

[0500] a time period determination module, configured to determine a first time period and a second time period based on the quality inspection results of the different output time periods; wherein the quality inspection pass rate of the objects to be inspected produced in the first time period is lower than the quality inspection pass rate of the objects to be inspected produced in the second time period;

[0501] An adjustment module, configured to perform at least one of the following adjustments based on the first time period and the second time period:

[0502] Adjusting the operating parameters of the first time period and / or the operating parameters of the second time period so that the image acquisition frequency of the device in the first time period is higher than the image acquisition frequency in the second time period;

[0503] Increase the quality inspection accuracy during the first period of time, and / or decrease the quality inspection accuracy during the second period of time.

[0504] Based on the same inventive concept, the present disclosure also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the quality control method for the production line described in the embodiment when executed.

[0505] The embodiment of the present disclosure further provides a computer-readable storage medium, which stores a computer program that enables a processor to execute the quality control method for a production line as described in the embodiment of the present disclosure.

[0506] Finally, it should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity, or device that includes a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, commodity, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, commodity, or device that includes the element.

[0507] The above is a detailed introduction to the quality control method, system and manufacturing platform of the production line provided by the present disclosure. Specific examples are used herein to illustrate the principles and implementation methods of the present disclosure. The description of the above embodiments is only used to help understand the method of the present disclosure and its core ideas. At the same time, for those skilled in the art, according to the ideas of the present disclosure, there will be changes in the specific implementation methods and application scopes. In summary, the content of this specification should not be understood as a limitation on the present disclosure.

[0508] Other embodiments of the present disclosure will readily occur to those skilled in the art after considering the specification and practicing the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art not disclosed herein. The description and examples are to be considered as exemplary only, with the true scope and spirit of the present disclosure being indicated by the following claims.

[0509] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes can be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

[0510] References herein to "one embodiment," "an embodiment," or "one or more embodiments" mean that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the present disclosure. Furthermore, please note that instances of the phrase "in one embodiment" do not necessarily all refer to the same embodiment.

[0511] In the description provided herein, numerous specific details are described. However, it is understood that embodiments of the present disclosure may be practiced without these specific details. In some instances, well-known methods, structures, and techniques are not shown in detail so as not to obscure the understanding of this description.

[0512] In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word "comprising" does not exclude the presence of elements or steps not listed in the claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The present disclosure may be implemented by means of hardware comprising several different elements and by means of a suitably programmed computer. In a unit claim enumerating several means, several of these means may be embodied by the same item of hardware. The use of the words first, second, and third etc. does not indicate any order. These words may be interpreted as names.

[0513] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present disclosure, rather than to limit them. Although the present disclosure has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present disclosure.

Claims

1. A quality control method for a production line, It is characterized in that The method comprises: In response to the triggering of the first preset condition, quality inspection is performed on a plurality of objects to be inspected produced on the target production line; Feeding back the quality inspection result to a control object associated with the target production line, wherein the control object is used to control the process flow of the target production line; In response to a second preset condition being triggered on the control object, regulating the target production line based on a quality inspection result of at least one of the objects to be inspected; The quality inspection includes: performing at least one image acquisition on the object to be inspected produced by the target production line, and inputting at least one acquired image into a defect detection model to perform quality inspection on the object to be inspected.

2. The quality control method according to claim 1, It is characterized in that The first preset condition includes: the start of operation of the target production line, and / or a quality inspection function triggered on a preset interface.

3. The quality control method according to claim 1, It is characterized in that The second preset condition includes: there are greater than or equal to a preset number of objects to be inspected that fail the quality inspection, and / or a preset operation is triggered on the control object.

4. The quality control method according to claim 1, It is characterized in that Before performing quality inspection on the multiple objects to be inspected produced on the target production line, the method further includes: In response to the triggered quality inspection configuration function, a first configuration interface is displayed; wherein the first configuration interface includes a first configuration item of a device for quality inspection and a second configuration item of the defect detection model; receiving configuration parameters respectively input through the first configuration item and the second configuration item; The quality inspection of the multiple objects to be inspected produced on the target production line includes: Based on the configuration parameters, quality inspection is performed on a plurality of objects to be inspected produced on the target production line.

5. The quality control method according to claim 1, It is characterized in that In the process of performing quality inspection on a plurality of objects to be inspected produced on the target production line, the method further comprises: In response to the triggered quality inspection debugging function, a second configuration interface is displayed; wherein the second configuration interface includes a target configuration item, and the target configuration item includes a first configuration item of a device for quality inspection, and / or a second configuration item of the defect detection model; On the second configuration interface, statistical results of the quality inspection results of the inspected objects to be inspected are displayed, and configuration parameters after the current configuration parameters of the target configuration item are modified are recorded; The quality inspection of the multiple objects to be inspected produced on the target production line includes: After the current moment, quality inspection is performed on a plurality of objects to be inspected produced on the target production line based on the modified configuration parameters.

6. The quality control method according to claim 4 or 5, It is characterized in that The equipment for quality inspection includes an image acquisition device and a controller for controlling the movement of the image acquisition device, and the first configuration item is used to input working parameters corresponding to the image acquisition device and the controller respectively; Among them, the working parameters corresponding to the image acquisition device include acquisition frequency and / or acquisition resolution, and the working parameters corresponding to the controller include at least one of the starting position, ending position and moving speed of the image acquisition.

7. The quality control method according to claim 4 or 5, It is characterized in that The second configuration item is used to input a threshold parameter corresponding to the defect detection model and / or to input a working parameter of a server where the defect detection model is located.

8. The quality control method according to claim 4, It is characterized in that The method further comprises: When it is detected that an associated production line associated with the target production line is running, in response to a parameter migration function being triggered, the configuration parameters are associated with the associated production line; wherein the defects of the object to be inspected targeted by the quality inspection in the associated production line are the same as or similar to the defects of the object to be inspected targeted by the quality inspection in the target production line; Based on the configuration parameters, the defect detection model is used to perform quality inspection on a plurality of objects to be inspected produced on the associated production line.

9. The quality control method according to claim 8, It is characterized in that Before performing quality inspection on a plurality of objects to be inspected produced on the associated production line using the defect detection model based on the configuration parameters, the method further includes: Acquire a target data set corresponding to the associated production line, the target data set including Associating multiple objects to be inspected on the production line with image samples after image acquisition and annotation; In response to the triggered model migration function, the defect detection model is updated using the target data set to obtain a migration model; Based on the configuration parameters, the defect detection model is used to perform quality inspection on a plurality of objects to be inspected produced on the associated production line, including: Based on the configuration parameters, the migration model is used to perform quality inspection on a plurality of objects to be inspected produced on the associated production line.

10. The quality control method according to claim 1, It is characterized in that Before the step of performing quality inspection on the multiple objects to be inspected produced on the target production line, the method further includes: In response to the triggered model building function, a third configuration interface is displayed; wherein the third configuration interface includes at least one trigger control, and different trigger controls are used to trigger different configuration windows in the model building process; Determining a configuration option selected in a different configuration window; wherein the configuration window includes a plurality of configuration options; Based on the preset configuration parameters corresponding to the selected configuration option, the defect detection model is trained using the pre-stored first data set as a training sample.

11. The quality control method according to claim 1, It is characterized in that After performing quality inspection on a plurality of objects to be inspected produced on the target production line, the method further includes: In response to the triggered model update function, a second data set is acquired; wherein the second data set includes a plurality of target image samples, and the target image samples include an area for re-marking defect areas of the inspected object; Using the second data set as a training sample, updating the defect detection model; The defect detection model currently used for the quality inspection is replaced with the updated defect detection model.

12. The quality control method according to claim 11, It is characterized in that The second data set is obtained by the following steps: In response to the triggered data annotation function, obtaining quality inspection results of a plurality of objects to be inspected that have undergone the quality inspection, and displaying the plurality of quality inspection results; In response to an error correction operation triggered for a target quality inspection result among the plurality of quality inspection results, display showing at least one target image corresponding to the target quality inspection result, wherein a defective area is marked in the target image; In response to the modification operation on the defective area, marking the modified defective area on the target image; The target image marked with the modified defective area is added to the second data set as the target image sample.

13. The quality control method according to claim 11, It is characterized in that The quality inspection includes quality inspection of multiple defect types of the object to be inspected; The defect detection model includes detection branches corresponding to the plurality of defect types, and the updating of the defect detection model using the second data set as a training sample includes: In response to a selection operation of at least one defect type among the multiple defect types, acquiring a labeling subset corresponding to the selected defect type from the second data set, wherein the defect area labeled by the target image sample in the labeling subset corresponds to the defect type; The detection branch corresponding to the selected defect type in the defect detection model is updated using the annotation subset.

14. The quality control method according to claim 1, It is characterized in that Feeding back the quality inspection result to the control object associated with the target production line includes: In response to the triggered quality inspection statistics function, the quality inspection results of the plurality of objects to be inspected are counted to obtain statistical results; Based on the statistical results, generate statistical charts; The statistical chart is sent to the control object; wherein the statistical chart includes at least one of a pie chart, a bar chart, a line chart and a heat map.

15. The quality control method according to claim 14, It is characterized in that The statistical analysis of the quality inspection results of the plurality of objects to be inspected includes at least one of the following: According to the locations where the defects occur, statistics are collected on the quality inspection results of the plurality of objects to be inspected; According to the time period when the objects to be inspected are produced, the quality inspection results of the multiple objects to be inspected are counted; According to the types of defects, the quality inspection results of the multiple objects to be inspected are counted; wherein the quality inspection is used to detect multiple defect types of the objects to be inspected.

16. The quality control method according to any one of claims 1 to 5 or any one of claims 8 to 15, It is characterized in that In response to the second preset condition being triggered on the control object, the target production line is regulated based on the quality inspection result of at least one of the objects to be inspected, including: In the case where the second preset condition is a preset operation triggered on the control object, in response to the preset operation, quality inspection information is displayed, the quality inspection information including at least one of a target defect position, a target time period, and a target defect type; wherein the number of objects to be inspected having defects at the target defect position exceeds a first preset number, the number of objects to be inspected that fail the quality inspection at the target time period exceeds a second preset number, and the number of objects to be inspected having defects of the target defect type exceeds a third preset number; receiving debugging parameters entered for the quality inspection information, wherein the debugging parameters include operating parameters of process equipment on the target production line; Based on the debugging parameters, the process equipment is regulated.

17. The quality control method according to claim 1, It is characterized in that The defect detection model includes a first model and a second model, and the image acquisition is performed at least once on the object to be inspected on the target production line, and at least one acquired image is input into the defect detection model to perform quality inspection on the object to be inspected, including: Inputting the at least one image into the first model to perform a first quality inspection on the object to be inspected; When the result of the first quality inspection indicates that the quality inspection has not passed, re-capturing images of the object to be inspected at least once, and inputting the re-captured images into the second model to perform a second quality inspection on the object to be inspected; Among them, the quality inspection accuracy of the first quality inspection is lower than the quality inspection accuracy of the second quality inspection.

18. The quality control method according to claim 1, It is characterized in that The method further comprises: In the process of performing the quality inspection on the plurality of objects to be inspected produced on the target production line, determining the number of objects to be inspected whose quality inspection is completed within a unit time; Based on the quantity, a transmission speed of the objects to be inspected on the target production line is adjusted.

19. A quality control method for a production line, It is characterized in that The method comprises: Capturing at least one image of the object to be inspected produced on the target production line; Inputting at least one collected image into a defect detection model to perform quality inspection on the object to be inspected through the defect detection model; The target production line is regulated based on the quality inspection result of at least one of the objects to be inspected on the target production line.

20. The method according to claim 19, It is characterized in that Before capturing at least one image of the object to be inspected prepared on the target production line, the method further includes: Acquire configuration parameters corresponding to the target production line, wherein the configuration parameters include working parameters of equipment used for quality inspection and threshold parameters of the defect detection model; At least one image acquisition is performed on the object to be inspected prepared on the target production line, including: Based on the working parameters, performing at least one image acquisition on the object to be inspected; Inputting at least one collected image into a defect detection model to perform quality inspection on the object to be inspected through the defect detection model, including: Inputting at least one collected image into a defect detection model, and instructing the defect detection model to perform quality inspection on the object to be inspected based on the threshold parameter.

21. The method according to claim 19, It is characterized in that The quality inspection result includes the defect position of the object to be inspected. After inputting the at least one acquired image into the defect detection model to perform quality inspection on the object to be inspected through the defect detection model, the method further includes: Determine a first position and a second position according to the defect position distribution of the plurality of objects to be inspected, wherein the number of objects to be inspected with defects at the first position exceeds a first preset number; the number of objects to be inspected with defects at the second position does not exceed a second preset number, and the first preset number is greater than the second preset number; Based on the first position and the second position, the working parameters are updated so that the image acquisition parameters of the device at the first position are lower than the image acquisition parameters at the second position; wherein the image acquisition parameters include acquisition frequency and / or acquisition resolution.

22. The method according to claim 19, It is characterized in that The method further comprises: Determine the quality inspection results of multiple objects to be inspected in different output periods; Based on the quality inspection results of the different output time periods, determining a first time period and a second time period; wherein the quality inspection pass rate of the objects to be inspected produced in the first time period is lower than the quality inspection pass rate of the objects to be inspected produced in the second time period; Based on the first time period and the second time period, at least one of the following adjustments is performed: Adjusting the working parameters of the first time period and / or the working parameters of the second time period so that the image acquisition frequency of the device in the first time period is higher than the image acquisition frequency in the second time period; Increase the quality inspection accuracy of the first time period, and / or decrease the quality inspection accuracy of the second time period.

23. A quality control system, It is characterized in that Includes infrastructure layer and quality control layer; The infrastructure layer includes a plurality of first devices, and different first devices are used to provide different services required for quality inspection; The quality control layer communicates with the infrastructure layer and is used to execute the quality control method described in any one of claims 1-18 or any one of claims 19-22.

24. The quality control system according to claim 23, It is characterized in that The multiple first devices include: At least one server, wherein different servers are used to provide different business services for the production line, wherein the business services at least include: control services and quality inspection services; at least one gateway, configured to establish a network connection between the at least one server; Among them, at least one server includes a quality inspection server configured with a defect detection model, and the quality inspection server is used to input the image into the defect detection model after receiving the image captured by the second device, so as to perform quality inspection on multiple objects to be inspected; wherein, the second device is located on the production line.

25. The quality control system according to claim 23, It is characterized in that The quality control layer includes: An information management module, configured to perform at least one of the following information processing: processing the data generated in the quality inspection, processing the status information of the plurality of first devices, and processing the user information of the user; A detection module, used to perform at least one of the following quality inspections on the object to be inspected: parameter configuration in the quality inspection, quality inspection of the object to be inspected, error correction of the quality inspection of the object to be inspected, and access configuration of multiple devices; The model management module is used to construct the defect detection model.

26. The quality control system according to claim 25, It is characterized in that The information management module comprises: A user management unit, configured to, in response to a user's triggering operation on a target function among the multiple functions, determine whether the target function complies with the user's authority, and activate the target function if the target function complies with the user's authority; Among them, after being activated, the multiple functions are used to perform the following operations respectively: Parameter configuration of multiple types of the first equipment, debugging of process equipment of the production line, statistics of quality inspection results, updating of defect detection models, associating configuration parameters of the target production line to associated production lines, and error correction of misdetected objects to be inspected.

27. The quality control system according to claim 25, It is characterized in that The data generated in the quality inspection includes at least one of the following: quality inspection results and images collected in the quality inspection; The processing of the data generated in the quality inspection includes at least one of the following processing: storage, statistics, retrieval, annotation and query.

28. A manufacturing platform, It is characterized in that It comprises a computer integrated system, and the quality control system according to any one of claims 23 to 27, wherein the quality control system is integrated into the computer integrated system, and the computer integrated system is provided with a production information management system; Wherein, the quality control system communicates with the production information management system; The quality control system is used to execute the quality control method described in any one of claims 1 to 18 or any one of claims 19 to 22, and to send debugging instructions to the production information management system when regulating the target production line; The production information management system is used to regulate and control the process equipment in the target production line according to the debugging instructions.

29. The manufacturing platform according to claim 28, It is characterized in that The manufacturing platform further includes: a plurality of second devices, which communicate with the quality control system and are used to collect information on the object to be inspected on the production line, wherein the information collection at least includes image collection.

30. The manufacturing platform according to claim 28, It is characterized in that The plurality of second devices include: an image acquisition device, a controller, a sensor, and a mechanical arm; the controller is connected to the image acquisition device via the mechanical arm, and the sensor is connected to the controller; The sensor is used to detect whether the object to be detected has arrived; The controller is used to control the image acquisition to move when the object to be inspected arrives. The image acquisition device is used to acquire an image of the object to be inspected at least once.

Images