Equipment collaborative quality inspection method and system

Abstract

The invention provides an equipment collaborative quality inspection method and system, and the method comprises the steps: S1, obtaining product image information on an assembly line, carrying out the preprocessing of the obtained product image information, and obtaining the preprocessed product image information; s2, identifying product types based on the preprocessed product image information, and obtaining quality inspection parameters of the corresponding product types; performing quality inspection on the products on the assembly line based on the quality inspection parameters to distinguish qualified products from unqualified products, and calculating the qualified rate; s3, counting the productivity of the current assembly line based on the preprocessed product image information; s4, carrying out equipment working efficiency analysis based on the preprocessed product image information; and S5, when any one or more of the qualified rate, the working efficiency and the productivity of the current assembly line are lower than preset values, triggering early warning, and adjusting the current assembly line according to preset requirements.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of production management, in particular to a device collaborative quality inspection method and system. BACKGROUND

[0002] In modern industrial production, the flow line operation mode has become the mainstream production method, and product quality inspection and equipment management are the key links to ensure production quality and efficiency. Traditional product quality inspection mainly relies on manual visual inspection, and the quality inspector observes the appearance, size and other characteristics of the product by naked eye to judge whether the product is qualified. This method has many drawbacks. On the one hand, manual detection is greatly affected by subjective factors of the quality inspector, and there are differences in understanding and judgment of quality standards by different quality inspectors, which is prone to false detection and missed detection, and it is difficult to ensure the consistency and accuracy of the quality inspection results. On the other hand, manual detection is low in efficiency and cannot meet the needs of high-speed production of flow line, and long-time work easily leads to visual fatigue of the quality inspector, further reducing the detection accuracy.

[0003] In terms of equipment management, the existing technology mainly uses independent monitoring systems for equipment efficiency analysis and capacity statistics, and lacks cooperation and linkage among devices. The production data and quality inspection information cannot be shared in real time among devices, and it is difficult to comprehensively and dynamically monitor and adjust the running state of the entire flow line. When there are problems such as equipment failure, low efficiency or insufficient capacity, it is difficult to find and take effective measures to deal with them in time, which easily causes production delay, resource waste and increases the production cost of enterprises.

[0004] Patent document CN116740385A (application number: 202310990804.X) discloses a device quality inspection method, device and system, wherein the method can realize accurate identification and detection of straight lines in the device image through the application of the double-channel backbone network and the deep convolution Hough transform. By fitting the straight lines of several standard devices, the standard position range can be accurately determined, and the straight lines of the devices to be tested are judged according to the standard position range, so that whether the devices to be tested are qualified can be accurately judged. The determination process and detection process of the standard position range are realized automatically, the participation of manual work is reduced, and the detection efficiency is improved. SUMMARY

[0005] In view of the defects in the prior art, the purpose of the present application is to provide a device collaborative quality inspection method and system.

[0006] According to the device collaborative quality inspection method provided by the present application, the following steps are included:

[0007] Step S1: acquiring product image information on the flow line, and pre-processing the acquired product image information to obtain pre-processed product image information;

[0008] Step S2: identifying the product type based on the pre-processed product image information, obtaining the quality inspection parameters of the corresponding product type; performing quality inspection on the products on the assembly line based on the quality inspection parameters, for distinguishing qualified products and unqualified products, and calculating the qualified rate;

[0009] Step S3: counting the number of qualified products to obtain the current assembly line productivity;

[0010] Step S4: performing equipment working efficiency analysis based on the pre-processed product image information;

[0011] Step S5: when any one or more of the qualified rate, working efficiency and current assembly line productivity is lower than the preset value, triggering an early warning, and adjusting the current assembly line according to the preset requirements.

[0012] Preferably, the step S1 comprises:

[0013] Step S1.1: obtaining product images on the assembly line based on the camera through high-frequency shooting meeting the preset requirements;

[0014] Step S1.2: performing image enhancement, geometric correction and normalization processing on the obtained product image information on the assembly line to obtain the processed product image.

[0015] Preferably, the step S2 comprises:

[0016] Step S2.1: the pre-processed product image is based on a convolutional neural network for product type identification;

[0017] Step S2.2: obtaining the quality inspection parameters of the corresponding product based on the standard database according to the identified product type, the quality inspection parameters including color, length-width-height and texture feature information;

[0018] Step S2.3: the pre-processed product image separates the product from the background through image segmentation technology, converts the separated product from RGB color space to HSV color space, performs pixel-by-pixel scanning on the converted image, counts the HSV value of each pixel point, and obtains the color distribution information of the product; comparing the obtained product color distribution information with the color standard in the quality inspection parameters, calculating the similarity of the color distribution; when the similarity of the color distribution is less than the first threshold value, it is considered that the current product is an unqualified product;

[0019] Step S2.4: based on the pre-processed product image, the edge profile of the product is extracted by Canny edge detection, the circumscribed matrix of the product is calculated according to the edge profile, and the length and width of the product on the image plane are obtained; then, combined with the position information of the product on the pipeline and the installation parameters of the camera, the actual height of the product is calculated by the principle of triangulation; the length, width and height of the product obtained are compared with the length, width and height standards in the quality inspection parameters, and the errors of the length, width and height are calculated respectively; when the error of any one or more of the length, width and height is greater than the second threshold value, the current product is considered to be an unqualified product;

[0020] Step S2.5: the texture feature information of the product is extracted by using the local binary pattern (LBP) algorithm, and the similarity of the texture features is calculated based on the comparison between the extracted texture feature information and the texture feature information in the quality inspection parameters; when the similarity of the texture features is less than the third threshold value, the current product is considered to be an unqualified product.

[0021] Preferably, the step S4 comprises: obtaining a pre-processed product image under the current device, and classifying the current product according to the obtained pre-processed product image under the current device, including: processable product, unprocessable product and processed product;

[0022] When the current product is classified as a processable product, the working efficiency of the current device is analyzed by the device downstream of the pipeline;

[0023] The working efficiency of the current device is analyzed by the device downstream of the pipeline, including:

[0024] The pre-processed product image under the device downstream of the pipeline is obtained, and the current product is classified according to the obtained pre-processed product image under the device downstream of the pipeline, including: processable product, unprocessable product and processed product;

[0025] When the number / percentage of processable products decreases, the number / percentage of processed products increases, and the trend of the number / percentage of unprocessable products is stable, the current device works effectively; when the trend of the number / percentage of processable products is unchanged or increases, the working efficiency of the current device decreases.

[0026] Preferably, the step S5 comprises:

[0027] When the qualified rate is lower than the preset value, the working angle of the robot is adjusted to make the qualified rate meet the preset requirement;

[0028] When the working efficiency of the device is lower than the preset value, the speed state of the pipeline is obtained, and it is judged whether the speed of the pipeline is lower than the preset value to reduce the product yield; if so, the speed of the pipeline is increased to increase the product yield;

[0029] When the device working efficiency is lower than the preset value, it is checked whether the available raw materials are too few; when it is checked that the available raw materials are too few, the raw materials are adjusted;

[0030] When the device working efficiency is lower than the preset value, it is checked whether the strategy calculation time is too long, and when it is checked that the operation consumes the overall working time, the operation amount is reduced, the precision is reduced, and the fastest output processing result is obtained as the target;

[0031] When the device working efficiency is lower than the preset value, the working time of the execution mechanism is checked, and if the working time exceeds the qualified level, it represents that the robot execution speed is too slow, and the execution speed of the robot is adjusted at this time.

[0032] According to the device cooperative quality inspection system provided by the application, the following technical effects are achieved:

[0033] Module M1: obtaining product image information on the assembly line, and pre-processing the obtained product image information to obtain pre-processed product image information;

[0034] Module M2: identifying the product type based on the pre-processed product image information, obtaining the quality inspection parameters of the corresponding product type, and performing quality inspection on the products on the assembly line based on the quality inspection parameters, for distinguishing qualified products and unqualified products, and calculating the qualified rate;

[0035] Module M3: counting the number of qualified products to obtain the current assembly line capacity;

[0036] Module M4: performing device working efficiency analysis based on the pre-processed product image information;

[0037] Module M5: when any one or more of the qualified rate, the working efficiency, and the current assembly line capacity is lower than the preset value, a warning is triggered, and the current assembly line is adjusted according to the preset requirements.

[0038] Preferably, the module M1 comprises:

[0039] Module M1.1: obtaining product images on the assembly line based on the camera through high-frequency shooting meeting the preset requirements;

[0040] Module M1.2: performing image enhancement, geometric correction, and normalization processing on the obtained product image information on the assembly line to obtain processed product images.

[0041] Preferably, the module M2 comprises:

[0042] Module M2.1: the pre-processed product image is based on a convolutional neural network to identify the product type;

[0043] Module M2.2: obtaining quality inspection parameters of the corresponding product based on a standard database according to the identified product type, the quality inspection parameters including color, length, width, height, and texture feature information;

[0044] Module M2.3: separating the product from the background through image segmentation technology on the pre-processed product image, converting the separated product from an RGB color space to an HSV color space, performing pixel-by-pixel scanning on the converted image, and counting the HSV values of each pixel point to obtain color distribution information of the product; comparing the obtained color distribution information of the product with the color standard in the quality inspection parameters, and calculating the similarity of the color distribution; when the similarity of the color distribution is less than a first threshold value, the current product is considered to be an unqualified product;

[0045] Module M2.4: extracting the edge contour of the product based on the pre-processed product image through Canny edge detection, calculating the circumscribed matrix of the product according to the edge contour to obtain the length and width of the product on the image plane; combining the position information of the product on the pipeline and the installation parameters of the camera, and calculating the actual height of the product through the principle of triangulation; comparing the length, width, and height of the product obtained with the length, width, and height standard in the quality inspection parameters, and calculating the error of the length, width, and height, respectively; when the error of any one or more of the length, width, and height is greater than a second threshold value, the current product is considered to be an unqualified product;

[0046] Module M2.5: extracting the texture feature information of the product using a local binary pattern (LBP) algorithm, comparing the extracted texture feature information with the texture feature information in the quality inspection parameters, and calculating the similarity of the texture features; when the similarity of the texture features is less than a third threshold value, the current product is considered to be an unqualified product.

[0047] Preferably, the module M4 includes:

[0048] Obtaining a pre-processed product image under the current device, and classifying the current product based on the obtained pre-processed product image under the current device, including: processable product, unprocessable product, and processed product;

[0049] When the current product is classified as a processable product, analyzing the working efficiency of the current device through the device downstream of the pipeline;

[0050] The working efficiency analysis of the current device through the device downstream of the pipeline includes:

[0051] Obtaining a pre-processed product image under the device downstream of the pipeline, and classifying the current product based on the obtained pre-processed product image under the device downstream of the pipeline, including: processable product, unprocessable product, and processed product;

[0052] When the number / percentage of processable products decreases, the number / percentage of processed products increases, and the number / percentage of unprocessable products remains stable, the current equipment is working effectively; when the number / percentage of processable products remains stable or increases, the working efficiency of the current equipment decreases.

[0053] Preferably, the module M5 comprises:

[0054] When the qualified rate is lower than the preset value, the working angle of the robot is adjusted to make the qualified rate meet the preset requirement.

[0055] When the working efficiency of the equipment is lower than the preset value, the speed state of the flow line is obtained, and it is judged whether the speed of the flow line is lower than the preset value, so that the current product output is reduced, and if so, the speed of the flow line is increased to increase the product output.

[0056] When the working efficiency of the equipment is lower than the preset value, it is checked whether there is too little processable raw material; when it is checked that there is too little processable raw material, the raw material is adjusted.

[0057] When the working efficiency of the equipment is lower than the preset value, it is checked whether the strategy calculation time is too long, and when it is checked that the operation consumes the whole working time, the operation amount is reduced to output the processing result as fast as possible, and the accuracy is reduced.

[0058] When the working efficiency of the equipment is lower than the preset value, the working time of the execution mechanism is checked, and if the working time exceeds the qualified level, it means that the execution speed of the robot is too slow, and the execution speed of the robot is adjusted at this time.

[0059] Compared with the prior art, the present application has the following beneficial effects:

[0060] 1. The present application uses convolutional neural network, image segmentation, Canny edge detection and other technologies to quantitatively analyze products based on color, size, texture and other multi-dimensional parameters, avoids the false detection and missed detection problems caused by visual fatigue and subjective standard inconsistency in manual quality inspection, ensures constant quality inspection rules, and effectively improves the accuracy and consistency of product quality inspection.

[0061] 2. The present application can realize real-time and high-speed detection of products on the flow line through high-frequency shooting and rapid analysis of the camera, greatly shortens the quality inspection time of a single product, can meet the demand of high-speed production of the flow line, significantly improves the overall quality inspection efficiency, and reduces the labor cost of enterprises.

[0062] 3. The present application analyzes the working efficiency of the equipment based on product image information, can timely find problems existing in the working of the equipment, is convenient for enterprises to optimize equipment operation parameters or adjust production processes, and improves the utilization rate of the equipment and the overall production efficiency.

[0063] 4. When any one or more of the pass rate, work efficiency, and production line capacity fall below the preset value, an early warning will be triggered in a timely manner and adjustments will be made. This will enable a rapid response to abnormal situations in the production process, prevent problems from escalating, reduce production losses, and at the same time, maintain the production line's efficient and stable operation through automatic adjustments, thereby improving the flexibility and competitiveness of the enterprise's production. Attached Figure Description

[0064] Other features, objects, and advantages of the present invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0065] Figure 1 This is a flowchart of the equipment collaborative quality inspection method.

[0066] Figure 2 This is a flowchart of the equipment capacity control method. Detailed Implementation

[0067] The present invention will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present invention, but do not limit the invention in any way. It should be noted that those skilled in the art can make several changes and improvements without departing from the concept of the present invention. These all fall within the protection scope of the present invention.

[0068] Example 1

[0069] According to the present invention, a collaborative quality inspection method for equipment is provided, such as... Figure 1 As shown, this includes: when multiple devices are working on the production line, vision-based inspection is performed simultaneously with product evaluation; specifically, it includes the following steps:

[0070] Step S1: Obtain product image information from the production line and preprocess the obtained product image information to obtain preprocessed product image information;

[0071] Specifically, step S1 includes:

[0072] Step S1.1: Acquire product images on the production line using a camera through high-frequency shooting;

[0073] Step S1.2: Perform image enhancement, geometric correction and normalization processing on the acquired product image information from the production line to obtain the processed product image.

[0074] Step S2: Identify the product type based on the preprocessed product image information, obtain the corresponding product type quality inspection parameters; perform quality inspection on the products on the production line based on the quality inspection parameters to distinguish between qualified and unqualified products, and calculate the pass rate;

[0075] Specifically, the step S2 comprises:

[0076] Step S2.1: The preprocessed product image is subjected to product type recognition based on a convolutional neural network;

[0077] Step S2.2: The product quality inspection parameters of the corresponding product are obtained based on a standard database according to the recognized product type, and the product quality inspection parameters comprise color, length-width-height, and texture feature information;

[0078] Step S2.3: The preprocessed product image is subjected to image segmentation technology to separate the product from the background, the separated product is converted from an RGB color space to an HSV color space, the converted image is subjected to pixel-by-pixel scanning, the HSV value of each pixel point is counted, and the color distribution information of the product is obtained; the obtained product color distribution information is compared with the color standard in the quality inspection parameters, the similarity of the color distribution is calculated; when the similarity of the color distribution is less than a first threshold value, it is considered that the current product is an unqualified product;

[0079] Step S2.4: The edge contour of the product is extracted based on the preprocessed product image through Canny edge detection, the circumscribed matrix of the product is calculated according to the edge contour, the length and width of the product on the image plane are obtained, the actual height of the product is calculated through the principle of triangulation in combination with the position information of the product on the pipeline and the installation parameters of the camera, the length, width, and height of the product are compared with the length, width, and height standard in the quality inspection parameters, and the error of the length, width, and height is calculated respectively; when the error of any one or more of the length, width, and height is greater than a second threshold value, it is considered that the current product is an unqualified product;

[0080] Step S2.5: The texture feature information of the product is extracted by using a local binary pattern (LBP) algorithm, the texture feature information extracted is compared with the texture feature information in the quality inspection parameters, and the similarity of the texture feature is calculated; when the similarity of the texture feature is less than a third threshold value, it is considered that the current product is an unqualified product.

[0081] Step S3: The number of qualified products is counted to obtain the production capacity of the current pipeline;

[0082] Step S4: Device working efficiency analysis is performed based on the preprocessed product image information;

[0083] The preprocessed product image under the current device is obtained, and the current product is classified according to the obtained preprocessed product image under the current device, including processable products, unprocessable products, and processed products;

[0084] When the current product is classified as a processable product, the working efficiency of the current device is analyzed by the device downstream of the pipeline;

[0085] The process of analyzing the working efficiency of the current equipment through downstream equipment in the production line includes:

[0086] The system acquires pre-processed product images from downstream equipment on the production line and classifies the current products based on these images, including processable products, unprocessable products, and processed products.

[0087] When the number / percentage of processable products decreases, the number / percentage of processed products increases, and the number / percentage of unprocessable products remains stable, the current equipment is working effectively; when the number / percentage of processable products remains unchanged or increases, the current equipment's efficiency decreases.

[0088] More specifically, in this embodiment, taking a food processing production line as an example, firstly, we analyze the situation of the raw materials themselves;

[0089] Raw materials are recorded as they pass through the equipment's field of view, and the vision system identifies and categorizes them into three types: processable, unprocessable, and processed. Unprocessable materials are further subdivided based on the reasons for their unprocessability. Common reasons include: being too close to the source, sticking together, stacking, or being too large / too small. These are inherent properties of the raw materials that make them unprocessable. If this type of material constitutes a high percentage, the final production capacity will be affected. This impact cannot be resolved by adjusting the equipment's own parameters; therefore, an alarm on the control panel is used to alert on-site personnel for intervention, thereby improving the quality of the raw materials.

[0090] Then, analyze the condition of the raw materials after processing:

[0091] After the equipment processes raw materials visually marked as processable, downstream equipment on the production line will conduct quality inspections on that equipment. The overall logic of the quality inspection analysis is as follows:

[0092] After visual recognition, the types of raw materials within the field of view are classified and compared with upstream equipment. The overall trend should show a decreasing trend, meaning the quantity / percentage of processable raw materials decreases, the quantity / percentage of processed raw materials increases, and the quantity of unprocessable raw materials remains stable. If this data trend is observed, it indicates that the upstream production line equipment is working effectively and the raw material processing is efficient.

[0093] If the overall trend of the quantity / proportion of raw materials that can be processed remains unchanged or increases, it means that the working efficiency of the front-end equipment is decreasing. At this time, the cloud platform needs to adjust the rules and work configuration.

[0094] Step S5: When any one or more of the pass rate, work efficiency, and current production line capacity are lower than the preset value, an early warning is triggered, and the current production line is adjusted according to the preset requirements.

[0095] Specifically, the step S5 comprises: reporting the quality inspection analysis data to the cloud in real time, at this time, the device data of the same workshop and the same production line will be recorded and summarized, the cloud platform manages and maintains the device position information, etc.; the device quality inspection data is analyzed;

[0096] In the embodiment, when the current pipeline production capacity is lower than the preset value, the factory raw material needs to be adjusted through the display screen; the quality of the raw material is not caused by human, and is not controllable by the device.

[0097] When the qualified rate is lower than the preset value, the working angle of the robot is adjusted to make the qualified rate meet the preset requirement; the shape of the raw material after quality inspection is still abnormal, and the angle of the mechanical arm movement is adjusted according to the abnormal position and shape deviation direction.

[0098] When the device working efficiency is lower than the preset value, that is, the processed target in unit time does not reach the qualified level, first, the speed state of the pipeline is checked to exclude the decrease of the fish fillet flow caused by the slow speed, at this time, the speed can be increased; the recognition data of the vision and strategy is checked.

[0099] When the device working efficiency is lower than the preset value, it is checked whether the processable raw material is too little, and when the raw material is too little, the factory raw material needs to be adjusted.

[0100] When the device working efficiency is lower than the preset value, it is checked whether the strategy calculation time is too long, that is, the operation consumes the whole working time, at this time, the operation amount needs to be reduced to achieve the goal of fastest output processing result, and the accuracy is reduced. For example, n targets are analyzed at the same time, and n possibilities of each target are calculated, and the path with the fastest speed and the smallest error is finally selected. When the operation amount is too large, the working speed of the device is slow, at this time, the calculation thread needs to be reduced to improve the operation speed.

[0101] When the device working efficiency is lower than the preset value, the working time of the execution mechanism is checked, if the working time exceeds the qualified level, it represents that the robot execution speed is too slow, at this time, the speed, acceleration and jerk value of the device need to be adjusted, and the action speed of the cylinder, such as the speed of pressing down, grabbing and clamping, is improved, so that the fine motion speed is also accelerated.

[0102] The application also provides a device cooperative quality inspection system, which can be realized by executing the process steps of the device cooperative quality inspection method, that is, the device cooperative quality inspection method can be understood by those skilled in the art as the preferred embodiment of the device cooperative quality inspection system.

[0103] Embodiment 2

[0104] Embodiment 2 is a preferred example of embodiment 1

[0105] According to the device capacity regulation method provided by the application, as shown in the specification, the method comprises the following steps: Figure 2

[0106] Step 1: predicting the current pipeline capacity based on the raw material data and the device data through a prediction model;

[0107] Specifically, the step 1 comprises the following steps:

[0108] Step 1.1: constructing the prediction model based on a long short-term memory network;

[0109] Step 1.2: constructing a data set based on the collected raw material data, device data and historical capacity data;

[0110] Step 1.3: training the prediction model by using the constructed data set to obtain a trained prediction model;

[0111] Step 1.4: predicting the current pipeline capacity prediction value based on the current raw material data and device data by using the trained prediction model.

[0112] The raw material data comprises the type, quantity and quality parameters of the raw materials; the quality parameters comprise the size specification, shape feature and physical strength of the raw materials.

[0113] The device data comprises the model, quantity and operation parameters of the devices; the operation parameters comprise the working speed, time configuration of each action step, acceleration and working interval range of the devices, historical failure record and maintenance period of the devices.

[0114] The collected raw material data and device data are preprocessed, including standardization processing and abnormality processing.

[0115] Step 2: judging whether the capacity in a preset time period can meet a preset requirement based on the predicted current pipeline capacity, and determining a regulation strategy according to the judgment result;

[0116] Specifically, the step 2 comprises the following steps:

[0117] When it is judged that the capacity in the preset time period cannot meet the preset requirement, the current pipeline is adjusted based on the preset regulation strategy, so that the capacity is improved and the improved capacity can meet the preset requirement, and the quality of the target product is reduced.

[0118] When it is judged that the capacity in the preset time period exceeds the preset requirement, the current pipeline is adjusted based on the preset regulation strategy, so that the capacity is reduced and the reduced capacity can meet the preset requirement, and the quality of the target product is improved.

[0119] ​The step of adjusting the current production line based on a preset control strategy to increase production capacity so that the increased capacity can meet preset requirements, while reducing the quality of the target product, includes:

[0120] Expanding the camera's field of view increases the number of products to be identified, thereby increasing the number of workable targets. Based on the increased number of workable targets, the working speed and working range of the robot on the production line are adjusted to improve productivity. The working speed of the robot on the production line includes the robot's movement speed, movement acceleration, and downward clamping speed. With the expansion of the robot's working range, the corresponding robot working speed also increases, thus increasing productivity.

[0121] Lowering the threshold of the camera's optical sensor reduces the image matching accuracy requirements and shortens the single-frame processing time, thereby increasing the number of target products that meet the preset requirements.

[0122] Optimize the robot path, prioritize processing near-end materials, and match it with the increased camera cycle speed;

[0123] The device clock is synchronized at the microsecond level using the IEEE 1588 protocol. Once the camera completes recognition, it immediately triggers the robot's action. The robot's return signal controls the conveyor belt feeding, eliminating upstream and downstream waiting.

[0124] In this embodiment, taking fish fillets as an example, the area pressed down by the end clamp is fixed. When this value is relaxed, the end may press down on part of the fish fillets, but the number of fish fillets to be filtered will decrease, thus increasing the production capacity.

[0125] Adjust the shaping priority based on the fish raft arrival situation and staffing ratio on the day to achieve the expected production capacity:

[0126] When the quality of incoming materials is generally in line with requirements, but the factory has insufficient manpower, the distance and centerline weights are adjusted to allow the robot to prioritize the shaping of fish fillets in positions that are difficult for humans to reach. This human-robot collaboration improves the working rhythm of both humans and robots.

[0127] When the staffing ratio is sufficient, by adjusting the similarity weight to be higher, the robot will prioritize the removal of targets that the AI ​​considers to be fish rafts and that are closer to the robot, thereby increasing the removal cycle and thus increasing production capacity. At this time, the requirement for similarity in AI recognition has been reduced, and the number of removals performed by the robot will naturally increase.

[0128] The step of adjusting the current production line based on a preset control strategy to reduce capacity while ensuring the reduced capacity meets preset requirements and improves the quality of the target product includes:

[0129] Increase the zoom lens precision of the camera, for example: similarity threshold from 0.9 to 0.95; and exclude abnormal raw materials that do not meet the preset requirements through the aspect ratio filtering algorithm; improve the reference standard of the target product, and output stable quality products;

[0130] Ensure that the robot working speed is above the basic baseline, control the robot motion precision, compensate the pose executed by the end clamp, and thus improve the quality of the target product; in this embodiment, the force control sensor precision (±0.1N) and the servo reducer (speed-25%) of the robot are adjusted;

[0131] Control the threshold parameters of adjacent target products to realize accurate filtering of adjacent target products;

[0132] Add a physical buffer zone on the belt line, and temporarily store raw materials when the robot slows down; when the packaging station is ready, the buffered raw materials are processed first to realize beat self-adaptation. Through the elastic buffer mechanism, 3-5 pieces of physical buffer zone are added to the belt line, and when the robot slows down, the buffer zone temporarily stores raw materials to avoid accumulation.

[0133] In this embodiment, taking fish fillets as an example, the intensity of robot shaping (pressing down and clamping) is adjusted to ensure that each fish fillet is clamped and pressed down in place, more in line with the standard of fish fillets, and improve the quality of fish fillet shaping.

[0134] Increase the similarity threshold of fish fillets, and accurately filter unqualified fish fillet raw materials. Some non-standard and abnormal fish fillets are completely filtered out, and only raw materials close to standard fish fillets are worked on to improve the quality of work.

[0135] Filter out large and small fish fillets, and AI identifies that such fish fillets cannot approach the standard fish fillet pattern after shaping.

[0136] During AI identification of fish fillets, basic information such as the area, length and width, and color of standard fish fillets is collected, and the fish fillet shaping reference template is continuously optimized as a reference for fish fillet shaping, to improve the quality of the template and make subsequent shaping closer to the standard shaped template.

[0137] Step 3: Adjust the current assembly line based on the control strategy, so that the productivity of the current assembly line in the preset time period meets the preset requirements.

[0138] The present application also provides a device productivity control system, which can be realized by executing the process steps of the device productivity control method, i.e. the device productivity control method can be understood by those skilled in the art as the preferred embodiment of the device productivity control system.

[0139] In the production capacity regulation process, when the production capacity needs to be increased, the recognition quantity of products to be processed can be quickly increased by expanding the visual field area recognized by the camera, so as to provide sufficient processing targets for subsequent processes; at the same time, the similarity threshold of target products is reduced, and the recognition standard of target products is relaxed, so as to further increase the production capacity. Based on this, the working speed and range of the robot are adjusted, so that the robot can operate the target products more efficiently; since the regulation strategy is based on the prediction and analysis of the overall production capacity of the assembly line, the operation efficiency and range of the robot at the back end are improved while the front-end recognition is increased, so as to avoid the accumulation of products to be processed due to the increase of the front-end quantity; when the production capacity needs to be reduced and the product quality needs to be improved, the motion accuracy of the robot is controlled, the end clamp execution pose is compensated, and the accuracy of the target product is ensured; by controlling the adjacent fish fillet threshold parameter and increasing the similarity threshold, the accurate filtering and screening of the fish fillet are realized. These regulation methods improve the product quality on the premise that the working speed of the robot is within a reasonable benchmark, and do not affect the upstream feeding rhythm due to excessive reduction of speed, causing material accumulation or downstream waiting, realizing seamless connection and efficient cooperation between upstream and downstream processes of the assembly line, greatly improving production efficiency and product quality, and reducing labor regulation cost and production resource waste.

[0140] Those skilled in the art know that, in addition to implementing the system, device and each module thereof provided by the present application in the form of pure computer readable program code, the same program can also be realized by logically programming the method steps to form a logic gate, a switch, an application specific integrated circuit, a programmable logic controller and an embedded microcontroller, etc. Therefore, the system, device and each module thereof provided by the present application can be considered as a hardware component, and the modules included therein for realizing various programs can also be considered as structures within the hardware component; the modules for realizing various functions can also be considered as both software programs for realizing methods and structures within the hardware component.

[0141] The specific embodiments of the present application are described above. It should be understood that the present application is not limited to the above specific embodiments, and those skilled in the art can make various changes or modifications within the scope of the claims, which does not affect the essential content of the present application. The embodiments of the present application and the features in the embodiments can be arbitrarily combined with each other without conflict.

Claims

1. A device collaborative quality inspection method, characterized in that, The method comprises the following steps: Step S1: obtaining product image information on a production line and pre-processing the obtained product image information to obtain pre-processed product image information; Step S2: identifying the product type based on the pre-processed product image information and obtaining quality inspection parameters of the corresponding product type; performing quality inspection on the products on the production line based on the quality inspection parameters to distinguish qualified products and unqualified products and calculate the qualified rate; Step S3: counting the number of qualified products to obtain the production capacity of the current production line; Step S4: performing equipment working efficiency analysis based on the pre-processed product image information; Step S5: when any one or more of the qualified rate, the working efficiency and the current production line capacity is lower than the preset value, triggering an early warning and adjusting the current production line according to the preset requirements.

2. The device coordination quality inspection method of claim 1, wherein, The step S1 comprises: Step S1.1: obtaining product images on the production line based on the camera through high-frequency shooting meeting the preset requirements; Step S1.2: performing image enhancement, geometric correction and normalization processing on the obtained product image information on the production line to obtain processed product images.

3. The device coordination quality inspection method of claim 1, wherein, The step S2 comprises: Step S2.1: performing product type identification on the pre-processed product image based on a convolutional neural network; Step S2.2: obtaining the quality inspection parameters of the corresponding product based on a standard database according to the identified product type, wherein the quality inspection parameters include color, length, width, height and texture feature information; Step S2.3: separating the product from the background through image segmentation technology based on the pre-processed product image, converting the separated product from the RGB color space to the HSV color space, performing pixel-by-pixel scanning on the converted image, counting the HSV value of each pixel point to obtain the color distribution information of the product, comparing the obtained color distribution information of the product with the color standard in the quality inspection parameters, and calculating the similarity of the color distribution; when the similarity of the color distribution is less than a first threshold value, the current product is considered to be an unqualified product; Step S2.4: extracting the edge contour of the product based on the pre-processed product image through Canny edge detection, calculating the circumscribed matrix of the product according to the edge contour to obtain the length and width of the product in the image plane; then combining the position information of the product on the production line and the installation parameters of the camera, the actual height of the product is calculated through the principle of triangulation; comparing the length, width and height of the product obtained with the length, width and height standard in the quality inspection parameters to calculate the error of the length, width and height respectively; when any one or more of the length, width and height errors is greater than a second threshold value, the current product is considered to be an unqualified product; Step S2.5: extracting the texture feature information of the product by using the local binary pattern (LBP) algorithm, comparing the extracted texture feature information with the texture feature information in the quality inspection parameters, and calculating the similarity of the texture features; when the similarity of the texture features is less than a third threshold value, the current product is considered to be an unqualified product.

4. The device coordination quality inspection method of claim 1, wherein, The step S4 comprises: obtaining pre-processed product images under the current equipment and classifying the current products according to the obtained pre-processed product images under the current equipment, including processable products, unprocessable products and processed products; When the current product is classified as a processable product, the current device is analyzed for work efficiency by a device downstream of the assembly line; The analysis of the work efficiency of the current device by the device downstream of the assembly line comprises: The pre-processed product image downstream of the device is obtained, and the current product is classified according to the obtained pre-processed product image downstream of the device, including processable products, unprocessable products and processed products; When the number / percentage of processable products decreases, the number / percentage of processed products increases, and the trend of the number / percentage of unprocessable products is stable, the current device is working effectively; when the trend of the number / percentage of processable products is unchanged or increases, the work efficiency of the current device decreases.

5. The device coordination quality inspection method of claim 1, wherein, The step S5 comprises: When the qualified rate is lower than the preset value, the working angle of the robot is adjusted to make the qualified rate meet the preset requirement; When the work efficiency of the device is lower than the preset value, the speed state of the assembly line is obtained, and it is judged whether the speed of the assembly line is lower than the preset value, so that the product output of the current product is reduced, and if so, the speed of the assembly line is increased to increase the product output; When the work efficiency of the device is lower than the preset value, it is checked whether the amount of processable raw materials is too small; when it is checked that the amount of processable raw materials is too small, the raw materials are adjusted; When the work efficiency of the device is lower than the preset value, it is checked whether the strategy calculation time is too long, and when it is checked that the operation consumes the whole working time, the operation amount is reduced to output the processing result as soon as possible, and the accuracy is reduced; When the work efficiency of the device is lower than the preset value, the working time of the execution mechanism is checked, and if the working time exceeds the qualified level, it means that the execution speed of the robot is too slow, and the execution speed of the robot is adjusted at this time.

6. A device coordination quality inspection system, characterized in that, Comprise: Module M1: obtaining product image information on the assembly line, and pre-processing the obtained product image information to obtain pre-processed product image information; Module M2: identifying the product type based on the pre-processed product image information, obtaining the quality inspection parameters of the corresponding product type; based on the quality inspection parameters, the products on the assembly line are inspected to distinguish qualified products and unqualified products, and the qualified rate is calculated; Module M3: counting the number of qualified products to obtain the production capacity of the current assembly line; Module M4: analyzing the work efficiency of the device based on the pre-processed product image information; Module M5: when any one or more of the qualified rate, work efficiency and current assembly line production capacity is lower than the preset value, a warning is triggered, and the current assembly line is adjusted according to the preset requirement.

7. The device collaboration quality inspection system of claim 6, wherein, The module M1 comprises: Module M1.1: based on the camera, the product image on the assembly line is obtained by high-frequency shooting that meets the preset requirement; Module M1.2: the obtained product image information on the assembly line is processed, including image enhancement, geometric correction and normalization, to obtain the processed product image.

8. The device collaboration quality inspection system of claim 6, wherein, The module M2 comprises: Module M2.1: the pre-processed product image is identified for product type based on a convolutional neural network; Module M2.2: according to the identified product type, the quality inspection parameters of the corresponding product are obtained based on a standard database, and the quality inspection parameters include color, length, width, height and texture feature information; Module M2.3: The pre-processed product image is separated from the background by image segmentation technology, the separated product is converted from RGB color space to HSV color space, the converted image is scanned pixel by pixel, the HSV value of each pixel point is counted, and the color distribution information of the product is obtained; the obtained product color distribution information is compared with the color standard in the quality inspection parameter, and the similarity of the color distribution is calculated; when the similarity of the color distribution is less than the first threshold value, it is considered that the current product is unqualified product; Module M2.4: Based on the pre-processed product image, the edge contour of the product is extracted by Canny edge detection, the circumscribed matrix of the product is calculated according to the edge contour, and the length and width of the product on the image plane are obtained; combined with the position information of the product on the pipeline and the installation parameters of the camera, the actual height of the product is calculated by the principle of triangulation; the length, width and height of the product obtained are compared with the length, width and height standard in the quality inspection parameter, and the errors of length, width and height are calculated respectively; when the error of any one or more of length, width and height is greater than the second threshold value, it is considered that the current product is unqualified product; Module M2.5: The texture feature information of the product is extracted by using local binary pattern (LBP) algorithm, and the texture feature information extracted is compared with the texture feature information in the quality inspection parameter to calculate the similarity of the texture feature; when the similarity of the texture feature is less than the third threshold value, it is considered that the current product is unqualified product.

9. The device collaboration quality inspection system of claim 6, wherein, The module M4 comprises: Obtaining the pre-processed product image under the current device, and classifying the current product according to the obtained pre-processed product image under the current device, including: processable product, unprocessable product and processed product; When the current product is classified as processable product, the working efficiency of the current device is analyzed by the device downstream of the pipeline; The working efficiency of the current device is analyzed by the device downstream of the pipeline, comprising: Obtaining the pre-processed product image under the device downstream of the pipeline, and classifying the current product according to the obtained pre-processed product image under the device downstream of the pipeline, including processable product, unprocessable product and processed product; When the number / percentage of processable products decreases, the number / percentage of processed products increases, and the number / percentage of unprocessable products tends to be stable, the current device works effectively; when the number / percentage of processable products remains unchanged or increases, the working efficiency of the current device decreases.

10. The device collaboration quality inspection system of claim 6, wherein, The module M5 comprises: When the qualified rate is lower than the preset value, the working angle of the robot is adjusted to make the qualified rate meet the preset requirement; When the working efficiency of the device is lower than the preset value, the speed state of the pipeline is obtained, and it is judged whether the speed of the pipeline is lower than the preset value to reduce the product yield if the current product yield is reduced; when the working efficiency of the device is lower than the preset value, it is checked whether the processable raw material is too little; when it is checked that the processable raw material is too little, the raw material is adjusted; When the working efficiency of the device is lower than the preset value, it is checked whether the processable raw material is too little; when it is checked that the processable raw material is too little, the raw material is adjusted; When the device efficiency is lower than the preset value, check whether the strategy calculation time is too long. When it is found that the calculation consumes the overall working time, reduce the calculation amount, aim at the fastest output processing result, and reduce the accuracy. When the device efficiency is lower than the preset value, check the working time of the execution mechanism. If the working time exceeds the qualified level, it represents that the robot execution speed is too slow. At this time, the execution speed of the robot is adjusted.

Images