Intelligent fusion terminal sorting method and system based on visual recognition

Abstract

This application relates to a sorting method and system for intelligent fusion terminals based on visual recognition, belonging to the technical field of computer vision. The method includes controlling a pre-set appearance sorting device to inspect the appearance of a pre-set intelligent fusion terminal to generate an initial inspection result; determining whether the initial inspection result is consistent with a pre-set qualified inspection result; if consistent, determining the qualified inspection result as the pre-set final inspection result, and continuing to control the appearance sorting device to inspect the appearance of the next intelligent fusion terminal to generate an initial inspection result; if inconsistent, controlling a pre-set reflective suppression component to assist the appearance sorting device in performing a secondary inspection of the intelligent fusion terminal's appearance to generate a final inspection result; collecting top surface data of the intelligent fusion terminal; and controlling the appearance sorting device to sort the intelligent fusion terminals based on the top surface data and the final inspection result. This application has the effect of improving sorting accuracy and reducing scrap rate.

Description

Technical Field

[0001] This application relates to the field of computer vision technology, and in particular to a sorting method and system for intelligent fusion terminals based on visual recognition. Background Technology

[0002] A six-axis articulated robot is an industrial robot with six rotary joints that mimics the dexterity of a human arm, enabling it to accurately complete tasks in various complex situations. A single end effector refers to a six-axis articulated robot being equipped with only one type of actuator at its end effector, including a rigid suction cup or a mechanical gripper. A rigid suction cup is an actuator that uses external pressure difference to attract objects, while a mechanical gripper uses the opening and closing motion of a mechanical structure to clamp objects.

[0003] In related technologies, visual recognition technology uses a camera to capture images of the appearance of a smart fusion terminal, and then uses algorithms to analyze the features in the images to automatically determine whether the product is qualified.

[0004] Regarding the aforementioned technologies, when analyzing the appearance of products under test using visual recognition technology, imaging of reflective and translucent material surfaces is difficult, increasing the probability of missed or incorrect detections and thus reducing sorting accuracy. Furthermore, when using a single end effector to grasp the product under test, unexpected situations such as path interference or collisions may cause the product to fall off or experience increased force on one side, resulting in damage and increasing the scrap rate. There is still room for improvement. Summary of the Invention

[0005] To improve sorting accuracy and reduce scrap rate, this application provides a sorting method and system based on visual recognition intelligent fusion terminal.

[0006] Firstly, this application provides a sorting method for an intelligent fusion terminal based on visual recognition, employing the following technical solution:

[0007] A sorting method for an intelligent fusion terminal based on visual recognition, comprising:

[0008] The preset appearance sorting device is controlled to inspect the appearance of the preset intelligent fusion terminal to generate initial inspection results;

[0009] Determine whether the initial test result is consistent with the preset qualified test result;

[0010] If they match, the qualified test result is determined as the preset final test result, and the appearance sorting device continues to be controlled to inspect the appearance of the next intelligent fusion terminal to generate the initial test result.

[0011] If there is a discrepancy, the preset reflective suppression component will be controlled to assist the appearance sorting device in performing a secondary inspection of the appearance of the smart fusion terminal in order to generate the final inspection result.

[0012] Collect top-surface data from the intelligent converged terminal;

[0013] Based on the top surface data and the final inspection results, the appearance sorting device is controlled to sort the intelligent fusion terminals.

[0014] By adopting the above technical solution, it is determined whether the initial test result is consistent with the qualified test result. If they are consistent, the qualified test result is determined as the final test result, and the appearance sorting device continues to be controlled to test the appearance of the next intelligent fusion terminal to obtain the initial test result. If they are inconsistent, the reflective suppression component is controlled to assist the appearance sorting device in performing a second test on the appearance of the intelligent fusion terminal to generate the final test result. Thus, based on the top surface data and the final test result, the appearance sorting device is controlled to sort the intelligent fusion terminal, thereby improving the sorting accuracy and reducing the scrap rate.

[0015] Optionally, the reflection suppression component includes a light-shielding plate and a polarizer. The step of controlling the preset reflection suppression component to assist the appearance sorting device in performing a secondary inspection of the appearance of the intelligent fusion terminal to generate the final inspection result includes:

[0016] Determine whether the detection result is consistent with the preset single fuzzy defect detection result;

[0017] If there is a discrepancy, the preset non-conforming test result will be determined as the final test result;

[0018] If they match, obtain the rotation angle of the light-shielding plate, the target center position of the light-shielding plate, and the rotation angle of the polarizer;

[0019] The movement and deflection angle of the light-shielding plate are controlled according to the rotation angle of the light-shielding plate and the target center position of the light-shielding plate, and the deflection of the polarizer is controlled according to the rotation angle of the polarizer.

[0020] The appearance sorting device is controlled to inspect the appearance of the intelligent fusion terminal in order to generate a suppressed inspection result;

[0021] The final detection result is determined based on the detection results after inhibition.

[0022] By adopting the above technical solution, it is determined whether the detection result is consistent with the single fuzzy defect detection result. If they are inconsistent, the unqualified detection result is determined as the final detection result. If they are consistent, the rotation angle of the light shield, the target center position of the light shield, and the rotation angle of the polarizer are obtained. The light shield is moved and deflected according to the rotation angle and the target center position of the light shield. The polarizer is deflected according to the rotation angle. The appearance sorting device is controlled to detect the appearance of the intelligent fusion terminal to obtain the suppressed detection result. The final detection result is determined according to the suppressed detection result. Thus, the influence of reflection on the detection result is eliminated through the synergistic effect of the light shield and the polarizer. Then, the intelligent fusion terminal is sorted according to the final detection result.

[0023] Optionally, the steps for obtaining the rotation angle of the light-shielding plate, the target center position of the light-shielding plate, and the rotation angle of the polarizer include:

[0024] Collect the appearance image data of the intelligent fusion terminal, and perform grayscale and edge contour quantization calculation on the appearance image data through a preset image feature extraction algorithm. Then compare the calculation results with the preset normal calculation data to determine the diagonal position of the blurred area.

[0025] The diagonal position of the blurred region is calculated based on the arctangent function to generate the rotation angle of the light-shielding plate;

[0026] The diagonal position of the fuzzy region is calculated based on the preset midpoint coordinate formula to generate the center position of the fuzzy region.

[0027] Subtract the x-coordinate and y-coordinate of the diagonal position of the blurred region to generate the width and height of the blurred region.

[0028] The target center position of the light shield is generated by calculating based on the rotation angle of the light shield, the center position of the blurred area, the width of the blurred area, the height of the blurred area, the preset width of the light shield, and the preset height of the light shield.

[0029] The direction of surface light wave vibration is collected;

[0030] The polarizer rotation angle is generated by adding the surface light wave vibration direction to a preset vertical angle.

[0031] By adopting the above technical solution, appearance image data of the intelligent fusion terminal is collected, and grayscale and edge contour quantization calculations are performed on the appearance image data using image feature extraction algorithms. The calculation results are compared with normal calculation data to obtain the diagonal position of the blurred area. The rotation angle of the light shield is calculated based on the arctangent function of the diagonal position of the blurred area. The center position of the blurred area is calculated based on the midpoint coordinate formula. The width and height of the blurred area are obtained by subtracting the x and y coordinates of the diagonal position of the blurred area. The target center position of the light shield is calculated based on the rotation angle of the light shield, the center position of the blurred area, the width and height of the blurred area, the width and height of the light shield. The rotation angle of the polarizer is obtained by adding the vibration direction of the surface light wave to the vertical angle. Thus, the movement and deflection angle of the light shield are controlled according to the rotation angle and the target center position of the light shield. The deflection of the polarizer is controlled according to the rotation angle of the polarizer. In this way, the influence of reflection on the detection results is eliminated through the synergistic effect of the light shield and the polarizer.

[0032] Optionally, the steps for determining the final detection result based on the post-suppression detection results include:

[0033] Determine whether the test results after inhibition are consistent with the qualified test results;

[0034] If they match, the qualified test result will be determined as the final test result;

[0035] If there is a discrepancy, the non-compliant test result will be determined as the final test result.

[0036] By adopting the above technical solution, it is determined whether the suppressed detection result is consistent with the qualified detection result. If they are consistent, the qualified detection result is determined as the final detection result; if they are inconsistent, the unqualified detection result is determined as the final detection result. This allows the appearance sorting device to sort the intelligent fusion terminal according to the final detection result, thereby improving the accuracy of the sorting result.

[0037] Optionally, the appearance sorting device includes a gripping component and an auxiliary gripping component. The steps of controlling the appearance sorting device to sort the intelligent fusion terminal based on the top surface data and the final detection result include:

[0038] Determine whether the top surface data meets the preset top surface adsorption requirements;

[0039] If the conditions are not met, the intelligent fusion terminal will be flipped by the grasping component based on the top surface data in order to collect the top surface data after flipping, and the top surface data after flipping will be determined as the top surface data for cyclic judgment.

[0040] If the conditions are met, the optimal sorting path is calculated using kinematic models and dynamic programming algorithms based on the preset qualified area location, the preset defective area location, and the final inspection results.

[0041] The optimal sorting path is used to control the auxiliary gripping component to adhere to the top surface of the intelligent fusion terminal, and to control the gripping component to grip the intelligent fusion terminal.

[0042] By adopting the above technical solution, it is determined whether the top surface data meets the top surface adsorption requirements. If not, the gripping component is controlled to flip the intelligent fusion terminal according to the top surface data to collect the top surface data after flipping, and the flipped top surface data is determined as the top surface data for cyclic judgment. If it meets the requirements, the optimal sorting path is calculated using kinematic model and dynamic programming algorithm based on the qualified area position, the defective area position and the final detection result. This allows the auxiliary gripping component to adsorb the top surface of the intelligent fusion terminal according to the optimal sorting path, thereby enhancing the stability of the gripping component in grasping the intelligent fusion terminal and reducing the scrap rate.

[0043] Optionally, the step of controlling the grasping component to flip the intelligent fusion terminal based on the top surface data to collect the top surface data after flipping includes:

[0044] The top surface number is found in the preset surface data number correspondence relationship based on the top surface data;

[0045] Find the smooth surface data and the corresponding smooth surface number in the surface data sequence number correspondence;

[0046] Find the smooth surface area in the smooth surface data;

[0047] The smooth surface areas are sorted and selected to determine the minimum smooth surface number corresponding to the minimum smooth surface area.

[0048] Collect the actual center position of the top surface, and collect the center position of the minimum smooth surface according to the minimum smooth surface index;

[0049] Calculate the difference between the coordinate values ​​on the corresponding axes of the actual top surface center position and the minimum smooth surface center position to generate a flip vector;

[0050] The actual top surface center position and the minimum smooth surface center position are substituted into the preset vector dot product formula to calculate the flip angle.

[0051] The gripping component is controlled by the flip angle and flip vector to flip the intelligent fusion terminal and collect the top surface data after flipping.

[0052] By adopting the above technical solution, the top surface number is found in the surface data sequence correspondence relationship based on the top surface data. The smooth surface data and corresponding smooth surface number are found in the surface data sequence correspondence relationship. The smooth surface area is found in the smooth surface data. The smooth surface areas are sorted and selected to obtain the minimum smooth surface number corresponding to the minimum smooth surface area. The center position of the minimum smooth surface is collected based on the minimum smooth surface number. The difference between the coordinate values ​​on the corresponding coordinate axis of the actual top surface center position and the minimum smooth surface center position is calculated to generate a flip vector. The actual top surface center position and the minimum smooth surface center position are substituted into the vector dot product formula to generate a flip angle. Based on the flip angle and the flip vector, the grasping component is controlled to flip the intelligent fusion terminal, and then the auxiliary grasping component is controlled to adsorb the top surface of the intelligent fusion terminal.

[0053] Optionally, the steps of controlling the auxiliary gripping component to adhere to the top surface of the intelligent fusion terminal according to the optimal sorting path, and controlling the gripping component to grip the intelligent fusion terminal, include:

[0054] The top surface dimension data is obtained by searching within the top surface data.

[0055] Calculate the difference between the top surface dimension data and the preset auxiliary gripping component dimension data to generate the auxiliary gripping component dimension difference;

[0056] Calculate the hypotenuse lengths corresponding to the lengths of the two right-angled sides in the size difference of the auxiliary gripping component to generate the distance the auxiliary gripping component needs to move;

[0057] The movement of the auxiliary gripping component is controlled according to the required moving distance, so as to enhance the stability of the auxiliary gripping component's adsorption on the top surface of the smart fusion terminal.

[0058] By adopting the above technical solution, the top surface size data is obtained by searching in the top surface data, the difference between the top surface size data and the preset auxiliary gripping component size data is calculated to obtain the auxiliary gripping component size difference value, and the hypotenuse length corresponding to the length of the two right-angled sides in the auxiliary gripping component size difference value is calculated to obtain the distance that the auxiliary gripping component needs to move. Thus, the movement of the auxiliary gripping component is controlled according to the distance that the auxiliary gripping component needs to move, so that the auxiliary gripping component can adsorb the top surface of the smart fusion terminal to the maximum extent, thereby enhancing the stability of the auxiliary gripping component adsorbing the top surface of the smart fusion terminal.

[0059] Secondly, this application provides a visual recognition-based intelligent fusion terminal sorting system, which adopts the following technical solution:

[0060] A sorting method and system for an intelligent fusion terminal based on visual recognition, comprising:

[0061] The data acquisition module is used to collect data from the top surface.

[0062] A memory for storing the program of the visual recognition-based intelligent fusion terminal sorting method as described in any of the above;

[0063] The processor and the program in the memory can be loaded and executed by the processor to implement the intelligent fusion terminal sorting method based on vision recognition as described in any of the above.

[0064] By adopting the above technical solution, the processor loads and executes a program stored in the memory for a visual recognition-based intelligent fusion terminal sorting method. This program controls the acquisition module to obtain a series of data related to the implementation of visual recognition-based intelligent fusion terminal sorting, thereby determining whether the initial detection result is consistent with the qualified detection result. If they are consistent, the qualified detection result is determined as the final detection result, and the appearance sorting device continues to be controlled to detect the appearance of the next intelligent fusion terminal to obtain the initial detection result. If they are inconsistent, the reflective suppression component is controlled to assist the appearance sorting device in performing a secondary detection of the intelligent fusion terminal's appearance to generate the final detection result. Based on the top surface data and the final detection result, the appearance sorting device is controlled to sort the intelligent fusion terminals, thereby improving sorting accuracy and reducing the scrap rate.

[0065] In summary, this application includes at least one of the following beneficial technical effects:

[0066] 1. By determining whether the initial inspection result is consistent with the qualified inspection result, if they are consistent, the qualified inspection result is determined as the final inspection result, and the appearance sorting device continues to be controlled to inspect the appearance of the next intelligent fusion terminal to obtain the initial inspection result; if they are inconsistent, the reflective suppression component is controlled to assist the appearance sorting device in performing a second inspection of the appearance of the intelligent fusion terminal to generate the final inspection result. Thus, based on the top surface data and the final inspection result, the appearance sorting device is controlled to sort the intelligent fusion terminals, thereby improving the sorting accuracy and reducing the scrap rate.

[0067] 2. By judging whether the detection result is consistent with the single fuzzy defect detection result, if they are inconsistent, the unqualified detection result is determined as the final detection result; if they are consistent, the rotation angle of the light shield, the target center position of the light shield, and the rotation angle of the polarizer are obtained. The light shield is moved and deflected according to the rotation angle and target center position of the light shield, and the polarizer is deflected according to the rotation angle of the polarizer. The appearance sorting device is controlled to detect the appearance of the intelligent fusion terminal to obtain the suppressed detection result. The final detection result is determined according to the suppressed detection result. Thus, the influence of reflection on the detection result is eliminated through the synergistic effect of the light shield and the polarizer, and the intelligent fusion terminal is sorted according to the final detection result.

[0068] 3. By determining whether the top surface data meets the top surface adsorption requirements, if not, the gripping component is controlled to flip the intelligent fusion terminal to collect the top surface data after flipping, and the flipped top surface data is determined as the top surface data for cyclic judgment; if it meets the requirements, the optimal sorting path is calculated using kinematic model and dynamic programming algorithm based on the qualified area position, the defective area position and the final detection result, thereby controlling the auxiliary gripping component to adsorb the top surface of the intelligent fusion terminal according to the optimal sorting path, thereby enhancing the stability of the gripping component in gripping the intelligent fusion terminal and reducing the scrap rate. Attached Figure Description

[0069] Figure 1 This is a flowchart of a sorting method for an intelligent fusion terminal based on visual recognition, as described in an embodiment of this application.

[0070] Figure 2 This is a flowchart of the steps in this application embodiment to control a preset reflective suppression component to assist the appearance sorting device in performing secondary inspection of the appearance of the smart fusion terminal in order to generate the final inspection result.

[0071] Figure 3 This is a flowchart illustrating the steps for obtaining the rotation angle of the light-shielding plate, the target center position of the light-shielding plate, and the rotation angle of the polarizer in embodiments of this application.

[0072] Figure 4 This is a flowchart of the steps for determining the final detection result based on the suppressed detection result in the embodiments of this application.

[0073] Figure 5 This is a flowchart of the steps in this application embodiment to control the appearance sorting device to sort the intelligent fusion terminal based on the top surface data and the final detection result.

[0074] Figure 6 This is a flowchart of the steps in this application embodiment to control the grasping component to flip the intelligent fusion terminal according to the top surface data in order to collect the top surface data after flipping.

[0075] Figure 7 This is a flowchart of the steps in this application embodiment to control the auxiliary grasping component to adsorb the top surface of the intelligent fusion terminal according to the optimal sorting path, and to control the grasping component to grasp the intelligent fusion terminal. Detailed Implementation

[0076] To make the purpose, technical solution, and advantages of this application clearer, the following description is provided in conjunction with the appendix. Figures 1 to 7 The present application will be further described in detail below with reference to embodiments. It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of the application.

[0077] This application discloses a sorting method for an intelligent fusion terminal based on visual recognition. This method primarily addresses the issues of improving sorting accuracy and reducing scrap rate. Specifically, it discloses an intelligent fusion terminal, an appearance sorting device, a reflective suppression component, and a processing terminal. The processing terminal is communicatively connected to both the reflective suppression component and the appearance sorting device to achieve data interaction and control. After the appearance sorting device transmits the collected detection results to the processing terminal, the processing terminal compares the detection results with normal detection results. When the detection results are inconsistent with normal detection results, the processing terminal controls the reflective suppression component to assist the appearance sorting device in performing a secondary detection of the intelligent fusion terminal's appearance to generate a final detection result. Based on the final detection result, the processing terminal controls the appearance sorting device to adjust the sorting process of the intelligent fusion terminal. This aims to quickly and reasonably control the appearance sorting device to adjust the sorting process of the intelligent fusion terminal, thereby improving sorting accuracy and reducing scrap rate.

[0078] Reference Figure 1 This application discloses a sorting method for an intelligent fusion terminal based on visual recognition, comprising the following steps:

[0079] Step S100: Control the preset appearance sorting device to detect the appearance of the preset intelligent fusion terminal to generate initial detection results.

[0080] The initial detection result refers to the appearance inspection conclusion of the intelligent fusion terminal after data processing by the multimodal image acquisition component in the appearance sorting device. This includes three types of detection results: qualified detection result, single fuzzy defect detection result, and other mixed defect detection results. The multimodal image acquisition component in the appearance sorting device collects data from each surface of the intelligent fusion terminal, and the collected data is input into an algorithm to obtain integrated data. In one embodiment, the algorithm includes a multi-source data registration algorithm, a cross-modal feature extraction algorithm, a multi-dimensional feature fusion algorithm, and a defect classification and recognition algorithm. The processing terminal then compares the integrated data with the normal integrated data of the intelligent fusion terminal. If the processing terminal determines that the integrated data matches the normal integrated data, the initial detection result is a qualified detection result. If the processing terminal determines that the integrated data does not match the normal integrated data, it queries the integrated data for data that is inconsistent with the normal integrated data. If only the surface image is inconsistent with the normal surface image, the initial detection result is a single fuzzy defect detection result; otherwise, it is an other mixed defect detection result.

[0081] A qualified inspection result refers to the judgment result when the appearance of the intelligent fusion terminal fully meets the qualified inspection requirements. A single blurry defect inspection result refers to the judgment result when the appearance of the intelligent fusion terminal only has one defect: unclear surface image. Other mixed defect inspection results refer to the judgment result when the appearance of the intelligent fusion terminal has other defects besides avoiding unclear image.

[0082] The appearance sorting device refers to an integrated device that can automatically identify appearance defects of intelligent fusion terminals and sort them according to the degree of defects. It includes a multimodal image acquisition component, a six-axis articulated robot, a gripping component, and an auxiliary gripping component. The multimodal image acquisition component is a device that collects multi-dimensional optical information of the surface of the intelligent fusion terminal through the collaborative action of multiple modules. It consists of a 2D high-resolution color camera, a 3D structured light sensor, an infrared thermal imager, a polarization imaging unit, and a trigger controller. When the intelligent fusion terminal enters the acquisition range of the multimodal image acquisition module, the trigger controller activates all the above sensor modules to detect the intelligent fusion terminal, inputs the detection data into the algorithm to obtain integrated data, and then compares the integrated data with normal integrated data through the processing terminal to obtain the detection result of the appearance of the intelligent fusion terminal.

[0083] A six-axis articulated robot is an industrial robot with six rotary joints that can mimic the flexibility of a human arm. The range of motion of a six-axis articulated robot can cover the entire sorting area, thus enabling it to accurately complete visual recognition and grasping tasks for intelligent fusion terminals.

[0084] The gripping component refers to the main gripping module used to grip the smart fusion terminal to the sorting area. It consists of flexible adaptive grippers that can adaptively adjust the pressure of the grippers on the surface of the smart fusion terminal according to the size, surface roughness, and other factors of the smart fusion terminal being gripped.

[0085] The auxiliary gripping component is a device used to adhere to the flat surface of the smart fusion terminal, making the gripping of the smart fusion terminal more stable during the gripping process. It consists of a square fixed frame with a side length equal to the longest side length of the smart fusion terminal, two slide rails installed on the diagonals of the square frame, and four suction cups that can move on the slide rails. After receiving the size adjustment data of the processed terminal, the four suction cups move towards the center of the square fixed frame on the slide rails according to the size adjustment data, thereby adhering to the top surface of the smart fusion terminal.

[0086] A smart converged terminal is a device based on the Internet of Things and edge computing technologies. It is mainly used for the lean management of low-voltage power distribution networks. The device is rectangular in shape, with a metal casing and heat dissipation holes, cable interfaces and indicator lights on its surface.

[0087] Step S101: Determine whether the initial test result is consistent with the preset qualified test result.

[0088] In this step, the qualified test result is consistent with the qualified test result in step S100 above. After the processing terminal determines the initial test result, the processing terminal judges whether the initial test result is consistent with the qualified test result, thereby determining whether the appearance of the intelligent fusion terminal is qualified.

[0089] Step S1011: If they match, the qualified test result is determined as the preset final test result, and the appearance sorting device continues to be controlled to inspect the appearance of the next intelligent fusion terminal to generate an initial test result.

[0090] If the processing terminal determines that the initial detection result is consistent with the qualified detection result, it means that the appearance of the intelligent fusion terminal is qualified. Therefore, the qualified detection result is determined as the final detection result by the processing terminal, and the appearance sorting device is controlled by the processing terminal to detect the appearance of the next intelligent fusion terminal to generate the initial detection result, thereby providing data support for the sorting of the next intelligent fusion terminal.

[0091] The final test result refers to the signal used by the processing terminal to determine the sorting area of ​​the intelligent fusion terminal, including qualified test results and unqualified test results.

[0092] Unqualified test results refer to the test results corresponding to the appearance test results of the intelligent fusion terminal being a single ambiguous defect test result or other mixed defect test results.

[0093] The qualified test results, single fuzzy defect test results, and other mixed defect test results in this step are consistent with the qualified test results, single fuzzy defect test results, and other mixed defect test results in step S100 above, and will not be repeated here.

[0094] Step S1012: If there is a discrepancy, control the preset reflection suppression component to assist the appearance sorting device in performing a secondary inspection of the appearance of the smart fusion terminal to generate the final inspection result.

[0095] If the processing terminal determines that the initial test result is inconsistent with the qualified test result, it indicates that the appearance of the intelligent fusion terminal may be unqualified. Therefore, it is necessary to control the reflection suppression component through the processing terminal to assist the appearance sorting device in performing a secondary inspection of the appearance of the intelligent fusion terminal to eliminate the influence of reflection on the test result. The specific method is as follows: Figure 2 The process involves several steps to generate the final detection result, which in turn provides data support for controlling the grasping component and auxiliary grasping component to sort the intelligent fusion terminal.

[0096] A reflection suppression component is a part used to reduce or eliminate reflections on the surface of a smart fusion terminal and prevent reflections from affecting the detection results. It includes a light shield and a polarizer. The light shield is a fixed-size part used to physically block the reflective areas of the smart fusion terminal to reduce the direct light hitting the surface of the smart fusion terminal. It consists of a rotating shaft, a slide rail and a light shield body. After the reflective area of ​​the smart fusion terminal is determined, the light shield adjusts its angle and position according to the location of the area, thereby achieving the function of reducing reflections.

[0097] A polarizer is a component used to filter light sources. It consists of a polarizer installed at the light source to filter the light source, and an analyzer installed at the front of the lens of a 2D high-resolution color camera in a multimodal image acquisition assembly to filter the polarized reflections that may still remain after being covered by the polarizer.

[0098] Step S102: Collect top surface data from the intelligent fusion terminal.

[0099] The top surface data refers to the data set storing the top surface roughness, surface area, top surface length, and top surface width of the intelligent fusion terminal. The top surface of the intelligent fusion terminal is detected by the multimodal image acquisition component in the appearance sorting device to obtain the top surface roughness, surface area, top surface length, and top surface width. The detected data is then transmitted to the processing terminal, which summarizes the data to obtain the top surface data. The top surface data provides data support for the subsequent control auxiliary grasping component to adsorb the intelligent fusion terminal.

[0100] Step S103: Control the appearance sorting device to sort the intelligent fusion terminal based on the top surface data and the final inspection results.

[0101] After the processing terminal determines the top surface data and the final inspection result, it controls the appearance sorting device to sort the intelligent fusion terminal based on the top surface data and the final inspection result. The specific method is as follows: Figure 5 These steps improve the accuracy of sorting smart fusion terminals and reduce the chance of damage to them during the sorting process.

[0102] Reference Figure 2 The steps for controlling the preset reflective suppression components to assist the appearance sorting device in performing secondary inspections on the appearance of the intelligent fusion terminal to generate the final inspection result include:

[0103] Step S200: Determine whether the detection result is consistent with the preset single fuzzy defect detection result.

[0104] In this step, the detection result of a single fuzzy defect is consistent with the detection result of a single fuzzy defect in step S100 above. The processing terminal determines whether the detection result is consistent with the detection result of a single fuzzy defect, thereby determining whether it is necessary to eliminate the influence of reflection on the detection result through the reflection suppression component.

[0105] Step S2001: If there is a discrepancy, the preset non-conforming test result shall be determined as the final test result.

[0106] If the processing terminal determines that the detection result is inconsistent with the single fuzzy defect detection result, it means that it is not necessary to eliminate the influence of reflection on the detection result through the reflection suppression component. Therefore, the processing terminal determines the unqualified detection result as the final detection result, thereby providing data support for the subsequent processing terminal to control the appearance sorting device to sort the intelligent fusion terminal.

[0107] The non-conforming test results in this step are consistent with the non-conforming test results in step S1011 above, and will not be repeated here.

[0108] Step S2002: If they match, obtain the rotation angle of the light-shielding plate, the target center position of the light-shielding plate, and the rotation angle of the polarizer.

[0109] If the processing terminal determines that the detection result is consistent with the single fuzzy defect detection result, it indicates that the influence of reflection on the detection result needs to be eliminated by using a reflection suppression component. Therefore, the rotation angle of the light-shielding plate, the target center position of the light-shielding plate, and the rotation angle of the polarizer are obtained. The specific method is described in [reference needed]. Figure 3 This process provides data support for subsequent adjustments to the reflectivity suppression components to eliminate reflectivity effects.

[0110] The light-shielding plate rotation angle refers to the angle by which the light-shielding plate needs to be rotated to cover the blurred area of ​​the image. The target center position of the light-shielding plate refers to the position that the center of the light-shielding plate needs to be moved to in order to cover the blurred area of ​​the image. The polarizer rotation angle refers to the data set of the polarizer rotation angle and the analyzer rotation angle. The polarizer rotation angle refers to the angle by which the polarizer needs to be rotated, and the analyzer rotation angle refers to the angle by which the analyzer needs to be rotated. For specific methods of obtaining these angles, please refer to [link / reference]. Figure 3 The steps.

[0111] Step S20021: Control the movement and deflection angle of the light-shielding plate according to the rotation angle of the light-shielding plate and the target center position of the light-shielding plate, and control the deflection of the polarizer according to the rotation angle of the polarizer.

[0112] In this process, after the processing terminal determines the rotation angle of the light-shielding plate, the target center position of the light-shielding plate, and the polarization rotation angle, the processing terminal controls the light-shielding plate to move and deflect according to the rotation angle of the light-shielding plate and the target center position of the light-shielding plate. At the same time, the processing terminal controls the polarizer to rotate according to the polarizer rotation angle in the polarizer rotation angle, and controls the analyzer to rotate according to the analyzer rotation angle.

[0113] Step S20022: Control the appearance sorting device to inspect the appearance of the intelligent fusion terminal to generate a suppressed inspection result.

[0114] The post-suppression detection result refers to the appearance inspection conclusion of the intelligent fusion terminal after the processing terminal controls the reflection suppression component to suppress the reflection, and the data is processed by the multimodal image acquisition component in the appearance sorting device. The method for obtaining the post-suppression detection result in this step is the same as the method for obtaining the initial detection result in step S100 above, and will not be described in detail here.

[0115] Step S20023: Determine the final detection result based on the detection results after suppression.

[0116] After the processing terminal determines the detection result after suppression, it determines the final detection result based on the suppression detection result. The specific method is described in [reference needed]. Figure 4 The steps in this process provide data support for the subsequent processing terminal to control the appearance sorting device to sort the intelligent fusion terminal.

[0117] Reference Figure 3 The steps for obtaining the rotation angle of the light-shielding plate, the target center position of the light-shielding plate, and the rotation angle of the polarizer include:

[0118] Step S300: Collect the appearance image data of the intelligent fusion terminal, and perform grayscale and edge contour quantization calculation on the appearance image data through a preset image feature extraction algorithm, and compare the calculation results with the preset normal calculation data to determine the diagonal position of the blurred area.

[0119] Among them, appearance image data refers to the collection of image data of each surface of the intelligent fusion terminal. The multimodal image acquisition component in the appearance sorting device acquires images of each surface of the intelligent fusion terminal, and then the acquired image data is summarized by the processing terminal to obtain the appearance image data.

[0120] The diagonal position of the fuzzy region refers to the set of coordinates of the lower left and upper right corner vertices of the smallest bounding rectangle that completely covers the fuzzy region on the surface of the intelligent fusion terminal. The processing terminal inputs the appearance image data into the image feature extraction algorithm, performs grayscale and edge contour quantization calculations on the appearance image data, and then compares the calculation results with normal calculation data to identify different areas as fuzzy regions. The fuzzy regions are then input into the target detection model, which directly outputs the coordinates of the lower left and upper right corner vertices of the smallest bounding rectangle. Finally, the processing terminal summarizes the two coordinates to obtain the diagonal position of the fuzzy region.

[0121] Image feature extraction algorithms are algorithms that can automatically extract information from appearance image data to represent image features.

[0122] Normal calculation data refers to the set of grayscale value quantization data and edge contour quantization data when the appearance image of the intelligent fusion terminal is clear in the same environment. Normal calculation data is obtained by collecting appearance image data of qualified intelligent fusion terminals in the same environment, performing grayscale and edge contour quantization calculation on the appearance image data through image feature extraction algorithm, and then summarizing the quantization calculation results through the processing terminal.

[0123] Step S301: Calculate the diagonal position of the blurred region based on the arctangent function to generate the rotation angle of the light-shielding plate.

[0124] In this step, the rotation angle of the light-shielding plate is the same as that in step S2002 above. After the processing terminal determines the diagonal position of the blurred area, the processing terminal uses the arctangent function angle conversion formula. The rotation angle of the light-shielding plate can be obtained by calculation, where, This refers to the rotation angle of the light-shielding plate. It refers to pi, in For example, =3.14, It refers to the x-coordinate of the lower left vertex of the fuzzy region. The ordinate of the lower left vertex of the blurred region. It refers to the x-coordinate of the top-right vertex of the fuzzy region. The formula refers to the ordinate of the top right corner vertex of the blurred area. By substituting the diagonal position of the blurred area into the arctangent function angle transformation formula, the rotation angle of the light shield is obtained. This angle is the angle at which the light shield needs to be tilted to maximize the coverage of the blurred area.

[0125] Step S302: Calculate the diagonal position of the fuzzy region according to the preset midpoint coordinate formula to generate the center position of the fuzzy region.

[0126] In this step, the center position of the fuzzy region is the same as that in step S2002 above. After the processing terminal determines the diagonal position of the fuzzy region, the processing terminal uses the midpoint coordinate formula... The center position of the fuzzy region can be obtained by calculation, where, It refers to the center location of the fuzzy region. This refers to the x-coordinate of the center position of the fuzzy region. This refers to the ordinate of the center position of the fuzzy region. It refers to the x-coordinate of the lower left vertex of the fuzzy region. The ordinate of the lower left vertex of the blurred region. It refers to the x-coordinate of the top-right vertex of the fuzzy region. The formula uses the ordinate of the top right corner vertex of the blurred area to calculate the coordinates of the center of the blurred area. This provides data support for determining the target center position of the light-shielding plate based on these coordinates, ensuring that the width and height of the light-shielding plate can cover the blurred area.

[0127] Step S303: Subtract the x-coordinate and y-coordinate of the diagonal position of the blurred region to generate the width and height of the blurred region.

[0128] The width of the fuzzy region refers to the length of the smallest bounding rectangle that completely covers the fuzzy region on the surface of the intelligent fusion terminal. The width of the fuzzy region can be obtained by subtracting the x-coordinate of the bottom left corner from the x-coordinate of the top right corner of the fuzzy region from the x-coordinate of the bottom left corner of the fuzzy region.

[0129] The height of the fuzzy region refers to the length of the smallest bounding rectangle that completely covers the fuzzy region on the surface of the intelligent fusion terminal. The width of the fuzzy region can be obtained by subtracting the ordinate of the bottom left corner from the ordinate of the top right corner of the fuzzy region from the ordinate of the bottom left corner of the fuzzy region.

[0130] Step S304: Calculate the target center position of the light shield based on the rotation angle of the light shield, the center position of the blurred area, the width of the blurred area, the height of the blurred area, the preset width of the light shield, and the preset height of the light shield.

[0131] Here, the target center position of the light-shielding plate refers to the coordinates of the position that the center of the light-shielding plate needs to be moved to, which is determined by the processing terminal according to the formula. The target center position of the light-shielding plate can be obtained by calculation, where, This refers to the x-coordinate of the center position of the target on the light-shielding plate. This refers to the ordinate of the center position of the target on the light-shielding plate. It refers to the center location of the fuzzy region. This refers to the rotation angle of the light-shielding plate. This refers to the width of the light-shielding plate. This refers to the width of the fuzzy region. This refers to the height of the sunshade. This refers to the height of the blurred area. The formula first calculates the width and height of one side of the light-shielding plate that extend beyond the blurred area. Then, it calculates the projected width of the one side in the horizontal direction and the projected height in the vertical direction. Finally, the horizontal coordinate of the target position that the light-shielding plate needs to move to so that the horizontal edge of the light-shielding plate can cover the horizontal edge of the blurred area is obtained by subtracting the projected height from the horizontal coordinate of the center position of the blurred area. The vertical coordinate of the target position that the light-shielding plate needs to move to so that the vertical edge of the light-shielding plate can cover the horizontal edge of the blurred area is obtained by subtracting the projected height from the vertical coordinate of the center position of the blurred area.

[0132] The width of the light-shielding plate refers to the length corresponding to the length of the light-shielding plate. In one embodiment, the width of the light-shielding plate can be obtained by the operator measuring the length of the light-shielding plate using a length measuring tool.

[0133] The height of the light-shielding plate refers to the length corresponding to the width of the light-shielding plate. In one embodiment, the height of the light-shielding plate can be obtained by the operator measuring the width of the light-shielding plate using a length measuring tool.

[0134] Step S305: Collect the direction of surface light wave vibration.

[0135] The surface light wave vibration direction refers to the light wave vibration direction of the light reflected from the surface of the intelligent fusion terminal. In one embodiment, four images with different polarization angles are obtained by taking pictures of the reflective area of ​​the intelligent fusion terminal with a polarization camera. Then, the polarization direction of each pixel in the four images is calculated by software. The average value of the calculation results is then obtained by the processing terminal to obtain the surface light wave vibration direction. When the rotation angle of the polarizer is perpendicular to the surface light wave vibration direction, more than 90% of the reflection can be filtered out. Therefore, by collecting the surface light wave vibration direction and adding the perpendicular angle, the polarizer rotation angle that can filter out the reflection to the greatest extent can be obtained.

[0136] Step S306: Add the surface light wave vibration direction to the preset vertical angle to generate the polarizer rotation angle.

[0137] In this step, the polarizer rotation angle is the same as that in step S1002 above. After the processing terminal determines the vibration direction of the surface light wave, the polarizer rotation angle is obtained by adding the vibration direction of the surface light wave to the vertical angle. Then, the polarizer rotation angle is obtained by adding the influence angle of the light shield to the polarizer rotation angle. Finally, the polarizer rotation angle and the polarizer rotation angle are summed by the processing terminal to obtain the polarizer rotation angle.

[0138] The vertical angle refers to the angle between the direction of light wave vibration on the surface and the rotation angle of the polarizer; the vertical angle is 90 degrees. 。 The angle of influence of the light-shielding plate refers to the offset angle caused by the influence of the light-shielding plate on the polarization direction of the reflected light. In one embodiment, the angle of influence of the light-shielding plate is 5°. 。 .

[0139] Reference Figure 4 The steps for determining the final detection result based on the post-suppression detection results include:

[0140] Step S400: Determine whether the detection result after suppression is consistent with the qualified detection result.

[0141] In this process, after the processing terminal determines the suppression detection result, it judges whether the suppression detection result is consistent with the qualified detection result, thereby determining whether the appearance of the intelligent fusion terminal is qualified.

[0142] Step S4001: If they match, the qualified test result is determined as the final test result.

[0143] If the processing terminal determines that the detection result after suppression meets the requirements of a qualified detection result, it indicates that the appearance of the intelligent fusion terminal is qualified. Therefore, the qualified detection result is determined as the final detection result by the processing terminal, thereby providing data support for the subsequent processing terminal to control the appearance sorting device to sort the intelligent fusion terminal.

[0144] Step S4002: If there is a discrepancy, the non-conforming test result shall be determined as the final test result.

[0145] If the processing terminal determines that the detection result after suppression does not meet the requirements of the qualified detection result, it means that the appearance of the intelligent fusion terminal is unqualified. Therefore, the processing terminal determines the unqualified detection result as the final detection result, thereby providing data support for the subsequent processing terminal to control the appearance sorting device to sort the intelligent fusion terminal.

[0146] Reference Figure 5 The steps for controlling the appearance sorting device to sort the intelligent fusion terminal based on the top surface data and the final inspection results include:

[0147] Step S500: Determine whether the top surface data meets the preset top surface adsorption requirements.

[0148] Among them, the top surface adsorption requirement means that the top surface of the intelligent fusion terminal must be smooth and without any protrusions or depressions in order to be adsorbed by the auxiliary grasping component.

[0149] After the processing terminal determines the top surface data, it judges whether the top surface data meets the preset top surface adsorption requirements, thereby determining whether the top surface of the intelligent fusion terminal can be adsorbed by the auxiliary grasping component.

[0150] Step S5001: If not satisfied, the intelligent fusion terminal is flipped according to the top surface data control grabbing component to collect the top surface data after flipping, and the flipped top surface data is determined as the top surface data for cyclic judgment.

[0151] If the processing terminal determines that the top surface data does not meet the top surface adsorption requirements, it means that the top surface of the intelligent fusion terminal cannot be adsorbed by the auxiliary grasping component. Therefore, based on the top surface data, the grasping component is controlled to flip the intelligent fusion terminal to collect the top surface data after the flip. The processing terminal then identifies the flipped top surface data as the top surface data and performs a cyclical judgment. The specific method is described in [reference needed]. Figure 6 This process involves several steps, allowing the flipped surface and top surface to be attracted by the auxiliary gripping component, thereby improving the stability of the gripping component in the gripping process of the intelligent fusion terminal.

[0152] The flipped top surface data refers to the data set storing the top surface roughness, surface area, top surface length, and top surface width after the flipped intelligent fusion terminal is installed. For specific data collection methods, please refer to [link / reference needed]. Figure 6 This process provides data support for the processing terminal to further determine whether the top surface of the intelligent fusion terminal meets the top surface adsorption requirements.

[0153] The grabbing component in this step is the same as the grabbing component in step S100 above, and will not be described again here.

[0154] Step S5002: If satisfied, the optimal sorting path is calculated using a kinematic model and dynamic programming algorithm based on the preset qualified area location, the preset defective area location, and the final detection result.

[0155] If the processing terminal determines that the top surface data meets the top surface adsorption requirements, it means that the top surface of the intelligent fusion terminal can be adsorbed by the auxiliary grasping component. Therefore, by substituting the qualified area location, the defective area location, and the final detection result into the kinematic model and dynamic programming algorithm, the optimal sorting path can be obtained.

[0156] The qualified area refers to the coordinates of the area where smart converged terminals with acceptable appearance are placed. The qualified area is located on the left side of the end of the conveyor belt. The defective area refers to the coordinates of the area where smart converged terminals with unacceptable appearance are placed. The defective area is located on the right side of the end of the conveyor belt.

[0157] Step S50021: Control the auxiliary gripping component to adsorb the top surface of the intelligent fusion terminal according to the optimal sorting path, and control the gripping component to grip the intelligent fusion terminal.

[0158] In this process, after the processing terminal determines the optimal sorting path, it controls the auxiliary gripping component to adhere to the top surface of the intelligent fusion terminal based on the optimal sorting path, and controls the gripping component to grasp the intelligent fusion terminal. The specific method is described in [reference needed]. Figure 7 The process involves sorting the intelligent fusion terminals to the corresponding areas.

[0159] The auxiliary grasping component in this step is the same as the auxiliary grasping component in step S100 above, and will not be described again here.

[0160] Reference Figure 6 The steps for controlling the grasping component to flip the intelligent fusion terminal based on the top surface data to collect the top surface data after flipping include:

[0161] Step S600: Find the top surface number in the preset surface data number correspondence relationship based on the top surface data.

[0162] Among them, the surface data sequence number correspondence refers to the correspondence between the surface data of the intelligent fusion terminal and the surface sequence number. The operator forms a mapping table by matching the surface data with the surface sequence number of the corresponding surface data one by one.

[0163] The top surface number refers to the surface number corresponding to the top surface data, which is obtained by the processing terminal by looking up the top surface data in the mapping table corresponding to the surface data number correspondence.

[0164] Step S601: Find the smooth surface data and the corresponding smooth surface number in the surface data sequence number correspondence.

[0165] Among them, smooth surface data refers to the data set storing the top surface roughness, surface area, top surface length and top surface width of the smooth surface of one or more intelligent fusion terminals. Smooth surface serial number refers to the surface serial number corresponding to the smooth surface data. By processing the terminal, matching and searching in the surface data serial number correspondence according to the top surface adsorption requirements, the corresponding matching data is determined as smooth surface data, and the surface serial number corresponding to the matching data is determined as smooth surface serial number.

[0166] Step S602: Find the smooth surface area in the smooth surface data.

[0167] Among them, smooth surface area refers to a data set that stores the surface area of ​​smooth surfaces of one or more intelligent fusion terminals. By processing the terminal, a single smooth surface area can be obtained by searching in the smooth surface data, and then the smooth surface area can be obtained by summing up the single smooth surface areas.

[0168] Step S603: Sort and select the smooth surface areas to determine the minimum smooth surface number corresponding to the minimum smooth surface area.

[0169] Among them, the minimum smooth surface area refers to the surface area with the smallest numerical value among the smooth surfaces of the intelligent fusion terminal, and the minimum smooth surface index refers to the surface index of the smooth surface corresponding to the minimum smooth surface area. The smooth surface areas are sorted by the processing terminal, and the smooth surface area with the smallest numerical value is determined as the minimum smooth surface area. Then, the processing terminal performs matching and retrieval based on the minimum smooth surface area, and the surface data corresponding to the surface area data that matches the minimum smooth surface area is determined as the minimum smooth surface data. Finally, the minimum smooth surface index can be obtained by searching in the mapping table corresponding to the surface data index correspondence.

[0170] Step S604: Collect the actual top surface center position, and collect the minimum smooth surface center position according to the minimum smooth surface sequence number.

[0171] The actual top center position refers to the coordinates of the top center of the current intelligent fusion terminal. The multimodal image acquisition component in the appearance sorting device collects the top width, vertical height, and top length of the current top surface of the intelligent fusion terminal from the ground. Then, the three data are divided by 2 to determine the position of the center of the intelligent fusion terminal. This position is used as the origin of the spatial rectangular coordinate system. The spatial rectangular coordinate system is mapped onto the intelligent fusion terminal. Then, the x-axis coordinate of the actual top center position is half of the top width, the x-axis coordinate is half of the top length, and the z-axis coordinate is half of the vertical height.

[0172] The minimum smooth surface center position refers to the position coordinates of the minimum smooth surface center. After the processing terminal determines the minimum smooth surface number, it uses the above-mentioned spatial rectangular coordinate system and the origin position as a reference to determine the orientation of the corresponding smooth surface according to the minimum smooth surface number. Then, the multimodal image acquisition component in the appearance sorting device determines half the length and half the width of the smooth surface, thereby determining the offset of the smooth surface from the origin. Finally, the minimum smooth surface center position is determined according to the orientation of the smooth surface and the offset from the origin.

[0173] Step S605: Calculate the difference between the coordinate values ​​on the corresponding coordinate axes of the actual top surface center position and the minimum smooth surface center position to generate a flip vector.

[0174] The flip vector is a physical quantity used to describe the flipping direction of the intelligent fusion terminal. It is obtained by subtracting the x-axis, y-axis, and z-axis coordinates of the actual top surface center position from the x-axis, y-axis, and z-axis coordinates of the center position of the minimum smooth surface. The flip vector can be used to determine the gripping direction of the gripping component, so that the top surface of the intelligent fusion terminal is a smooth surface, and thus the auxiliary gripping component can adsorb the top surface of the intelligent terminal.

[0175] Step S606: Substitute the actual top surface center position and the minimum smooth surface center position into the preset vector dot product formula to calculate the flip angle.

[0176] Here, the flip angle refers to a physical quantity used to describe the flip angle of the intelligent fusion terminal, which is processed by the terminal according to the vector dot product formula. The flip angle can be obtained by calculation, where, This refers to the flip angle. This refers to the actual top surface normal vector. It refers to the minimum normal vector of a smooth surface.

[0177] The actual top surface normal vector refers to the vector perpendicular to the top surface of the current intelligent fusion terminal, used to describe the orientation of the top surface. The bottom surface center position can be obtained by processing the terminal to invert the z-axis coordinate of the actual top surface center position based on the actual top surface center position and the origin position, while keeping the x-axis and y-axis coordinate values ​​unchanged. Then, the actual top surface normal vector can be obtained by subtracting the x-axis, y-axis, and z-axis coordinates of the actual top surface center position from the x-axis, y-axis, and z-axis coordinates of the bottom surface center position.

[0178] The minimum smooth surface normal vector is a vector perpendicular to the minimum smooth surface, used to describe the orientation of the minimum smooth surface. The processing terminal determines the coordinates of the center position of the parallel plane of the intelligent fusion terminal surface that is parallel to the minimum smooth surface based on the center position and origin position of the minimum smooth surface. Then, the processing terminal subtracts the corresponding coordinates of the center position of the parallel plane from the coordinates of the center position of the minimum smooth surface to obtain the minimum smooth surface normal vector.

[0179] Step S607: Control the grasping component to flip the intelligent fusion terminal according to the flip angle and flip vector, and collect the top surface data after flipping.

[0180] In this process, after the processing terminal determines the flip angle and flip vector, it uses the flip vector as the rotation axis and then determines the flip direction using the right-hand rule. The processing terminal then controls the grasping component to flip the intelligent fusion terminal based on the flip direction and flip angle, thereby providing support for the subsequent processing terminal to control the auxiliary grasping component to adsorb the intelligent fusion terminal and collect the top surface data after flipping.

[0181] The flipped top surface data in this step is consistent with the flipped top surface data in step S5001 above. The top surface of the intelligent fusion terminal is detected by the multimodal image acquisition component in the appearance sorting device to obtain the top surface roughness, surface area, top surface length and top surface width after flipping. The detected data is then transmitted to the processing terminal, and the data is summarized by the processing terminal to obtain the top surface data.

[0182] Reference Figure 7 The steps of controlling the auxiliary gripping component to adhere to the top surface of the intelligent fusion terminal according to the optimal sorting path, and controlling the gripping component to grip the intelligent fusion terminal, include:

[0183] Step S700: Find the top surface dimension data in the top surface data.

[0184] Among them, the top surface dimension data refers to the data set storing the top surface length and top surface width of the intelligent fusion terminal. The top surface length and top surface width can be obtained by searching in the top surface data by the processing terminal, and the top surface dimension data can be obtained by summing the top surface length and top surface width.

[0185] Step S701: Calculate the difference between the top surface dimension data and the preset auxiliary gripping component dimension data to generate the auxiliary gripping component dimension difference.

[0186] The auxiliary grasping component size difference refers to the data set of lengths of the two legs of the right triangle corresponding to the distance that the auxiliary grasping component needs to adjust to maximize its adsorption on the top surface of the smart fusion terminal. After the processing terminal determines the top surface size data, it subtracts the top surface length and top surface width from the top surface size data, respectively, from the maximum adsorption length and maximum adsorption width of the auxiliary grasping component. This gives the lengths of the two hypotenuses of the right triangle corresponding to the distance that the auxiliary grasping component needs to adjust. The processing terminal then summarizes the calculated data to obtain the auxiliary grasping component size difference.

[0187] The size of the auxiliary gripping component refers to the data set storing the maximum adsorption length and maximum adsorption width of the auxiliary gripping component. In one embodiment, the operator can obtain the maximum adsorption length and maximum adsorption width of the auxiliary gripping component by consulting the technical manual of the auxiliary gripping component. Then, the processing terminal summarizes the maximum adsorption length and maximum adsorption width to obtain the size data of the auxiliary gripping component.

[0188] Step S702: Calculate the hypotenuse lengths corresponding to the lengths of the two right-angled sides in the size difference of the auxiliary gripping component to generate the distance the auxiliary gripping component needs to move.

[0189] The required moving distance of the auxiliary gripping component refers to the distance that the auxiliary gripping component needs to be adjusted to maximize its adsorption on the top surface of the smart fusion terminal. By processing the terminal, the lengths of the two right-angled sides in the size difference of the auxiliary gripping component are calculated using the Pythagorean theorem to obtain the length of the hypotenuse of the right triangle. This hypotenuse length is the required moving distance of the auxiliary gripping component. By calculating the required moving distance of the auxiliary gripping component, the suction cup of the auxiliary gripping component can be moved on the slide rail, so that the suction cup can adsorb the top surface of the smart fusion terminal to the maximum extent.

[0190] Step S703: Control the movement of the auxiliary grasping component according to the required moving distance of the auxiliary grasping component, so as to enhance the stability of the auxiliary grasping component adsorbing onto the top surface of the smart fusion terminal.

[0191] In this process, after the processing terminal determines the distance that the auxiliary grasping component needs to move, the processing terminal controls the movement of the auxiliary grasping component according to the distance that the auxiliary grasping component needs to move, thereby enhancing the stability of the auxiliary grasping component adsorbing onto the top surface of the intelligent fusion terminal.

[0192] Based on the same inventive concept, embodiments of this application provide a sorting method for an intelligent fusion terminal based on visual recognition, including:

[0193] The acquisition module is used to acquire top surface data, appearance image data, surface light wave vibration direction, top surface data after flipping, actual top surface center position, and minimum smooth surface center position;

[0194] A memory for storing the program of a visual recognition-based intelligent fusion terminal sorting method;

[0195] The processor and memory can load and execute programs to implement a visual recognition-based intelligent fusion terminal sorting method.

[0196] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional modules is used as an example. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. The specific working process of the system, device, and unit described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0197] This application provides a computer-readable storage medium storing a computer program that can be loaded by a processor and executed as a sorting method for an intelligent fusion terminal based on visual recognition.

[0198] Computer storage media include, for example, USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, optical disks, and other media that can store program code.

[0199] Based on the same inventive concept, embodiments of this application provide a smart terminal, including a memory and a processor, wherein the memory stores a computer program that can be loaded by the processor and executed as a visual recognition-based smart fusion terminal sorting method.

[0200] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional modules is used as an example. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. The specific working process of the system, device, and unit described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0201] The above are all preferred embodiments of this application and are not intended to limit the scope of protection of this application. Any feature disclosed in this specification (including the abstract and drawings) may be replaced by other equivalent or similar features unless specifically stated otherwise. That is, unless specifically stated otherwise, each feature is only one example of a series of equivalent or similar features.

Claims

1. A sorting method for an intelligent fusion terminal based on visual recognition, characterized in that, include: The preset appearance sorting device is controlled to inspect the appearance of the preset intelligent fusion terminal to generate initial inspection results; Determine whether the initial test result is consistent with the preset qualified test result; If they match, the qualified test result is determined as the preset final test result, and the appearance sorting device continues to be controlled to inspect the appearance of the next intelligent fusion terminal to generate the initial test result. If there is a discrepancy, the preset reflective suppression component will be controlled to assist the appearance sorting device in performing a secondary inspection of the appearance of the smart fusion terminal in order to generate the final inspection result. Collect top-surface data from the intelligent converged terminal; Based on the top surface data and the final inspection results, the appearance sorting device is controlled to sort the intelligent fusion terminals; The reflection suppression component includes a light-shielding plate and a polarizer. The steps of controlling the preset reflection suppression component to assist the appearance sorting device in performing secondary inspection of the appearance of the intelligent fusion terminal to generate the final inspection result include: Determine whether the detection result is consistent with the preset single fuzzy defect detection result; If there is a discrepancy, the preset non-conforming test result will be determined as the final test result; If they match, obtain the rotation angle of the light-shielding plate, the target center position of the light-shielding plate, and the rotation angle of the polarizer; The movement and deflection angle of the light-shielding plate are controlled according to the rotation angle of the light-shielding plate and the target center position of the light-shielding plate, and the deflection of the polarizer is controlled according to the rotation angle of the polarizer. The appearance sorting device is controlled to inspect the appearance of the intelligent fusion terminal in order to generate a suppressed inspection result; The final detection result is determined based on the detection results after inhibition. The steps to obtain the rotation angle of the light-shielding plate, the target center position of the light-shielding plate, and the rotation angle of the polarizer include: Collect the appearance image data of the intelligent fusion terminal, and perform grayscale and edge contour quantization calculation on the appearance image data through a preset image feature extraction algorithm. Then compare the calculation results with the preset normal calculation data to determine the diagonal position of the blurred area. The diagonal position of the blurred region is calculated based on the arctangent function to generate the rotation angle of the light-shielding plate; The diagonal position of the fuzzy region is calculated based on the preset midpoint coordinate formula to generate the center position of the fuzzy region. Subtract the x-coordinate and y-coordinate of the diagonal position of the blurred region to generate the width and height of the blurred region. The target center position of the light shield is generated by calculating based on the rotation angle of the light shield, the center position of the blurred area, the width of the blurred area, the height of the blurred area, the preset width of the light shield, and the preset height of the light shield. The direction of surface light wave vibration is collected; The polarizer rotation angle is generated by adding the surface light wave vibration direction to a preset vertical angle.

2. The intelligent fusion terminal sorting method based on visual recognition according to claim 1, characterized in that, The steps for determining the final detection result based on the post-suppression detection results include: Determine whether the test results after inhibition are consistent with the qualified test results; If they match, the qualified test result will be determined as the final test result; If there is a discrepancy, the non-compliant test result will be determined as the final test result.

3. The intelligent fusion terminal sorting method based on visual recognition according to claim 1, characterized in that, The appearance sorting device includes a gripping component and an auxiliary gripping component. The steps for controlling the appearance sorting device to sort the intelligent fusion terminal based on the top surface data and the final detection result include: Determine whether the top surface data meets the preset top surface adsorption requirements; If the conditions are not met, the intelligent fusion terminal will be flipped by the grasping component based on the top surface data in order to collect the top surface data after flipping, and the top surface data after flipping will be determined as the top surface data for cyclic judgment. If the conditions are met, the optimal sorting path is calculated using kinematic models and dynamic programming algorithms based on the preset qualified area location, the preset defective area location, and the final inspection results. The optimal sorting path is used to control the auxiliary gripping component to adhere to the top surface of the intelligent fusion terminal, and to control the gripping component to grip the intelligent fusion terminal.

4. The intelligent fusion terminal sorting method based on visual recognition according to claim 3, characterized in that, The steps for controlling the grasping component to flip the intelligent fusion terminal based on the top surface data to collect the top surface data after flipping include: The top surface number is found in the preset surface data number correspondence relationship based on the top surface data; Find the smooth surface data and the corresponding smooth surface number in the surface data sequence number correspondence; Find the smooth surface area in the smooth surface data; The smooth surface areas are sorted and selected to determine the minimum smooth surface number corresponding to the minimum smooth surface area. Collect the actual center position of the top surface, and collect the center position of the minimum smooth surface according to the minimum smooth surface index; Calculate the difference between the coordinate values ​​on the corresponding axes of the actual top surface center position and the minimum smooth surface center position to generate a flip vector; The actual top surface center position and the minimum smooth surface center position are substituted into the preset vector dot product formula to calculate the flip angle. The gripping component is controlled by the flip angle and flip vector to flip the intelligent fusion terminal and collect the top surface data after flipping.

5. A sorting method for an intelligent fusion terminal based on visual recognition according to claim 3, characterized in that, The steps of controlling the auxiliary gripping component to adhere to the top surface of the intelligent fusion terminal according to the optimal sorting path, and controlling the gripping component to grip the intelligent fusion terminal include: The top surface dimension data is obtained by searching within the top surface data. Calculate the difference between the top surface dimension data and the preset auxiliary gripping component dimension data to generate the auxiliary gripping component dimension difference; Calculate the hypotenuse lengths corresponding to the lengths of the two right-angled sides in the size difference of the auxiliary gripping component to generate the distance the auxiliary gripping component needs to move; The movement of the auxiliary gripping component is controlled according to the required moving distance, so as to enhance the stability of the auxiliary gripping component's adsorption on the top surface of the smart fusion terminal.

6. A visual recognition-based intelligent fusion terminal sorting system, characterized in that, include: The acquisition module is used to acquire top surface data, appearance image data, and surface light wave vibration direction; A memory for storing the program of the intelligent fusion terminal sorting method based on visual recognition as described in any one of claims 1 to 5; The processor and the program in the memory can be loaded and executed by the processor to implement the intelligent fusion terminal sorting method based on vision recognition as described in any one of claims 1 to 5.

Images