A detection method, system, electronic device and storage medium for special-shaped plug-in machines

Abstract

This application discloses a method, system, electronic device, and storage medium for detecting a special-shaped plug-in machine. The method comprises: obtaining a module control flow of the special-shaped plug-in machine; detecting and processing the control connections of the special-shaped plug-in machine according to the module control flow to obtain an initial detection result; calibrating and debugging the special-shaped plug-in machine based on the initial detection result to obtain a calibration result; and based on the calibration result, delivering a component to be inserted into the special-shaped plug-in machine, performing component testing on the special-shaped plug-in machine through a human-computer interaction interface to obtain a target detection result. Embodiments of this application can improve plug-in accuracy and can be widely applied in the field of automation technology.

Description

Technical Field

[0001] The present application relates to the field of automation technology, and in particular to a detection method, system, electronic equipment and storage medium for a special-shaped plug-in machine. Background Art

[0002] In related technology, there are special-shaped insertion machines that process component insertion. These machines use high-precision robotic arms and fixtures, combined with visual recognition systems, to accurately identify and grasp components, then precisely insert them into designated locations. However, in actual applications, these machines are prone to problems such as offsets in the designated positions when inserting multiple components. Without proper inspection, this can lead to potential problems that could affect insertion accuracy.

[0003] In summary, the technical problems existing in the relevant technologies need to be improved. Summary of the Invention

[0004] The main purpose of the embodiments of the present application is to provide a detection method, system, electronic device and storage medium for a special-shaped plug-in machine, which can improve the accuracy of the plug-in.

[0005] To achieve the above objectives, one aspect of an embodiment of the present application provides a method for detecting a special-shaped plug-in machine, the method comprising:

[0006] Get the module control flow of the special-shaped plug-in machine;

[0007] Performing detection processing on the control connection of the special-shaped plug-in machine according to the module control process to obtain an initial detection result;

[0008] Calibration and debugging are performed on the special-shaped plug-in machine based on the initial detection result to obtain a calibration result;

[0009] Based on the calibration result, the component to be inserted is sent to the special-shaped insertion machine, and the special-shaped insertion machine is subjected to component testing through a human-computer interaction interface to obtain a target detection result.

[0010] In some embodiments, the method of obtaining the module control process of the special-shaped plug-in machine includes the following steps:

[0011] Analyze and process the hardware modules of the special-shaped plug-in machine to obtain a motion control module, an input / output control module, and an image acquisition module;

[0012] The control processes of the motion control module, the input / output control module and the image acquisition module are subjected to signal transmission relationship analysis and processing to obtain the module control process.

[0013] In some embodiments, the detecting and processing of the control connection of the special-shaped plug-in machine according to the module control process to obtain an initial detection result includes the following steps:

[0014] According to the module control process, a motion control module, an input and output control module and an image acquisition module are obtained;

[0015] The motion control module detects the sensor input signal and the motor input signal of the special-shaped plug-in machine, and detects the motion control signal output by the motion control module to obtain a signal detection result;

[0016] Performing image acquisition processing on the image acquisition device of the special-shaped plug-in machine by the image acquisition module to obtain an acquired image;

[0017] The input-output control module performs signal output processing on the special-shaped plug-in machine according to the signal detection result and the collected image to obtain the initial detection result.

[0018] In some embodiments, the calibration and debugging of the special-shaped plug-in machine based on the initial detection result to obtain the calibration result includes the following steps:

[0019] Acquire a captured image according to the initial detection result;

[0020] Performing feature point coordinate determination processing on the collected image to obtain physical coordinates and pixel coordinates;

[0021] Performing affine transformation and coordinate conversion processing on the physical coordinates and the pixel coordinates to obtain a calibrated rotation center;

[0022] The image acquisition device of the special-shaped plug-in machine is calibrated according to the calibrated rotation center to obtain the calibration result.

[0023] In some embodiments, the component to be inserted is sent into the special-shaped insertion machine based on the calibration result, and the component test processing is performed on the special-shaped insertion machine through a human-computer interaction interface to obtain a target detection result, including the following steps:

[0024] According to the calibration result, the image of the component to be inserted is collected and processed by the special-shaped insertion machine to obtain a target image;

[0025] Displaying the target image in the human-computer interaction interface;

[0026] generating a test machine model according to the target image in response to a first operation instruction on the human-computer interaction interface;

[0027] In response to a second operation instruction on the human-computer interaction interface, the component to be plug-in is debugged according to the test machine model to obtain the target detection result.

[0028] In some embodiments, in response to the first operation instruction on the human-computer interaction interface, generating a test model according to the target image includes the following steps:

[0029] In response to a first operation instruction on the human-computer interaction interface, entering a parameter modification interface to perform parameter modification processing to obtain machine model parameters;

[0030] Enter the component interface to adjust the material picking position of each axis to obtain the coordinate position;

[0031] Performing a marking point setting process according to the machine model parameters and the coordinate position to obtain a marking point;

[0032] The target image is moved according to the marking points to obtain the test model.

[0033] In some embodiments, in response to the second operation instruction on the human-computer interaction interface, debugging the component to be plugged in according to the test machine model to obtain the target detection result includes the following steps:

[0034] In response to a second operation instruction on the human-computer interaction interface, identifying and locating the center position of the component pin of the component to be inserted to obtain a positioning position;

[0035] The positioning position is subjected to position correction calculation processing according to the test machine model, and the motor is controlled to perform component plug-in processing at the correction position to obtain the target detection result.

[0036] In some embodiments, the identifying and locating the center position of the component pin of the component to be inserted to obtain the positioning position includes the following steps:

[0037] Performing image acquisition processing on the center position of the component pin of the plug-in component to obtain a component pin image;

[0038] Inputting the component pin image into a pre-trained image recognition model for recognition processing to obtain recognition coordinates;

[0039] The identified coordinates are subjected to coordinate conversion processing to obtain the positioning position.

[0040] To achieve the above-mentioned purpose, another aspect of the present application provides a detection system for a special-shaped plug-in machine, the system comprising:

[0041] The first module is used to obtain the module control flow of the special-shaped plug-in machine;

[0042] The second module is used to detect and process the control connection of the special-shaped plug-in machine according to the module control process to obtain an initial detection result;

[0043] The third module is used to calibrate and debug the special-shaped plug-in machine through a human-computer interaction interface based on the initial detection result to obtain a calibration result;

[0044] The fourth module is used to send the component to be inserted into the special-shaped insertion machine based on the calibration result, perform component testing on the special-shaped insertion machine through the human-computer interaction interface, and obtain the target detection result.

[0045] To achieve the above-mentioned purpose, another aspect of an embodiment of the present application provides an electronic device, which includes a memory and a processor, wherein the memory stores a computer program, and the processor implements the above-mentioned method when executing the computer program.

[0046] To achieve the above objectives, another aspect of an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method described above is implemented.

[0047] The embodiments of the present application include at least the following beneficial effects: The present application provides a detection method, system, electronic device, and storage medium for a special-shaped plug-in machine. The solution obtains the module control flow of the special-shaped plug-in machine and detects and processes the control connection of the special-shaped plug-in machine according to the module control flow to obtain an initial detection result. Based on the initial detection result, the special-shaped plug-in machine is calibrated and debugged to obtain a calibration result. Based on the calibration result, the component to be plugged in is sent into the special-shaped plug-in machine. The special-shaped plug-in machine is subjected to component testing through a human-computer interaction interface to obtain a target detection result. This makes it easier to test the control connection and function of the special-shaped plug-in machine. Moreover, since the present solution calibrates and debugs the special-shaped plug-in machine, it can alleviate the problem of deviation in the calibration position. By performing component testing on the special-shaped plug-in machine through a human-computer interaction interface, it can detect the position and control accuracy of the plug-in, which is conducive to improving the accuracy of the plug-in. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 This is a flow chart of a method for detecting a special-shaped plug-in machine provided in an embodiment of the present application;

[0049] Figure 2 This is a schematic diagram of the module control relationship of a special-shaped plug-in machine provided in an embodiment of the present application;

[0050] Figure 3 This is a structural diagram of a detection system for a special-shaped plug-in machine provided in an embodiment of the present application;

[0051] Figure 4 This is a schematic diagram of the hardware structure of the electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0052] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application is further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the embodiments of the present application. They are merely examples of systems and methods consistent with some aspects of the embodiments of the present application as detailed in the appended claims.

[0053] It will be understood that the terms "first", "second", etc. used in this application may be used herein to describe various concepts, but unless otherwise specified, these concepts are not limited by these terms. These terms are only used to distinguish one concept from another. For example, without departing from the scope of the embodiments of the present application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the words "if" and "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0054] The terms "at least one", "plurality", "each", "any", etc. used in this application include "at least one", "two" or more, "plurality" or "each", "any" or "any one", "each" or "any one" as used herein.

[0055] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this application pertains. The terms used herein are for the purpose of describing the embodiments of this application only and are not intended to limit this application.

[0056] Before explaining the embodiments of the present application in detail, some of the nouns and terms involved in the embodiments of the present application are first explained. The nouns and terms involved in the embodiments of the present application are subject to the following explanations.

[0057] A special-shaped insertion machine is a device specifically designed for automatically inserting unusually shaped electronic components. Unlike traditional insertion machines, this one can handle electronic components of irregular shapes and sizes, such as capacitors, inductors, and connectors. Using a high-precision robotic arm and visual recognition system, it achieves precise component insertion, significantly improving production efficiency and product quality.

[0058] Related technologies exist for inserting multiple or multiple components using special-shaped insertion machines. However, in actual applications, it has been found that differences between different components can lead to deviations in calibration positions, and related technologies do not test special-shaped insertion machines, making them prone to hardware or software issues, affecting insertion accuracy.

[0059] For example, if special-shaped insertion machines are not inspected, component installation accuracy issues can easily arise. Because special-shaped components are irregular in shape, lack of inspection can increase positioning errors, affecting product quality. Then there are production efficiency and cost issues: frequent downtime leads to decreased efficiency and increased maintenance costs. Another example is safety risks: mechanical failures can cause equipment damage or personal injury. For example, if a mechanical failure causes an emergency stop reset or a server failure, these safety hazards may not be addressed in a timely manner without inspection.

[0060] In view of this, embodiments of the present application provide a method, system, electronic device, and storage medium for detecting a special-shaped plug-in machine. This solution obtains an initial detection result by obtaining the module control flow of the special-shaped plug-in machine and performing detection processing on the control connection of the special-shaped plug-in machine according to the module control flow. Based on the initial detection result, the special-shaped plug-in machine is calibrated and debugged to obtain a calibration result. Based on the calibration result, the component to be plugged in is sent into the special-shaped plug-in machine. The special-shaped plug-in machine is subjected to component testing through a human-computer interaction interface to obtain a target detection result. This facilitates testing the control connection and function of the special-shaped plug-in machine. Moreover, since the present solution calibrates and debugs the special-shaped plug-in machine, it can alleviate the problem of deviation in the calibration position. Furthermore, by performing component testing on the special-shaped plug-in machine through a human-computer interaction interface, it can detect the position and control accuracy of the plug-in, which is conducive to improving the accuracy of the plug-in.

[0061] The embodiment of the present application provides a method for detecting a special-shaped plug-in machine, which relates to the field of automation technology. The embodiment of the present application provides a method for detecting a special-shaped plug-in machine, which can be applied to a terminal for detecting a special-shaped plug-in machine, or can be applied to a server, or can be software running in a terminal or a server. In some embodiments, the detection terminal can be a tablet computer, a laptop computer, a desktop computer, an industrial control terminal, etc., but is not limited to this; the server side can be configured as an independent physical server, or as a server cluster or distributed system composed of multiple physical servers, or as a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The server can also be a node server in a blockchain network; the software can be an application that implements the detection method for a special-shaped plug-in machine, etc., but is not limited to the above forms.

[0062] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, and the like. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, and the like that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments in which tasks are performed by remote processing devices connected via a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media, including storage devices.

[0063] Figure 1 This is an optional flow chart of a method for detecting a special-shaped plug-in machine provided in an embodiment of the present application. Figure 1 The method may include but is not limited to steps S101 to S104.

[0064] Step S101, obtaining the module control process of the special-shaped plug-in machine;

[0065] Step S102, detecting and processing the control connection of the special-shaped plug-in machine according to the module control process to obtain an initial detection result;

[0066] Step S103, calibrating and debugging the special-shaped plug-in machine based on the initial detection result to obtain a calibration result;

[0067] Step S104: sending the component to be inserted into the special-shaped insertion machine based on the calibration result, performing component testing on the special-shaped insertion machine through a human-computer interaction interface to obtain a target detection result.

[0068] In steps S101 to S104 of the embodiment of the present application, by acquiring the module control flow of the special-shaped plug-in machine, the control connection of the special-shaped plug-in machine can be detected and processed, thereby obtaining an initial detection result, which is used to test the control connection of the functional modules in the special-shaped plug-in machine. Among them, the module control flow in the embodiment of the present application refers to the control connection relationship between the various functional modules in the special-shaped plug-in machine. For example, the special-shaped plug-in machine includes a motion control module, an input / output control module, and an image acquisition module. The signal transmission relationship of the control flow of each module is analyzed to obtain the module control flow. For example, the motion control module performs motion control by receiving control instructions from the input / output control module and uploads the control results to the input / output control module. The module control flow is the motion control module receiving control instructions from the input / output control module and uploading the control results to the motion control module. Since the module composition and control process of each special-shaped plug-in machine are not exactly the same, the embodiment of the present application first obtains the module control flow. Through this module control flow, it is possible to detect which functional modules exist in the special-shaped plug-in machine and analyze the signal control transmission relationship of each functional module.

[0069] It should be noted that the embodiment of the present application can obtain the corresponding module control process based on the operating instructions of the special-shaped plug-in machine, for example, by identifying the hardware modules of the special-shaped plug-in machine through text recognition technology, and identifying and analyzing the signal transmission relationship of each hardware module, thereby obtaining the module control process. In addition, the embodiment of the present application can also analyze the software system programming file of the special-shaped plug-in machine, and combine programming language conversion and text recognition technology, etc. to analyze the control process of each module of the special-shaped plug-in machine in the programming file, thereby obtaining the module control process. In a feasible embodiment, the embodiment of the present application can analyze and process the hardware modules of the special-shaped plug-in machine according to the operating instructions to obtain a motion control module, an input / output control module and an image acquisition module. The motion control module can also include components such as a drive motor, and the image acquisition module includes units such as an upper camera and a lower camera. Then, the signal transmission relationship of the motion control module, the input / output control module and the image acquisition module is analyzed and processed, and the signal transmission relationship of each module can be identified through text recognition processing technology, thereby obtaining the module control process.

[0070] Then, based on the initial detection results, the special-shaped plug-in machine is calibrated and debugged to obtain the calibration results. By calibrating the camera of the special-shaped plug-in machine, calibration position errors are avoided. Finally, based on the calibration results, the components to be plugged in are sent to the special-shaped plug-in machine, and the components of the special-shaped plug-in machine are tested through the human-computer interaction interface to obtain the target detection results. By setting the test mode in the control interface, the pre-trained deep learning model can be used to perform real-time image analysis of the components after plug-in to identify the alignment status of the silver corners and the holes. If an offset is detected, such as if the silver corner is not fully inserted, an alarm is triggered and the offset is recorded, and the coordinates are dynamically compensated through the motion controller. The test results, such as the insertion success rate and positioning error distribution, are summarized in the database, and combined with the historical calibration parameters to generate optimization suggestions, such as adjusting the visual sampling frequency or the acceleration of the robot arm, and the iterative effect is visualized through the interface to obtain the target detection results.

[0071] One of the above technical solutions has the following advantages or beneficial effects: the embodiment of the present application can alleviate the problem of deviation in the calibration position by calibrating and debugging the special-shaped plug-in machine, and can test the components of the special-shaped plug-in machine through the human-computer interaction interface, so as to detect the position and control accuracy of the plug-in, which is conducive to improving the accuracy of the plug-in.

[0072] In some embodiments, the method of obtaining the module control process of the special-shaped plug-in machine includes the following steps:

[0073] Analyze and process the hardware modules of the special-shaped plug-in machine to obtain a motion control module, an input / output control module, and an image acquisition module;

[0074] The control processes of the motion control module, the input / output control module and the image acquisition module are subjected to signal transmission relationship analysis and processing to obtain the module control process.

[0075] In the examples of this application, please refer to Figure 2The special-shaped insertion machine utilizes industrial control. Its primary hardware consists of a motion control module, an input / output control module, and an image acquisition module. The machine's software system utilizes high-level C++, with multi-threaded, multi-module collaboration to achieve precise insertion. The motion control, input / output, and image acquisition modules analyze and process signal transmission relationships within the external and internal control processes. For example, the image acquisition module connects to an image acquisition device to capture images. This device can be a camera, consisting of an upper and lower camera working together. The upper camera locates the PCB's mark template to correct for deviations caused by PCB board movement. The lower camera, which can work in conjunction with multiple cameras depending on the number of insertion heads, accurately identifies and locates the component pins or outer edges to be attracted or gripped. The motion control module controls the servo motors and drivers, receiving commands from the upper-level controller and driving them to execute them, while also providing operational status feedback to the upper-level controller. The input / output control module processes and calculates the input signals fed to the control center, enabling real-time control of various actuators.

[0076] In some embodiments, the detecting and processing of the control connection of the special-shaped plug-in machine according to the module control process to obtain an initial detection result includes the following steps:

[0077] According to the module control process, a motion control module, an input and output control module and an image acquisition module are obtained;

[0078] The motion control module detects the sensor input signal and the motor input signal of the special-shaped plug-in machine, and detects the motion control signal output by the motion control module to obtain a signal detection result;

[0079] Performing image acquisition processing on the image acquisition device of the special-shaped plug-in machine by the image acquisition module to obtain an acquired image;

[0080] The input-output control module performs signal output processing on the special-shaped plug-in machine according to the signal detection result and the collected image to obtain the initial detection result.

[0081] In an embodiment of the present application, the motion control module detects the sensor input signals and motor input signals of the special-shaped plug-in machine. The sensor input signals can include input signals from various switches and sensors. The click input signal is the input signal of the motor driver. The motor driver can also be connected to a servo motor and an encoder to obtain the encoder input and output the signal to the servo motor. The image acquisition module performs image acquisition processing on the image acquisition device of the special-shaped plug-in machine to obtain the captured image. It should be noted that the special-shaped plug-in machine in the embodiment of the present application can capture different numbers of plug-in heads and lower cameras according to different plug-in requirements, can teach different material collection positions according to different feeder positions, and can teach different plug-in positions according to different plug-in requirements. That is, the device can be configured with different models to meet various incoming materials and plug-in requirements. The input and output control module inputs the signal detection results and the captured image into the control center of the special-shaped plug-in machine. The control center uses a cyclic scanning working mode to perform comprehensive calculations and processing on each input signal, realize real-time control of various actuators and perform signal output processing. By detecting various input signals and output signals, an initial detection result can be obtained.

[0082] In some embodiments, the calibration and debugging of the special-shaped plug-in machine based on the initial detection result to obtain the calibration result includes the following steps:

[0083] Acquire a captured image according to the initial detection result;

[0084] Performing feature point coordinate determination processing on the collected image to obtain physical coordinates and pixel coordinates;

[0085] Performing affine transformation and coordinate conversion processing on the physical coordinates and the pixel coordinates to obtain a calibrated rotation center;

[0086] The image acquisition device of the special-shaped plug-in machine is calibrated according to the calibrated rotation center to obtain the calibration result.

[0087] In an embodiment of the present application, an image captured by an image acquisition device can be obtained based on the initial detection results. Feature point coordinates are determined in the captured image. Specifically, multiple feature points are selected and their coordinates at their actual locations are recorded to obtain physical coordinates, as well as their coordinates within the image, resulting in pixel coordinates. This embodiment of the present application captures nine feature points with precisely known positions in space and obtains their pixel coordinates within the image. Based on these known physical coordinates and the corresponding image pixel coordinates, a mathematical model can be established to solve for the camera's intrinsic parameters (such as focal length, principal point coordinates, distortion coefficients, etc.) and extrinsic parameters (rotation and translation of the camera relative to the world coordinate system). Substituting these points into the affine transformation equation yields a system of linear equations. Solving this system of linear equations allows the affine transformation matrix to be calculated using the least squares method. The coordinates of the coordinate points are then transformed to obtain the transformed points. The coordinate position of the rotation center is calculated using three points, which is then used to calibrate the rotation center. By calibrating the rotation center, the image acquisition device of the special-shaped plug-in machine can be calibrated to obtain a calibration result.

[0088] In some embodiments, the component to be inserted is sent into the special-shaped insertion machine based on the calibration result, and the component test processing is performed on the special-shaped insertion machine through a human-computer interaction interface to obtain a target detection result, including the following steps:

[0089] According to the calibration result, the image of the component to be inserted is collected and processed by the special-shaped insertion machine to obtain a target image;

[0090] Displaying the target image in the human-computer interaction interface;

[0091] generating a test machine model according to the target image in response to a first operation instruction on the human-computer interaction interface;

[0092] In response to a second operation instruction on the human-computer interaction interface, the component to be plug-in is debugged according to the test machine model to obtain the target detection result.

[0093] In an embodiment of the present application, the camera of the special-shaped plug-in machine can be calibrated according to the calibration results, so that the image acquisition and processing of the plug-in component can be performed to obtain the target image. The embodiment of the present application displays the target image in the human-computer interaction interface, and the human-computer interaction interface includes an image display area, and the plug-in component can be monitored in real time and accurately positioned through the image display area. The human-computer interaction interface also includes a component bar, which is used to switch different operation interfaces. In response to the first operation instruction of the human-computer interaction interface, the first operation instruction can be operated by clicking the mouse. By clicking the model component with the mouse, the model interface can be entered, and the test model can be generated according to the operation of the target image. At the same time, in response to the second operation instruction of the human-computer interaction interface, the plug-in component is debugged and processed according to the test model to obtain the target detection result.

[0094] In some embodiments, in response to the first operation instruction on the human-computer interaction interface, generating a test model according to the target image includes the following steps:

[0095] In response to a first operation instruction on the human-computer interaction interface, entering a parameter modification interface to perform parameter modification processing to obtain machine model parameters;

[0096] Enter the component interface to adjust the material picking position of each axis to obtain the coordinate position;

[0097] Performing a marking point setting process according to the machine model parameters and the coordinate position to obtain a marking point;

[0098] The target image is moved according to the marking points to obtain the test model.

[0099] In this embodiment of the present application, in response to a first operation command on the human-computer interaction interface, the parameter modification interface can be entered to modify parameters. Specifically, in the PCB interface, PCB-related parameters such as track width and thickness can be modified by clicking Adjust Width. Clicking Next to enter the Feeder Parameters interface, set the corresponding feeder-related parameters. Clicking Next to enter the Components interface, adjust the material pickup position of each axis. Pressing the Tab key to enter the Axis Movement window and adjust axis coordinates. If any Z axis is not within the safe height range, continuous motion mode and absolute position mode cannot be used to move the axis. Alternatively, the handwheel can be used to move the corresponding axis. Once it moves to the appropriate position, click "Get Current Position" to complete the coordinate setting. Then, enter the Components interface to adjust the axis coordinates for each axis when taking a photo (all parameters apply to the Z axis and feeder combination). The XY coordinates for the photo are based on the XY coordinates associated with the Z1 axis. The rotation angles of the Z1 and Z2 axes are based on the angles specified in the Z1 axis parameters. The rotation angles of the Z3 and Z4 axes are based on the angles specified in the Z3 axis parameters. The photo height is set independently for each axis, and a separate range search position must be set for each axis. It is important to note that the component orientation during the photo must be consistent with the component orientation at the insertion point. Click Next to enter the Mark parameter setup interface. The default setting is two Mark points: Mark 1 is located in the upper right corner of the board, and Mark 2 is located in the lower left corner. The two marks cannot be on the same horizontal line, nor can the Y coordinate of Mark 1 be smaller than the Y coordinate of Mark 2. Multiple Mark points can be set: if there are multiple boards in the fixture and you want to take a separate Mark image for each board, the setup process is as follows. Assuming there are four boards, set the number of boards to 4, which requires setting data for eight Mark points. If you want to set a shared Mark image, enable the Shared Mark image option. Each two Mark points share a shared image, and the coordinates remain in the upper right corner. Click Next to enter the Plug-in Point interface. Click the image to move it to the desired plug-in location. Move the center of the plug-in's two legs to the center of the image to ensure plug-in accuracy. Select the feeder and Z-axis for the plug-in. Select the Mark number corresponding to the corresponding Mark set. Mark 1#2# represents the first set, Mark 3#4# represents the second set, and so on.

[0100] In some embodiments, in response to the second operation instruction on the human-computer interaction interface, debugging the component to be plugged in according to the test machine model to obtain the target detection result includes the following steps:

[0101] In response to a second operation instruction on the human-computer interaction interface, identifying and locating the center position of the component pin of the component to be inserted to obtain a positioning position;

[0102] The positioning position is subjected to position correction calculation processing according to the test machine model, and the motor is controlled to perform component plug-in processing at the correction position to obtain the target detection result.

[0103] In an embodiment of the present application, in response to the second operation instruction of the human-computer interaction interface, the debugging interface is entered, and the PCB board is fed and fed through the input switch sensor. When the sensing is in place, the PCB board is clamped and positioned, and then the motion module is controlled to control the motor driver to drive the servo motor to move to a preset position to adsorb or clamp the material component. After the material is taken, the component is positioned above the lower camera using a gate-shaped motion method for image acquisition and identification and positioning of the center position of the component pin. Subsequently, the device controls the upper camera to move to the Mark position of the PCB to be plugged in to perform image acquisition, positioning, and position correction calculation of the Mark point, and coordinates the position deviation of the lower camera for the final position correction, and finally controls the motor to the correct position for accurate component insertion. During the descent of the plug-in, the pressure generated by the plug-in can also be detected. If the pressure is too high, a material throwing alarm will be issued to obtain the target detection result.

[0104] In some embodiments, the identifying and locating the center position of the component pin of the component to be inserted to obtain the positioning position includes the following steps:

[0105] Performing image acquisition processing on the center position of the component pin of the plug-in component to obtain a component pin image;

[0106] Inputting the component pin image into a pre-trained image recognition model for recognition processing to obtain recognition coordinates;

[0107] The identified coordinates are subjected to coordinate conversion processing to obtain the positioning position.

[0108] In an embodiment of the present application, a component is positioned above a lower camera for image capture, resulting in an image of the component pins. This image is then input into a pre-trained image recognition model for recognition processing. This image recognition model can be a convolutional neural network (CNN), a Transformer model, or the like. The output coordinates of the component pin center are obtained, resulting in recognition coordinates. These recognition coordinates are the locations of the component pin centers in the image. These recognition coordinates are then converted into positioning positions based on a real-world coordinate system, resulting in the relative position of the component pin centers within the special-shaped insertion machine.

[0109] The following is a detailed description of the embodiments of the present application with reference to specific application examples:

[0110] The embodiment of the present application can be applied to the functional detection scenario of the special-shaped plug-in machine, for example, to perform functional detection on the special-shaped plug-in machine to improve the accuracy of the plug-in of the component by the special-shaped plug-in machine. The embodiment of the present application performs a preliminary detection on the control signals of each functional module of the special-shaped plug-in machine, and calibrates and debugs the special-shaped plug-in machine after the detection is passed, specifically calibrating the camera of the special-shaped plug-in machine to avoid calibration position errors. Then, based on the calibration results, the accuracy of the component plug-in function of the special-shaped plug-in machine is detected, and the component test processing of the special-shaped plug-in machine is performed by combining the human-computer interaction interface, setting the test mode in the control interface, and using the pre-trained deep learning model to perform real-time image analysis on the plug-in component to identify the alignment status of the silver corner and the hole position, thereby obtaining the overall detection result of the special-shaped plug-in machine, which includes the control signal detection of the functional module, the calibration detection of the camera, and the effect detection of the plug-in function. The special-shaped plug-in machine can be adjusted according to the final detection result, thereby improving the accuracy of the plug-in of the component by the special-shaped plug-in machine.

[0111] See also Figure 3 The present application also provides a detection system for a special-shaped plug-in machine, which can implement the above-mentioned detection method for a special-shaped plug-in machine. The system includes:

[0112] The first module 301 is used to obtain the module control process of the special-shaped plug-in machine;

[0113] The second module 302 is used to detect the control connection of the special-shaped plug-in machine according to the module control process to obtain an initial detection result;

[0114] The third module 303 is configured to perform calibration and debugging on the special-shaped plug-in machine through a human-computer interaction interface based on the initial detection result to obtain a calibration result;

[0115] The fourth module 304 is used to send the component to be inserted into the special-shaped insertion machine based on the calibration result, perform component testing on the special-shaped insertion machine through the human-computer interaction interface, and obtain a target detection result.

[0116] It can be understood that the contents of the above method embodiments are all applicable to the present system embodiments, the functions specifically implemented by the present system embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0117] The present application also provides an electronic device comprising a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the above-mentioned detection method for a special-shaped plug-in machine. The electronic device can be any smart terminal including a tablet computer, an in-vehicle computer, or the like.

[0118] It can be understood that the contents of the above method embodiments are applicable to the present device embodiments, the functions specifically implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0119] See also Figure 4 , Figure 4 The hardware structure of an electronic device according to another embodiment is shown. The electronic device includes:

[0120] The processor 401 may be implemented as a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is configured to execute relevant programs to implement the technical solutions provided in the embodiments of the present application.

[0121] Memory 402 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). Memory 402 can store an operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in memory 402 and is called by processor 401 to execute the detection method for the special-shaped plug-in machine in the embodiments of this application.

[0122] Input / output interface 403, used to implement information input and output;

[0123] Communication interface 404, used to implement communication interaction between this device and other devices, which can be achieved through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WiFi, Bluetooth, etc.);

[0124] Bus 405 , which transmits information between various components of the device (e.g., processor 401 , memory 402 , input / output interface 403 , and communication interface 404 );

[0125] The processor 401 , the memory 402 , the input / output interface 403 and the communication interface 404 are connected to each other in communication within the device via a bus 405 .

[0126] An embodiment of the present application further provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the above-mentioned detection method for the special-shaped plug-in machine is implemented.

[0127] It can be understood that the contents of the above method embodiments are all applicable to the present storage medium embodiment, the functions specifically implemented by the present storage medium embodiment are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0128] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely arranged relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0129] The embodiments of the present application provide a method, system, electronic device, and storage medium for detecting a special-shaped plug-in machine. This solution obtains an initial detection result by obtaining the module control flow of the special-shaped plug-in machine and detecting and processing the control connection of the special-shaped plug-in machine according to the module control flow. Based on the initial detection result, the special-shaped plug-in machine is calibrated and debugged to obtain a calibration result. Based on the calibration result, the component to be plugged in is sent into the special-shaped plug-in machine. The component test process of the special-shaped plug-in machine is performed through a human-computer interaction interface to obtain a target detection result. This can facilitate the testing of the control connection and function of the special-shaped plug-in machine. Moreover, since the present solution calibrates and debugs the special-shaped plug-in machine, it can alleviate the problem of deviation in the calibration position. The component test of the special-shaped plug-in machine is performed through the human-computer interaction interface, which can detect the position and control accuracy of the plug-in, which is conducive to improving the accuracy of the plug-in.

[0130] The embodiments described in the embodiments of this application are intended to more clearly illustrate the technical solutions of the embodiments of this application and do not constitute a limitation on the technical solutions provided by the embodiments of this application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0131] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0132] The system embodiment described above is merely illustrative. The units described as separate components may or may not be physically separate, i.e., they may be located in one place or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of this embodiment.

[0133] Those skilled in the art will appreciate that all or some of the steps in the methods, systems, and functional modules / units in the devices disclosed above may be implemented as software, firmware, hardware, or appropriate combinations thereof.

[0134] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0135] It should be understood that in this application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0136] In the several embodiments provided in this application, it should be understood that the disclosed systems and methods can be implemented in other ways. For example, the system embodiments described above are merely illustrative. For example, the division of the above units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of systems or units, which can be electrical, mechanical or other forms.

[0137] The units described above as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0138] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0139] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes multiple instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes: various media that can store programs, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0140] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but are not intended to limit the scope of the present invention. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and essence of the present invention should be within the scope of the present invention.

Claims

1. A detection method for a special-shaped plug-in machine, characterized in that: The method comprises the following steps: Get the module control flow of the special-shaped plug-in machine; Performing detection processing on the control connection of the special-shaped plug-in machine according to the module control process to obtain an initial detection result; Calibration and debugging are performed on the special-shaped plug-in machine based on the initial detection result to obtain a calibration result; Based on the calibration results, the component to be inserted is sent to the special-shaped insertion machine, and the special-shaped insertion machine is subjected to component testing through a human-computer interaction interface to obtain a target detection result; The module control process of obtaining the special-shaped plug-in machine includes the following steps: Analyze and process the hardware modules of the special-shaped plug-in machine to obtain a motion control module, an input / output control module, and an image acquisition module; Analyzing and processing the signal transmission relationship among the motion control module, the input / output control module, and the image acquisition module to obtain the module control flow; The calibration and debugging process of the special-shaped plug-in machine is performed based on the initial detection result to obtain the calibration result, including the following steps: Performing image acquisition processing on the image acquisition device of the special-shaped plug-in machine by the image acquisition module to obtain an acquired image; Performing feature point coordinate determination processing on the collected image to obtain physical coordinates and pixel coordinates; Performing affine transformation and coordinate conversion processing on the physical coordinates and the pixel coordinates to obtain a calibrated rotation center; The image acquisition device of the special-shaped plug-in machine is calibrated according to the calibrated rotation center to obtain the calibration result.

2. The method according to claim 1, characterized in that The detecting and processing of the control connection of the special-shaped plug-in machine according to the module control process to obtain an initial detection result includes the following steps: The motion control module detects the sensor input signal and the motor input signal of the special-shaped plug-in machine, and detects the motion control signal output by the motion control module to obtain a signal detection result; The input-output control module performs signal output processing on the special-shaped plug-in machine according to the signal detection result and the collected image to obtain the initial detection result.

3. The method according to claim 1, characterized in that The method includes sending the component to be inserted into the special-shaped insertion machine based on the calibration result, performing component testing on the special-shaped insertion machine through a human-computer interaction interface, and obtaining a target detection result, including the following steps: According to the calibration result, the image of the component to be inserted is collected and processed by the special-shaped insertion machine to obtain a target image; Displaying the target image in the human-computer interaction interface; generating a test machine model according to the target image in response to a first operation instruction on the human-computer interaction interface; In response to a second operation instruction on the human-computer interaction interface, the component to be plug-in is debugged according to the test machine model to obtain the target detection result.

4. The method according to claim 3, characterized in that The step of generating a test model according to the target image in response to a first operation instruction on the human-computer interaction interface includes the following steps: In response to a first operation instruction on the human-computer interaction interface, entering a parameter modification interface to perform parameter modification processing to obtain machine model parameters; Enter the component interface to adjust the material picking position of each axis to obtain the coordinate position; Performing a marking point setting process according to the machine model parameters and the coordinate position to obtain a marking point; The target image is moved according to the marking points to obtain the test model.

5. The method according to claim 3, characterized in that The step of performing debugging on the component to be plugged in according to the test machine model in response to the second operation instruction on the human-computer interaction interface to obtain the target detection result includes the following steps: In response to a second operation instruction on the human-computer interaction interface, identifying and locating the center position of the component pin of the component to be inserted to obtain a positioning position; The positioning position is subjected to position correction calculation processing according to the test machine model, and the motor is controlled to perform component plug-in processing at the correction position to obtain the target detection result.

6. The method according to claim 5, characterized in that The identifying and positioning processing of the center position of the component pin of the component to be inserted to obtain the positioning position includes the following steps: Performing image acquisition processing on the center position of the component pin of the plug-in component to obtain a component pin image; Inputting the component pin image into a pre-trained image recognition model for recognition processing to obtain recognition coordinates; The identified coordinates are subjected to coordinate conversion processing to obtain the positioning position.

7. A detection system for a special-shaped plug-in machine, characterized in that: The system comprises: The first module is used to obtain the module control flow of the special-shaped plug-in machine; The second module is used to detect and process the control connection of the special-shaped plug-in machine according to the module control process to obtain an initial detection result; The third module is used to calibrate and debug the special-shaped plug-in machine through a human-computer interaction interface based on the initial detection result to obtain a calibration result; A fourth module is configured to send the component to be inserted into the special-shaped insertion machine based on the calibration result, perform component testing on the special-shaped insertion machine through the human-computer interaction interface, and obtain a target detection result; The first module is used to obtain the module control process of the special-shaped plug-in machine, including: Analyze and process the hardware modules of the special-shaped plug-in machine to obtain a motion control module, an input / output control module, and an image acquisition module; Analyzing and processing the signal transmission relationship among the motion control module, the input / output control module, and the image acquisition module to obtain the module control flow; The third module is configured to perform calibration and debugging on the special-shaped plug-in machine based on the initial detection result to obtain a calibration result, including: Performing image acquisition processing on the image acquisition device of the special-shaped plug-in machine by the image acquisition module to obtain an acquired image; Performing feature point coordinate determination processing on the collected image to obtain physical coordinates and pixel coordinates; Performing affine transformation and coordinate conversion processing on the physical coordinates and the pixel coordinates to obtain a calibrated rotation center; The image acquisition device of the special-shaped plug-in machine is calibrated according to the calibrated rotation center to obtain the calibration result.

8. An electronic device, characterized in that: The electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the method according to any one of claims 1 to 6 when executing the computer program.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images