Automatic integration method and system for electromechanical prefabricated components

Abstract

The invention provides an electromechanical prefabricated component automatic integration method and system. The method comprises the following steps: firstly, receiving integration demand input through a user terminal; secondly, acquiring an image of a to-be-integrated electromechanical component in real time through a CCD camera; then, splicing the preprocessed integration demand input data, the image data of the to-be-integrated electromechanical component and the image information data in the feature database to form an N-dimensional input vector; thirdly, the N-dimensional input vector is input into the trained convolutional neural network, and the trained convolutional neural network outputs the name and the position of a current component needing to be integrated; thirdly, the PLC control system controls the manipulator and the transmission device to automatically integrate the components according to the names and the positions of the current components needing to be integrated; and finally, returning to repeatedly execute the steps until the assembly is completed. Accurate selection of the components is realized by adopting the convolutional neural network algorithm, and the accuracy and efficiency of automatic integration of the electromechanical prefabricated components are greatly improved by utilizing the PLC control system.

Description

Technical Field

[0001] The present invention relates to the field of intelligent manufacturing automation technology, and specifically to an automated integration method and system for prefabricated electromechanical components. Background Art

[0002] With the advent of the Industrial 4.0 era, the manufacturing industry is rapidly developing from the traditional manual assembly mode to automation and intelligence. The automated integration technology of prefabricated electromechanical components is an important part of modern industrial manufacturing. Its goal is to complete the assembly task of complex components with high precision in the shortest time. However, the existing automated assembly methods still have great limitations in terms of efficiency, precision and flexibility in dealing with the identification and assembly of complex components.

[0003] At present, the automation integration technologies used in industrial assembly processes are usually based on fixed processes and preset programs. These technologies perform well when assembling components of a single model or a small number of variants, but the adaptability of existing technologies is insufficient in the face of scenarios with diverse component types and complex assembly requirements. Specifically, it is manifested as follows: Difficult component recognition: Traditional image processing technologies (such as simple edge detection or feature matching) have low recognition accuracy when facing complex shapes, different lighting conditions or diverse components, and are prone to misjudgment or missed judgments. Lack of dynamic adjustment capabilities: Many systems rely on fixed assembly processes and cannot dynamically optimize the operation process according to different assembly requirements (such as time, accuracy or function). Low data transmission efficiency: The existing assembly system lacks efficient communication protocol support during the transmission of control signals, and data transmission delays may cause instability in the operation of the manipulator. Imperfect system feedback: During the assembly process, the lack of a real-time feedback mechanism makes it difficult to make timely adjustments when the assembly is abnormal or the error is large, reducing the reliability and efficiency of the system. In response to the above problems, the rise of artificial intelligence technology, especially deep learning algorithms, in recent years has provided new solutions for complex component recognition and assembly. Convolutional neural networks (CNNs) have demonstrated powerful capabilities in image feature extraction and classification, and their potential in combination with industrial automation control is gradually emerging. At the same time, with the widespread application of industrial communication protocols (such as Modbus), the real-time and reliability of data transmission have also been effectively improved.

[0004] In addition, the existing technology cannot produce and manufacture according to the dynamic needs of users, and does not involve input and analysis to combine the demand data with the component image features to achieve dynamic adjustment of different tasks. There is no intelligent equipment that receives the assembly target function, time and accuracy requirements through the user terminal, and then automatically and dynamically produces and manufactures in a targeted manner; and the existing technology only uses the existing artificial intelligence algorithm, and does not involve adding the number of components as a consideration factor into the intelligent matching process, resulting in a decrease in the accuracy of component selection in the intelligent automated production process. Because the recognition process cannot be flexibly and dynamically adjusted, the dynamic flexibility adjustment and accuracy of the intelligent manufacturing production process need to be improved. Summary of the invention

[0005] In view of the above problems mentioned in the prior art, the present invention provides an automated integration method and system for prefabricated electromechanical components. The method first receives the integration requirement input through a user terminal; secondly, the image of the electromechanical component to be integrated is collected in real time through a CCD camera, and the image information of each component sub-image is calculated; then, the preprocessed integration requirement input data, the image data of the electromechanical component to be integrated, and the image information data in the feature database are spliced ​​to form an N-dimensional input vector; thirdly, the N-dimensional input vector is input into a trained convolutional neural network, and the trained convolutional neural network outputs the name and position of the current component to be integrated; thirdly, the PLC control system is used to control the manipulator and the transmission device to automatically integrate the components according to the current name and position of the component to be integrated; finally, the steps are returned and repeated until the assembly is completed. This application uses a convolutional neural network algorithm to achieve accurate selection of components, and uses a PLC control system to greatly improve the accuracy and efficiency of automated integration of prefabricated electromechanical components.

[0006] The present application provides a method for automated integration of electromechanical prefabricated components, comprising the steps of:

[0007] S1: receiving an integration requirement input through a user terminal, wherein the integration requirement includes an integrated assembly target function, an assembly time requirement, and an assembly accuracy requirement of a prefabricated electromechanical component;

[0008] S2: The image of the electromechanical component to be integrated is collected in real time by a high-definition camera, and the image is segmented to obtain each component sub-image, and the image information of each sub-image is calculated. The image information includes grayscale mean, maximum value, number of pixels and pixel position information, and the image information is stored in the component feature database;

[0009] S3: splicing the preprocessed integration requirement input data, the image data of the electromechanical component to be integrated, and the image information data in the feature database to form an N-dimensional input vector;

[0010] S4: Input the N-dimensional input vector into the trained convolutional neural network, and the trained convolutional neural network outputs the name and position of the component to be integrated;

[0011] S5: The PLC control system controls the manipulator and the transmission device to automatically integrate the components according to the names and positions of the components to be integrated, and transmits information through the real-time communication protocol Modbus to ensure accurate instruction transmission;

[0012] S6: Return to step S2 until the assembly is completed.

[0013] Preferably, the integrated assembly target functions include: assembly of motor drive modules, modular integration of automation control cabinets or installation of end-of-arm tools; the assembly time requirements include: completion time limit for assembly tasks, in hours or minutes, to be completed within 2 hours, 5 hours or 10 hours; the assembly accuracy requirements include: error not exceeding 0.1 mm or error not exceeding 2 cm.

[0014] Preferably, segmenting the image to obtain each component sub-image includes denoising the image using Gaussian filtering, extracting component boundaries using a Canny operator edge detection algorithm, and then segmenting the image to obtain the component sub-images.

[0015] Preferably, the preprocessed integration requirement input data, the image data of the electromechanical component to be integrated, and the image information data in the feature database are spliced ​​to form an N-dimensional input vector, including: preprocessing the integration requirement input data including normalizing the time requirements, cost budget and accuracy requirements, and unique-hot encoding the target function, normalizing the image data and image information data of the electromechanical component to be integrated, and splicing them in the order of the integration requirement input data, the image data and image information data of the electromechanical component to be integrated to form an N-dimensional input vector as the input of the convolutional neural network.

[0016] Preferably, the trained convolutional neural network outputs the name and position of the component that currently needs to be integrated, including: S41: the convolutional neural network receives an N-dimensional input vector; S42: extracting local features in the input vector through the convolutional layer of the convolutional neural network, including the shape, edge, grayscale distribution and functional priority of the component; S43: classifying the extracted local features through the fully connected layer, and outputting the name and category label of the component that currently needs to be integrated; S44: performing position regression on the image features of the component through the regression layer, and predicting the center point coordinates or bounding box position of the component in the image; S45: based on the results of classification and regression, outputting the name of the component that currently needs to be integrated and its corresponding position coordinates, and passing the output data to the PLC for execution control.

[0017] Preferably, the trained convolutional neural network is expressed as follows using an improved activation function f(x):

[0018]

[0019] Among them, x represents the input value of the current neuron, e is the base of the natural logarithm, and N is the number of component sub-images.

[0020] Preferably, the PLC control system controls the manipulator and the transmission device to automatically integrate components according to the name and position of the component to be integrated, including: S51: the output of the convolutional neural network sends the component name and position data directly to the PLC through the Modbus protocol; S52: the PLC parses the received component name and maps it to the internal function module register; S53: the PLC parses the position data, converts it into three-dimensional coordinates and generates motion trajectory instructions for the manipulator; S54: the PLC controls the manipulator and the transmission device to perform grasping and installation tasks according to the generated motion trajectory instructions.

[0021] The present application also provides an automated integration system for prefabricated electromechanical components, comprising:

[0022] Demand input module: receiving integrated demand input through a user terminal, wherein the integrated demand includes integrated assembly target function, assembly time requirement and assembly accuracy requirement of the electromechanical prefabricated component;

[0023] Image processing module: collects images of the electromechanical components to be integrated in real time through a high-definition camera, segments the images to obtain sub-images of each component, and calculates the image information of each sub-image, including grayscale mean, maximum value, number of pixels and pixel position information, and stores the image information in the component feature database;

[0024] Vector generation module: splices the pre-processed integration requirement input data, the image data of the electromechanical components to be integrated, and the image information data in the feature database to form an N-dimensional input vector;

[0025] Convolutional neural network calculation module: inputs the N-dimensional input vector into the trained convolutional neural network, and the trained convolutional neural network outputs the name and position of the current component to be integrated;

[0026] Execution module: The PLC control system controls the manipulator and the transmission device to automatically integrate the components according to the names and positions of the components to be integrated, and transmits information through the real-time communication protocol Modbus to ensure accurate instruction transmission;

[0027] Loop judgment module: return to the image processing module until the assembly is completed.

[0028] Preferably, the integrated assembly target functions include: assembly of motor drive modules, modular integration of automation control cabinets or installation of end-of-arm tools; the assembly time requirements include: completion time limit for assembly tasks, in hours or minutes, to be completed within 2 hours, 5 hours or 10 hours; the assembly accuracy requirements include: error not exceeding 0.1 mm or error not exceeding 2 cm.

[0029] Preferably, segmenting the image to obtain each component sub-image includes denoising the image using Gaussian filtering, extracting component boundaries using a Canny operator edge detection algorithm, and then segmenting the image to obtain the component sub-images.

[0030] The present invention provides an automated integration method and system for electromechanical prefabricated components, which can achieve the following beneficial technical effects:

[0031] 1. The present invention first receives the integration demand input through the user terminal; secondly, the image of the electromechanical component to be integrated is collected in real time through the CCD camera, and the image information of each component sub-image is calculated; then, the pre-processed integration demand input data, the image data of the electromechanical component to be integrated and the image information data in the feature database are spliced ​​to form an N-dimensional input vector; again, the N-dimensional input vector is input into the trained convolutional neural network, and the trained convolutional neural network outputs the name and position of the current component to be integrated; again, the manipulator and the transmission device are controlled by the PLC control system to automatically integrate the components according to the name and position of the current component to be integrated; finally, the steps are returned and repeated until the assembly is completed. This application realizes the precise selection of components by adopting the convolutional neural network algorithm, and uses the PLC control system to greatly improve the accuracy and efficiency of the automated integration of electromechanical prefabricated components. This application receives the integration demand input through the user terminal, including the assembly target function, time requirements and accuracy requirements, to achieve dynamic adaptation to different task scenarios. Preprocessing and unique hot encoding technology are used to convert complex requirements into standardized data that can be used for neural network calculation, effectively improving the flexibility of the system and meeting the needs of large-scale industrial production and small-batch customized production.

[0032] 2. The present invention collects component images through a high-definition camera, and uses Gaussian filtering and Canny edge detection algorithms to segment the images, significantly reducing the interference of complex backgrounds on component recognition. The deep features in the component images are extracted using a convolutional neural network, and the names of the components that need to be integrated are classified and output. At the same time, the installation positions of the components are predicted through position regression to achieve high-precision recognition of complex components. The introduction of dynamic activation functions (with the number of component sub-images as dynamic parameters) enhances the adaptability of neural networks to tasks of different complexities. By taking the number of component sub-images as a consideration and adding it to the recognition calculation process of the convolutional neural network, the recognition accuracy is further improved.

[0033] 3. The present invention accurately controls the motion trajectory of the manipulator and the transmission device through the PLC control system, and ensures that the components can be accurately installed according to the design requirements according to the component names and position data output by the CNN. The real-time communication protocol Modbus is used to realize data transmission between the neural network, PLC and manipulator controller, ensuring the real-time and accuracy of instruction transmission, and effectively avoiding assembly errors caused by data delays. During the assembly process, a variety of sensors (such as position sensors, torque sensors and visual sensors) are used to monitor the installation status of the components in real time to determine whether there are problems such as position deviation, excessive force or wrong direction. Combined with the sensor monitoring results and the real-time feedback mechanism of the convolutional neural network, a closed-loop control system is formed to dynamically adjust the manipulator operation in abnormal situations until the component installation is completed. Through the cyclic judgment and feedback optimization of monitoring data, the fault tolerance and operational stability of the system are significantly improved.

[0034] 4. The present invention efficiently integrates demand data, image features and component information through the generation of N-dimensional input vectors, reduces data redundancy and improves computing efficiency. The system adopts a modular design, including a demand input module, an image processing module, a vector generation module, a convolutional neural network calculation module, an execution module and a loop judgment module. Each module works together to improve the overall operating efficiency. The automated processing of the assembly process significantly reduces the frequency of manual intervention, shortens the assembly time, and meets the needs of efficient production. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0036] Figure 1 It is a schematic diagram of the steps of an automated integration method for electromechanical prefabricated components of the present invention;

[0037] Figure 2 It is a schematic diagram of an automated integration system for prefabricated electromechanical components of the present invention. DETAILED DESCRIPTION

[0038] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0039] Embodiment 1:

[0040] In view of the above problems mentioned in the prior art, in order to solve the above technical problems, as shown in the attached Figure 1 As shown: This application provides a method for automated integration of electromechanical prefabricated components, including the steps of:

[0041] S1: receiving an integration requirement input through a user terminal, wherein the integration requirement includes an integrated assembly target function, an assembly time requirement, and an assembly accuracy requirement of a prefabricated electromechanical component; assembling a motor drive module

[0042] In one embodiment, the assembly target function is integrated, and the goal is to assemble the motor drive module to the joint part of the robot manipulator to ensure that the module can drive the joint rotation normally. Input: Select the "motor drive module assembly" option through the user terminal. The system identifies the "motor drive module" as the component to be assembled currently. Set the operation target to ensure the accurate connection of the module electrical interface and mechanical interface. The task requires that the assembly time of a single module in the industrial production line does not exceed 2 hours. Input: Enter the time requirement as "2 hours" through the terminal. The system uses this time requirement as an optimization target to dynamically adjust the assembly path and the robot operation speed. If the system detects that the actual assembly time may exceed 2 hours, an abnormal prompt is triggered. The installation angle error of the motor drive module is required to be no more than 0.1 mm, and the torque value is kept within the specified range. Input: Select the accuracy requirement as "0.1 mm" through the terminal. The system passes the error range parameter to the manipulator controller and torque sensor to calibrate the assembly position and force in real time. In one embodiment, the assembly of the automation control cabinet module integrates the assembly target function, and the goal is to modularly install the power module, communication module and controller module of the automation control cabinet. Input: Input the assembly tasks of "power module", "communication module" and "controller module" in sequence through the user terminal. The system identifies and determines the modules to be assembled in sequence. Automatically generates the assembly order and path of the modules. The task requires that the assembly time of a complete control cabinet should not exceed 5 hours during the production process of the assembly line. Input: Enter the time requirement of "5 hours" through the terminal. The system optimizes the task allocation and path planning of the manipulator based on the number of modules and assembly complexity. If the estimated time exceeds 5 hours, the system will optimize the path or issue an alarm. The plane error of the installation position of each module in the control cabinet shall not exceed 2 mm, and the connection deviation of the wiring port shall not exceed 1 mm. Input: Enter "plane error ≤2 mm, wiring port deviation ≤1 mm" through the terminal. The plane error is calibrated in real time through a laser positioning sensor. The deviation of the wiring port is monitored by a visual sensor, and the operating angle of the manipulator is adjusted.

[0043] In one embodiment, the installation of the tool at the end of the robot arm integrates the assembly target function, and the goal is to install the welding tool at the end of the robot arm to the specified position to ensure that the tool can complete the high-precision welding task. Input: Select "welding tool installation" through the terminal. The system identifies "welding tool" as the target component to be installed. Automatically set the alignment and fixing tasks of the tool connection interface. The task requires the rapid installation of the tool within 90 minutes. Input: Enter the time requirement as "90 minutes" through the terminal. The system gives priority to the efficient path to reduce the movement and adjustment time of the robot. If the installation process delay is detected, the system records and optimizes the subsequent tasks. The positioning error of the welding tool does not exceed 0.05 mm, and the angle deviation of the rotating axis does not exceed 0.1 degrees. Input: Enter "positioning error ≤ 0.05 mm, angle deviation ≤ 0.1 degrees" through the terminal. The system combines visual sensors and torque sensors to monitor the installation status, the positioning error is adjusted by the PLC to adjust the motion trajectory, and the angle deviation is calibrated through real-time feedback.

[0044] S2: The image of the electromechanical component to be integrated is collected in real time by a high-definition camera, and the image is segmented to obtain each component sub-image, and the image information of each sub-image is calculated, and the image information includes the grayscale mean, maximum value, number of pixels and pixel position information, and the image information is stored in the component feature database; In one embodiment, the image acquisition and processing of the motor drive module, the motor drive module is composed of multiple sub-components, such as the motor body, the drive circuit module and the interface module, and it is necessary to collect its overall image through the camera and identify the position and characteristics of the sub-components. Use a high-definition industrial camera (resolution 1920x1080) to shoot the motor drive module to be assembled. The light source is adjusted to a ring LED light source to ensure uniform brightness of the image. Denoising: Use a Gaussian filter to eliminate image noise and improve the effect of edge detection. Filter parameters: kernel size is 5×5, standard deviation is 1. Edge detection: The Canny operator is used to detect the edges of the motor body and the drive module. Threshold setting: The low threshold is 50 and the high threshold is 150. Connected domain analysis: Connected domain analysis is performed on the edge detection results to separate the contours of each sub-component. Three connected domains are detected, corresponding to the motor body, drive circuit module and interface module. Image information calculation: Extract the following image information for each connected domain: Grayscale mean: Calculate the grayscale average of the pixels in the connected domain to determine the material properties. Grayscale maximum: Extract the maximum grayscale value to determine the brightness of the component surface. Number of pixels: Count the number of pixels in the connected domain to estimate the area of ​​the component. Pixel position information: Record the center point coordinates (x, y) and bounding box (xmin, ymin, xmax, ymax) of the connected domain. Motor body: Grayscale mean 128, maximum 200, number of pixels 5000, center point coordinates (300, 400). Drive module: Grayscale mean 100, maximum 180, number of pixels 3000, center point coordinates (600, 400).

[0045] In one embodiment, the image acquisition and processing of the automation control cabinet module, the automation control cabinet contains multiple sub-modules (power module, controller module, communication module), and the position and size of each module need to be identified through image segmentation. Use a high-resolution camera to capture the overall image inside the control cabinet. Adjust the light source angle to avoid segmentation errors caused by reflection. Histogram equalization: Enhance the grayscale of the image to improve the contrast between modules. Edge detection: Use the Sobel operator to detect the module edge. Morphological processing: Eliminate small noise and close the module edge through expansion and corrosion operations. Region segmentation: Perform region segmentation on the image based on the edge result to extract the outline of each module. Image information calculation: Grayscale statistics calculate the grayscale mean and maximum value for each module area. Area calculation counts the number of pixels in the module area and estimates the physical size of the module. Position record records the center point and bounding box of the module. The grayscale mean of the power module is 120, the maximum value is 220, the number of pixels is 4500, and the center point coordinates are (250,300). Controller module: grayscale mean 110, maximum 200, number of pixels 4000, center point coordinates (500,300). Data is stored in the component feature database for subsequent feature matching and assembly optimization.

[0046] In one embodiment, the image acquisition and processing of the tool at the end of the robot arm, the tool at the end of the robot arm is irregular in shape, and the installation interface and grasping point position of the tool are identified by image segmentation. A 3D camera is used to acquire a depth image of the tool at the end of the robot arm. Depth resolution: 640x480. Depth slicing: Slice the depth image according to the set threshold (10-50mm) and only retain the tool area. Region growing: Perform region growing based on the depth slicing results to extract the overall contour of the tool. Image information calculation: Extract the grasping point and installation interface position of the tool. Grab point: The maximum gradient position of the depth map. Installation interface: The center position of the contour. Statistics of the geometric features of the tool, such as the maximum depth, average depth and grasping point coordinates. Feature storage: Store feature information for robot grasping and installation tasks.

[0047] S3: splicing the preprocessed integration requirement input data, the image data of the electromechanical component to be integrated, and the image information data in the feature database to form an N-dimensional input vector, splicing the N-dimensional input vector.

[0048] In one embodiment, the production line needs to assemble a motor drive module. The assembly requirements are input through the user terminal, and the image data and image feature information of the component are combined to generate an N-dimensional input vector for processing by the convolutional neural network. The integrated requirement input data, the user inputs the target function motor drive module assembly through the terminal, and is encoded as [1,0,0] (unique hot encoding). Assembly time requirement: 2 hours, normalized to 0.5. Accuracy requirement: the error does not exceed 0.1 mm, normalized to 0.9. The result generates a vector [1,0,0,0.5,0.9]. Image data of the electromechanical component to be integrated: After the image is collected by a high-definition camera, the features are extracted by Gaussian filtering and edge detection. After feature processing, the dimension is reduced to [0.12, 0.08, 0.15, 0.18]. Image information data (from the component feature database): grayscale mean: 0.6, grayscale maximum: 0.9, number of pixels: 0.8, position information (center point coordinates): [0.2, 0.3]. Result: Generate a vector [0.6, 0.9, 0.8, 0.2, 0.3]. Concatenate to form an N-dimensional vector. Concatenate the above three parts of data in order to generate a complete N-dimensional vector: [1, 0, 0, 0.5, 0.9, 0.12, 0.08, 0.15, 0.18, 0.6, 0.9, 0.8, 0.2, 0.3], and the final dimension N = 14.

[0049] In one embodiment, the convolutional neural network structure for processing the above-mentioned N-dimensional input vector by the CNN structure includes the following main parts: Input layer: Receive the spliced ​​N-dimensional vector as the initial input of the network. The data format is a one-dimensional vector, but it can be regarded as a row of a two-dimensional matrix, which is convenient for convolution operations. Convolution layer: Extract local features in the input vector through multiple convolution kernels, such as the relationship between different demand parameters or the association between image features and demand. Each convolution kernel focuses on different features, such as component location, function priority, etc. Activation layer: The combination of Swish and Mish activation functions makes the network's response to complex features smoother and more accurate. The activation layer can introduce nonlinearity and improve the model's ability to fit complex relationships. Pooling layer: Use the maximum pooling layer to reduce feature dimensions, extract key information and reduce computational complexity. In this example, the pooling operation aggregates some dimensions of the input vector to ensure that the model focuses on the main features. Fully connected layer: Map the extracted local features to a high-dimensional space for feature combination. Output the name (such as "motor drive module") and position (such as center point coordinates (300,400) or bounding box) of the current component to be assembled. Regression layer: Regression prediction of the specific position of the component (such as bounding box coordinates or center point coordinates); the output value is a continuous value to guide the robot to grasp and install. Output layer: The classification part outputs the name and category of the current component to be integrated; the regression part outputs the position coordinates of the current component.

[0050] In one embodiment, the working principle of CNN, feature extraction (convolution layer): The convolution layer gradually extracts local features of the input vector through a sliding window operation. For example: how the assembly time and accuracy requirements input by the user affect the component selection. The relationship between image information and demand priority. Different convolution kernels will focus on different features to form a multi-channel feature map. Feature screening (pooling layer): The pooling layer reduces the dimension of the feature map output by the convolution layer, retains the most important features, and reduces data redundancy. For example, the pooling operation may filter out the strong correlation features of "time" and "position". Feature integration (fully connected layer): The reduced-dimensional feature map is converted into a high-dimensional space feature vector. Through the weighted summation of weights and biases, the complex nonlinear relationship between different input features is learned. Classification and regression (output layer): The classification part outputs the name of the component currently to be assembled, such as "motor drive module". The regression part outputs the position coordinates of the component (such as the center point or bounding box position) to guide the operation of the manipulator. Input: [1,0,0,0.5,0.9,0.12,0.08,0.15,0.18,0.6,0.9,0.8,0.2,0.3], output classification result: the name of the component to be assembled is "motor drive module". Regression result: the position coordinates of the component are the center point (300,400).

[0051] S4: Input the N-dimensional input vector to the trained convolutional neural network, and the trained convolutional neural network outputs the name and position of the current component to be integrated; in one embodiment, the processing input layer of the convolutional neural network inputs: 14-dimensional vector data. The input is regarded as a one-dimensional sequence and input to the first layer of CNN. The convolution layer uses multiple convolution kernels to extract local features in the input vector: for example, the convolution kernel can extract the relationship between time and accuracy requirements, or extract the correlation between image features and requirements. Each convolution kernel outputs a feature map to form a multi-channel feature representation. The activation layer adopts the form of combining Swish and Mish activation functions to enhance the nonlinear expression ability of features. In the activation layer, the sparsity and local details of the input features are retained to improve the prediction accuracy of the model. The pooling layer uses the maximum pooling operation to compress the feature map, reduce the feature dimension, and retain the main information. For example: the strong correlation between time requirements and position relationships is retained. The fully connected layer maps the pooled features to a high-dimensional space to generate the classification label and position features of the component. The classification result outputs the category label ("motor drive module") through the Softmax function. Position prediction regresses the spatial position of the component through the fully connected weight. The output layer outputs the classification result (component name) and position coordinates: Component name: Motor drive module. Position center point coordinates (x=300, y=400). Classification output CNN identifies the name of the component to be assembled as "motor drive module" through the classification network module. Classification label encoding: Position regression output CNN predicts the spatial position coordinates of the component through the regression module: The predicted center point coordinates are (300, 400). The predicted bounding box is (xmin=250, ymin=350, xmax=350, ymax=450). System response and task transfer, the system packages the component name and position coordinates output by CNN into instruction data and transfers it to the PLC control system. The PLC generates robot motion instructions based on the name and coordinates to perform grasping and installation tasks.

[0052] S5: The PLC control system controls the manipulator and the transmission device to automatically integrate the components according to the name and position of the current components to be integrated, and transmits information through the real-time communication protocol Modbus to ensure accurate instruction transmission; in one embodiment, on an automated production line, the "motor drive module" needs to be moved from the storage rack to the assembly position and accurately installed on the specified robot joint. This process coordinates the movement of the manipulator and the transmission device through the PLC control system, and uses the real-time communication protocol Modbus to ensure the accuracy of instruction transmission. Step 1: Data reception and parsing, receiving the data output by CNN Component name: motor drive module (encoding [0x01]). Position information: storage position (x=100, y=200, z=50), installation position (x=300, y=400, z=60). PLC parsing data: PLC receives the above data through the Modbus protocol and maps it to internal registers: Name register: store [0x01] to represent the motor drive module. Position information register: store the source position and target position coordinates. Step 2: Control the manipulator to complete the grasping operation and generate the motion trajectory: The PLC calculates the motion trajectory of the manipulator based on the source position coordinates (x=100, y=200, z=50). The manipulator moves from the initial position to (x=100, y=200, z=50). Control the gripper to open and lower the manipulator to grasp the component. Execute the grasping: The torque sensor monitors the grasping force in real time to ensure that the force is within the safe range. The PLC adjusts the gripper force through real-time feedback to avoid damaging the component. Step 3: Control the transmission device to complete the handling and start the transmission device: The PLC instructs the transmission device (such as a roller conveyor belt) to move the component to a temporary storage position near the assembly area. The transmission device receives the target position data from the PLC via Modbus. Real-time monitoring of the transmission status: The sensor monitors whether the component has reached the temporary storage position. If the component position is offset, the PLC automatically adjusts the speed or direction of the transmission device. Step 4: Control the manipulator to complete the installation, and generate the motion trajectory: PLC generates the installation path of the manipulator according to the target position coordinates (x=300, y=400, z=60). The manipulator grabs the component and moves it from the temporary storage position to the installation position. The manipulator is aligned according to the accuracy requirements at the installation position. The installation torque sensor monitors the contact force during the installation process in real time to ensure that the component is installed in place. After the installation is completed, the PLC controls the manipulator to release the gripper and return to the initial position. The installation status detection position sensor and the visual sensor detect in real time whether the installation position meets the accuracy requirements. If the deviation exceeds the set range (such as 0.1 mm), the PLC readjusts the manipulator action. Instruction feedback PLC sends the installation completion signal to the main control system via Modbus. The main system records that the current component has been successfully installed and triggers the next assembly task. Application of Modbus protocol in information transmission, request frame structure data format: contains device address, function code, data length and check code.[Device address = 01, function code = 03, data length = 06, data = [0x01, 0x64, 0xC8, 0x32], CRC check = XXXX], data mapping: component name and position data are mapped to the function register of the PLC. The manipulator and transmission device read the register content and execute instructions through Modbus. Real-time guarantee Modbus supports master-slave mode, and the polling cycle is set to 10ms to ensure that data transmission is not delayed. Component installation is completed. The motor drive module is accurately installed to the specified position with an error of less than 0.1 mm. Feedback confirmation Through the Modbus protocol, the PLC sends a task completion signal to the main control system and enters the next assembly process.

[0053] S6: Return to step S2 until the assembly is completed.

[0054] Preferably, the integrated assembly target functions include: assembly of motor drive modules, modular integration of automation control cabinets or installation of end-of-arm tools; the assembly time requirements include: completion time limit for assembly tasks, in hours or minutes, to be completed within 2 hours, 5 hours or 10 hours; the assembly accuracy requirements include: error not exceeding 0.1 mm or error not exceeding 2 cm.

[0055] Preferably, segmenting the image to obtain each component sub-image includes denoising the image using Gaussian filtering, extracting component boundaries using a Canny operator edge detection algorithm, and then segmenting the image to obtain the component sub-images.

[0056] Preferably, the preprocessed integration requirement input data, the image data of the electromechanical component to be integrated, and the image information data in the feature database are spliced ​​to form an N-dimensional input vector, including: preprocessing the integration requirement input data including normalizing the time requirements, cost budget and accuracy requirements, and unique-hot encoding the target function, normalizing the image data and image information data of the electromechanical component to be integrated, and splicing them in the order of the integration requirement input data, the image data and image information data of the electromechanical component to be integrated to form an N-dimensional input vector as the input of the convolutional neural network.

[0057] Preferably, the trained convolutional neural network outputs the name and position of the component that currently needs to be integrated, including: S41: the convolutional neural network receives an N-dimensional input vector; S42: extracting local features in the input vector through the convolutional layer of the convolutional neural network, including the shape, edge, grayscale distribution and functional priority of the component; S43: classifying the extracted local features through the fully connected layer, and outputting the name and category label of the component that currently needs to be integrated; S44: performing position regression on the image features of the component through the regression layer, and predicting the center point coordinates or bounding box position of the component in the image; S45: based on the results of classification and regression, outputting the name of the component that currently needs to be integrated and its corresponding position coordinates, and passing the output data to the PLC for execution control.

[0058] Preferably, the trained convolutional neural network is expressed as follows using an improved activation function f(x):

[0059]

[0060] Among them, x represents the input value of the current neuron, e is the base of the natural logarithm, and N is the number of component sub-images.

[0061] Preferably, the PLC control system controls the manipulator and the transmission device to automatically integrate components according to the name and position of the component to be integrated, including: S51: the output of the convolutional neural network sends the component name and position data directly to the PLC through the Modbus protocol; S52: the PLC parses the received component name and maps it to the internal function module register; S53: the PLC parses the position data, converts it into three-dimensional coordinates and generates motion trajectory instructions for the manipulator; S54: the PLC controls the manipulator and the transmission device to perform grasping and installation tasks according to the generated motion trajectory instructions.

[0062] The present application also provides an automated integration system for prefabricated electromechanical components, such as Figure 2As shown in the figure, each hardware module is connected through a standard interface or protocol to collaboratively complete the automated integration task of the electromechanical prefabricated components. The user terminal receives the assembly requirements (such as target function, time, accuracy, etc.) input by the user. Touch screen industrial computer or mobile terminal. Ethernet interface or wireless communication module for connecting to the main control system. High-definition camera collects real-time images of the components to be assembled, industrial-grade high-definition camera (resolution 1920×1080). The interface is connected to the image processing module via USB or GigE protocol. The sensor module includes a position sensor to detect the installation position of the component. The torque sensor monitors the gripping force and installation contact force of the manipulator. The visual sensor verifies the assembly status of the component and is connected to the PLC through analog signals or Modbus protocol. The PLC controller is a central control unit that coordinates the operation of each module Programmable logic controller (such as Siemens S7 series). The interface is connected to the manipulator and transmission device through Modbus or Ethernet. Real-time communication with the main control system and sensor module. The manipulator realizes the gripping, handling and precise installation of components. Multi-degree-of-freedom industrial robot arm (such as ABB IRB series), the interface is controlled by PLC and receives motion instructions. The transmission device is used to move components from one workstation to another. The roller conveyor or robot mobile platform interface receives control signals for the transmission path and speed through the PLC. The main control system manages the entire assembly process, integrating user requirements, image data, sensor feedback, and PLC control instructions. The industrial computer or server interface is connected to the user terminal, PLC, and component feature database via Ethernet. The component feature database stores the feature information of the component (such as size, grayscale, position, etc.). The server or embedded database system interface interacts with other modules through the main control system. The connection between the hardware, user terminal → main control system: transmit the assembly requirements input by the user through Ethernet or wireless communication module. HD camera → image processing module: transmit the collected image data in real time through USB or GigE interface. Sensor module → PLC controller: transmit monitoring data (such as position, force) in real time through analog signal interface or Modbus protocol. PLC controller → manipulator / transmission device: send control instructions and receive feedback through Ethernet or Modbus communication. Main control system → component feature database: query or update component feature information through standard database protocols (such as SQL). User demand input and analysis: The user inputs assembly requirements through the terminal (such as target function: motor drive module, time requirement: 2 hours, accuracy requirement: error ≤ 0.1 mm). The main control system receives the requirements and generates a task plan. Image acquisition and processing: The high-definition camera captures the image of the component to be assembled and sends the image to the image processing module. The image processing module segments the image, extracts sub-component features, and stores them in the component feature database. N-dimensional input vector generation: The main control system integrates user input, image features, and database features to generate an N-dimensional input vector.The input vector is passed into the trained convolutional neural network, and the name and position of the current component to be assembled are output. PLC coordination and control: PLC receives the component name and position data output by CNN: generates the motion path of the robot to complete the grasping and installation of the component. Control the transmission device to transport the component to the specified location. The sensor monitors the grasping force and position accuracy in real time, and transmits the feedback data back to the PLC to adjust the action. Closed-loop feedback and assembly confirmation: The visual sensor confirms the installation status of the component: If the installation error exceeds the set range, the PLC triggers the recalibration process. After the installation is completed, the PLC sends a completion signal to the user terminal through the main control system. The cyclic task execution system returns to the initial step and processes the next task until all components are assembled. Each module is connected through a standard interface, which has good scalability and compatibility. Real-time response and efficient operation: Use the real-time control of the PLC and the data transmission of the Modbus protocol to ensure that the robot and the transmission device work together. High-precision assembly: Sensor feedback and visual calibration form a closed-loop control, and the error is controlled within 0.1 mm.

[0063] Demand input module: receiving integrated demand input through a user terminal, wherein the integrated demand includes integrated assembly target function, assembly time requirement and assembly accuracy requirement of the electromechanical prefabricated component;

[0064] Image processing module: collects images of the electromechanical components to be integrated in real time through a high-definition camera, segments the images to obtain sub-images of each component, and calculates the image information of each sub-image, including grayscale mean, maximum value, number of pixels and pixel position information, and stores the image information in the component feature database;

[0065] Vector generation module: splices the pre-processed integration requirement input data, the image data of the electromechanical components to be integrated, and the image information data in the feature database to form an N-dimensional input vector;

[0066] Convolutional neural network calculation module: inputs the N-dimensional input vector into the trained convolutional neural network, and the trained convolutional neural network outputs the name and position of the current component to be integrated;

[0067] Execution module: The PLC control system controls the manipulator and the transmission device to automatically integrate the components according to the names and positions of the components to be integrated, and transmits information through the real-time communication protocol Modbus to ensure accurate instruction transmission;

[0068] Loop judgment module: return to the image processing module until the assembly is completed.

[0069] Preferably, the integrated assembly target functions include: assembly of motor drive modules, modular integration of automation control cabinets or installation of end-of-arm tools; the assembly time requirements include: completion time limit for assembly tasks, in hours or minutes, to be completed within 2 hours, 5 hours or 10 hours; the assembly accuracy requirements include: error not exceeding 0.1 mm or error not exceeding 2 cm.

[0070] Preferably, segmenting the image to obtain each component sub-image includes denoising the image using Gaussian filtering, extracting component boundaries using a Canny operator edge detection algorithm, and then segmenting the image to obtain the component sub-images.

[0071] The present invention provides an automated integration method and system for electromechanical prefabricated components, which can achieve the following beneficial technical effects:

[0072] 1. The present invention first receives the integration demand input through the user terminal; secondly, the image of the electromechanical component to be integrated is collected in real time through the CCD camera, and the image information of each component sub-image is calculated; then, the pre-processed integration demand input data, the image data of the electromechanical component to be integrated and the image information data in the feature database are spliced ​​to form an N-dimensional input vector; again, the N-dimensional input vector is input into the trained convolutional neural network, and the trained convolutional neural network outputs the name and position of the current component to be integrated; again, the manipulator and the transmission device are controlled by the PLC control system to automatically integrate the components according to the name and position of the current component to be integrated; finally, the steps are returned and repeated until the assembly is completed. This application realizes the precise selection of components by adopting the convolutional neural network algorithm, and uses the PLC control system to greatly improve the accuracy and efficiency of the automated integration of electromechanical prefabricated components. This application receives the integration demand input through the user terminal, including the assembly target function, time requirements and accuracy requirements, to achieve dynamic adaptation to different task scenarios. Preprocessing and unique hot encoding technology are used to convert complex requirements into standardized data that can be used for neural network calculation, effectively improving the flexibility of the system and meeting the needs of large-scale industrial production and small-batch customized production.

[0073] 2. The present invention collects component images through a high-definition camera, and uses Gaussian filtering and Canny edge detection algorithms to segment the images, significantly reducing the interference of complex backgrounds on component recognition. The deep features in the component images are extracted using a convolutional neural network, and the names of the components that need to be integrated are classified and output. At the same time, the installation positions of the components are predicted through position regression to achieve high-precision recognition of complex components. The introduction of dynamic activation functions (with the number of component sub-images as dynamic parameters) enhances the adaptability of neural networks to tasks of different complexities. By taking the number of component sub-images as a consideration and adding it to the recognition calculation process of the convolutional neural network, the recognition accuracy is further improved.

[0074] 3. The present invention accurately controls the motion trajectory of the manipulator and the transmission device through the PLC control system, and ensures that the components can be accurately installed according to the design requirements according to the component names and position data output by the CNN. The real-time communication protocol Modbus is used to realize data transmission between the neural network, PLC and manipulator controller, ensuring the real-time and accuracy of instruction transmission, and effectively avoiding assembly errors caused by data delays. During the assembly process, a variety of sensors (such as position sensors, torque sensors and visual sensors) are used to monitor the installation status of the components in real time to determine whether there are problems such as position deviation, excessive force or wrong direction. Combined with the sensor monitoring results and the real-time feedback mechanism of the convolutional neural network, a closed-loop control system is formed to dynamically adjust the manipulator operation in abnormal situations until the component installation is completed. Through the cyclic judgment and feedback optimization of monitoring data, the fault tolerance and operational stability of the system are significantly improved.

[0075] 4. The present invention efficiently integrates demand data, image features and component information through the generation of N-dimensional input vectors, reduces data redundancy and improves computing efficiency. The system adopts a modular design, including a demand input module, an image processing module, a vector generation module, a convolutional neural network calculation module, an execution module and a loop judgment module. Each module works together to improve the overall operating efficiency. The automated processing of the assembly process significantly reduces the frequency of manual intervention, shortens the assembly time, and meets the needs of efficient production.

[0076] The above is a detailed introduction to an automated integration method and system for prefabricated electromechanical components. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the core idea of ​​the present invention. At the same time, for those skilled in the art, according to the ideas and methods of the present invention, there will be changes in the specific implementation methods and application scopes. In summary, the content of this specification should not be understood as limiting the present invention.

Claims

1. A method for automated integration of electromechanical prefabricated components, characterized in that: Includes steps: S1: receiving an integration requirement input through a user terminal, wherein the integration requirement includes an integrated assembly target function, an assembly time requirement, and an assembly accuracy requirement of a prefabricated electromechanical component; S2: The image of the electromechanical component to be integrated is collected in real time by a CCD camera, the image is segmented to obtain each component sub-image, and the image information of each sub-image is calculated. The image information includes grayscale mean, maximum value, number of pixels and pixel position information, and the image information is stored in the component feature database; S3: splicing the preprocessed integration requirement input data, the image data of the electromechanical component to be integrated, and the image information data in the feature database to form an N-dimensional input vector; S4: Input the N-dimensional input vector into the trained convolutional neural network, and the trained convolutional neural network outputs the name and position of the component to be integrated; S5: The PLC control system controls the manipulator and the transmission device to automatically integrate the components according to the names and positions of the components to be integrated, and transmits information through the real-time communication protocol Modbus to ensure accurate instruction transmission; S6: Return to step S2 until the assembly is completed.

2. The method for automated integration of electromechanical prefabricated components according to claim 1, characterized in that: The integrated assembly target functions include: assembly of motor drive modules, modular integration of automation control cabinets or installation of end-of-arm tools; the assembly time requirements include: completion time limit for assembly tasks, in hours or minutes, to be completed within 2 hours, 5 hours or 10 hours; the assembly accuracy requirements include: error not exceeding 0.1 mm or error not exceeding 2 cm.

3. The method for automated integration of electromechanical prefabricated components according to claim 2, characterized in that: The image is segmented to obtain each component sub-image, including performing denoising processing on the image using Gaussian filtering, extracting component boundaries using a Canny operator edge detection algorithm, and then segmenting the image to obtain the component sub-images.

4. The method for automated integration of electromechanical prefabricated components according to claim 1, characterized in that: The preprocessed integration requirement input data, the image data of the electromechanical component to be integrated, and the image information data in the feature database are spliced ​​to form an N-dimensional input vector, including: preprocessing the integration requirement input data including normalizing the time requirements, cost budget and accuracy requirements, and unique-hot encoding the target function, normalizing the image data and image information data of the electromechanical component to be integrated, and splicing them in the order of the integration requirement input data, the image data and image information data of the electromechanical component to be integrated to form an N-dimensional input vector as the input of the convolutional neural network.

5. The method for automated integration of electromechanical prefabricated components according to claim 1, characterized in that: The trained convolutional neural network outputs the name and position of the current component to be integrated, including: S41: the convolutional neural network receives an N-dimensional input vector; S42: extracting local features in the input vector through the convolutional layer of the convolutional neural network, including the shape, edge, grayscale distribution and functional priority of the component; S43: classifying the extracted local features through the fully connected layer, and outputting the name and category label of the current component to be integrated; S44: performing position regression on the image features of the component through the regression layer, and predicting the center point coordinates or bounding box position of the component in the image; S45: based on the results of classification and regression, outputting the name of the current component to be integrated and its corresponding position coordinates, and passing the output data to the PLC for execution control.

6. The method for automated integration of electromechanical prefabricated components according to claim 1, characterized in that: The activation function f(x) used by the trained convolutional neural network is expressed as follows: Among them, x represents the input value of the current neuron, e is the base of the natural logarithm, and N is the number of component sub-images.

7. The method for automated integration of electromechanical prefabricated components according to claim 1, characterized in that: The method of controlling the manipulator and the transmission device to automatically integrate components according to the names and positions of the components to be integrated currently through the PLC control system includes: S51: the output of the convolutional neural network directly sends the component name and position data to the PLC through the Modbus protocol; S52: the PLC parses the received component name and maps it to the internal function module register; S53: the PLC parses the position data, converts it into three-dimensional coordinates and generates motion trajectory instructions for the manipulator; S54: the PLC controls the manipulator and the transmission device to perform grasping and installation tasks according to the generated motion trajectory instructions.

8. An automated integration system for prefabricated electromechanical components, characterized in that: include: Demand input module: receiving integrated demand input through a user terminal, wherein the integrated demand includes integrated assembly target function, assembly time requirement and assembly accuracy requirement of the electromechanical prefabricated component; Image processing module: collects images of the electromechanical components to be integrated in real time through a CCD camera, segments the images to obtain sub-images of each component, and calculates the image information of each sub-image, including grayscale mean, maximum value, number of pixels and pixel position information, and stores the image information in the component feature database; Vector generation module: splices the pre-processed integration requirement input data, the image data of the electromechanical components to be integrated, and the image information data in the feature database to form an N-dimensional input vector; Convolutional neural network calculation module: inputs the N-dimensional input vector into the trained convolutional neural network, and the trained convolutional neural network outputs the name and position of the current component to be integrated; Execution module: The PLC control system controls the manipulator and the transmission device to automatically integrate the components according to the names and positions of the components to be integrated, and transmits information through the real-time communication protocol Modbus to ensure accurate instruction transmission; Loop judgment module: return to the image processing module until the assembly is completed.

9. The electromechanical prefabrication assembly automation integration system according to claim 8, characterized in that: The integrated assembly target functions include: assembly of motor drive modules, modular integration of automation control cabinets or installation of end-of-arm tools; the assembly time requirements include: completion time limit for assembly tasks, in hours or minutes, to be completed within 2 hours, 5 hours or 10 hours; the assembly accuracy requirements include: error not exceeding 0.1 mm or error not exceeding 2 cm.

10. The electromechanical prefabrication assembly automation integration system according to claim 8, characterized in that: The image is segmented to obtain each component sub-image, including performing denoising processing on the image using Gaussian filtering, extracting component boundaries using a Canny operator edge detection algorithm, and then segmenting the image to obtain the component sub-images.

Images