Household board carrying method and system based on machine vision

Abstract

The invention relates to a household board carrying method and system based on machine vision, and the method comprises the steps: obtaining a to-be-classified household board image, inputting the to-be-classified household board image into a household board recognition model, recognizing the type of a household board in the to-be-classified household board image, and marking the household board in the image; wherein the home panel identification model is obtained by training a deep learning model by using a training set; the training set comprises home panel images; carrying out corner detection on the marked home panel image, obtaining the geometric center of the home panel by using the detected corner, and obtaining the weight of the home panel by combining the density of the home panel; and determining a carrying strategy according to the category, geometric center and weight of the home panel, and controlling the mechanical arm to carry the home panel based on the carrying strategy. According to the invention, different coping strategies can be adaptively adopted, so that the flexibility and universality of the system are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of machine vision, and in particular to a method and system for conveying home furnishing panels based on machine vision. Background Art

[0002] Machine vision is an important branch of artificial intelligence, which uses computer vision technology to enable machines to perceive and understand the physical world. Machine vision collects image data through image sensors and uses computer vision technology to analyze and process it, thereby realizing functions such as recognition, detection, and measurement of target objects and scenes. Using machine vision to guide the movement of robotic arms is an important application.

[0003] In the field of home furnishing boards, the processing of finished boards mainly includes the steps of cutting, pressing, edge banding, and hole arrangement. Pressing refers to pasting decorative paper, PVC film, or other finishing materials on the surface of the board, and then putting it into a hot press to firmly adhere the finishing materials to the board under high temperature and high pressure. After pressing different materials, the board can present different textures, colors, surface roughness, etc.

[0004] In recent years, robotic arms have been widely used in manufacturing, especially in the field of object handling. Depending on the different clamps used by the robotic arm, they can be mainly divided into "clamping type" and "adsorption type". The handling of home panels is mainly achieved through "adsorption type" vacuum suction cups. The entire system is mainly composed of a robotic arm, a PLC (Programmable Logic Controller), a vacuum suction cup, an air source device, etc. Traditional handling tasks are mainly achieved through the teaching programming of the robotic arm, and each step of the action is controlled by the program, but this method is too rigid and has low flexibility and cannot cope with changing scenarios.

[0005] The existing technical solution proposes a design method for an industrial robot positioning and grasping system based on vision guidance, which enhances the flexibility of the robot arm's grasping by introducing a visual sensor. However, the image processing algorithm is relatively simple and has poor versatility for workpieces of different specifications and categories. For example, when the size, texture, posture, surface roughness, etc. of home furnishing boards are different, it is easy for the suction force to be too large or too small to cause damage to the board or grasping failure. The processing means of this method are relatively single, the success rate of handling is not high, and it is difficult to achieve strong versatility. Summary of the invention

[0006] In order to solve the problems existing in the above-mentioned prior art, the purpose of the present invention is to provide a method and system for handling home furnishing panels based on machine vision, which can adaptively adopt different coping strategies when processing home furnishing panels of different sizes, textures, postures, and surface roughness, so as to improve the flexibility and versatility of the system.

[0007] To achieve the above object, the present invention provides the following solutions:

[0008] A method for transporting home panels based on machine vision, comprising:

[0009] Acquire a furniture board image to be classified, input the furniture board image to be classified into a furniture board recognition model to identify the furniture board category in the furniture board image to be classified, and mark the furniture board in the image; wherein the furniture board recognition model is obtained by training a deep learning model using a training set; the training set includes: a furniture board image;

[0010] Performing corner point detection on the annotated furniture board image, using the detected corner points to obtain the geometric center of the furniture board, and combining the density of the furniture board to obtain the weight of the furniture board;

[0011] A transport strategy is determined according to the type, geometric center and weight of the home furnishing board, and the robotic arm is controlled to transport the home furnishing board based on the transport strategy.

[0012] Optionally, obtaining the furniture board image to be classified includes:

[0013] The industrial camera is calibrated and distortion corrected, and the image of the furniture board to be classified is obtained using the corrected industrial camera.

[0014] Optionally, performing corner point detection on the annotated furniture board image includes:

[0015] Obtaining a corner point response value of each pixel in the annotated home board image, performing non-maximum suppression on the corner point response value, and retaining a local maximum point;

[0016] Setting a corner point response threshold, marking the corner point response value of the local maximum point higher than the corner point response threshold as a corner point;

[0017] Perform sub-pixel level refinement processing on the corner points to obtain the refined corner points.

[0018] Optionally, obtaining the geometric center of the home furnishing board includes:

[0019] Using the horizontal coordinates of the detected corner points, obtain the horizontal coordinates of the geometric center of the home board:

[0020]

[0021] Among them, x cis the abscissa of the geometric center of the home board, x1 is the abscissa of the first corner point, x2 is the abscissa of the second corner point, x3 is the abscissa of the third corner point, and x4 is the abscissa of the fourth corner point;

[0022] Using the ordinate of the detected corner point, obtain the ordinate of the geometric center of the home board:

[0023]

[0024] Among them, y c is the ordinate of the geometric center of the home board, y1 is the ordinate of the first corner point, y2 is the ordinate of the second corner point, y3 is the ordinate of the third corner point, and y4 is the ordinate of the fourth corner point.

[0025] Optionally, obtaining the weight of the home furnishing board includes:

[0026] Determine the shooting focal length of the industrial camera, and use the distance between the corner points combined with the shooting focal length to obtain the pixel length, pixel width, and pixel thickness of the home board;

[0027] Determine the distance between the industrial camera and the home furnishing board, and obtain the actual length, actual width and actual thickness of the home furnishing board according to the shooting focal length, the pixel length, the pixel width, the pixel thickness and the distance;

[0028] The volume of the home furnishing board is obtained according to the actual length, the actual width and the actual thickness, and the weight of the home furnishing board is obtained based on the volume of the home furnishing board and the density of the home furnishing board.

[0029] Optionally, the method for obtaining the actual length, actual width and actual thickness of the home furnishing board is:

[0030]

[0031] Among them, l is the actual length, L is the pixel length, d is the distance between the object and the camera, f is the shooting focal length, w is the actual width, W is the pixel width, h is the actual thickness, and H is the pixel thickness.

[0032] Optionally, the method for obtaining the weight of the home furnishing board is:

[0033] M=l*w*h*ρ

[0034] Among them, M is the weight, l is the actual length, w is the actual width, h is the actual thickness, and ρ is the density.

[0035] Optionally, determining the transport strategy includes:

[0036] Based on the category of the home furnishing board, the surface roughness of the home furnishing board is determined. According to the surface roughness and weight of the home furnishing board, the number of working suction cups and the vacuum degree of the suction cups in the suction cup array in the robotic arm are determined. At the same time, according to the geometric center of the home furnishing board, the robotic arm is used to grab the home furnishing board.

[0037] To achieve the above purpose, the present invention also provides a furniture board handling system based on machine vision, comprising:

[0038] An image acquisition module, used to acquire images of home furnishing panels to be classified;

[0039] The target detection module is used to input the furniture board image to be classified into the furniture board recognition model to identify the furniture board category in the furniture board image to be classified, and mark the furniture boards in the image; wherein the furniture board recognition model is obtained by training a deep learning model using a training set; the training set includes: furniture board images;

[0040] An image analysis module is used to detect corner points of the annotated furniture board image, obtain the geometric center of the furniture board using the detected corner points, and obtain the weight of the furniture board in combination with the density of the furniture board;

[0041] The execution module is used to determine a transport strategy according to the type, geometric center and weight of the home furnishing board, and control the robot arm to transport the home furnishing board based on the transport strategy.

[0042] The beneficial effects of the present invention are:

[0043] The present invention can automatically classify the categories of household boards, which is convenient for subsequent targeted measures, and measures the size and geometric center of the household boards in a non-contact manner, and then infers the weight of the household boards, and guides the robot arm to achieve automatic grasping; the present invention can accurately control the suction cup array according to the size and category of the household boards, and while improving energy utilization, it can also avoid damage to the boards or grasping failures caused by excessive or insufficient suction, thereby maximizing benefits; the present invention can control the suction cup array according to the size and category of the household boards, and one system can realize the transportation of household boards of different specifications, thereby improving the versatility and flexibility of the entire transportation system.

[0044] The invention is helpful for the automation upgrade of the home board manufacturing industry, conforms to the trend of Industry 4.0 and intelligent manufacturing, and helps to promote the technological progress of the entire industry and the development of social productivity. The automated handling system reduces the need for manual handling of boards, reduces the physical labor intensity of workers, and improves work safety.

[0045] The present invention can improve production efficiency, reduce sorting and handling time, and thus speed up the production line. By reducing manual operations and reducing labor costs, precise control of the suction cup array can avoid damage to the board caused by improper suction, reduce material waste, and further reduce production costs. The handling system provided by the present invention can be applied to home furnishing boards of different sizes, textures, postures, and surface roughness, reducing the need to customize specific handling tools and saving additional investment costs.

[0046] The present invention can automatically measure the size and estimate the weight of household boards through machine vision technology, which provides key information for accurately controlling the suction cup array. According to the type, weight and geometric center of the household board, the system can intelligently adjust the working state of the suction cup array to ensure stability and safety during the transportation process. Since the system can adaptively adjust the transportation strategy according to the different properties of the household board, a set of systems can realize the transportation of boards of different specifications, which improves the versatility and flexibility of the system. In addition, by accurately controlling the number of working suction cups and the vacuum degree of the suction cup array, unnecessary energy consumption is avoided and energy utilization is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0048] Figure 1 This is a flow chart of a method for transporting furniture panels based on machine vision according to an embodiment of the present invention;

[0049] Figure 2 A schematic diagram of a furniture board handling system based on machine vision according to an embodiment of the present invention;

[0050] Figure 3 It is a schematic diagram of the suction cup array structure of an embodiment of the present invention; wherein, 1-suction cup base, 2-cross beam, 3-flange bottom plate, 4-flange, 5-vertical rod, 6-suction cup. DETAILED DESCRIPTION

[0051] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0052] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0053] like Figure 1 As shown, this embodiment discloses a method for transporting household boards based on machine vision, including: obtaining images of household boards to be classified, inputting the images of household boards to be classified into a household board recognition model to identify the category of household boards in the images of household boards to be classified, and marking the household boards in the image; wherein the household board recognition model is obtained by training a deep learning model with a training set; the training set includes: the household board image; performing corner point detection on the labeled household board image, using the detected corner points to obtain the geometric center of the household board, and combining the density of the household board to obtain the weight of the household board; determining a transport strategy according to the category, geometric center and weight of the household board, and controlling the robotic arm to transport the household board based on the transport strategy.

[0054] Specifically, a home board image dataset is obtained through an industrial camera, and a deep learning model is trained using the home board image dataset to obtain a home board recognition model. The home board image is photographed by an industrial camera deployed on the production line, and the home board recognition model is used to identify the home board category and mark the home board in the image. The marked image is detected for corner points, and the weight of the home board is calculated by combining the corner points with the density. The geometric center of the home board is calculated using the corner points, and different handling strategies are adopted according to the category of the home board, the weight of the home board, and the geometric center of the home board.

[0055] Among them, the deep learning model is YOLOv5, and the industrial camera is a monocular industrial camera.

[0056] Furthermore, obtaining the images of the home furnishing boards to be classified includes: calibrating and correcting the distortion of the industrial camera, and using the corrected industrial camera to obtain the images of the home furnishing boards to be classified.

[0057] Specifically, before taking an image with an industrial camera, the method further includes: using OpenCV to calibrate and correct distortion of the industrial camera.

[0058] Specifically, the home board material recognition model is used to identify and mark the home board materials in the image, including: using the home board material recognition model to identify the home board materials in the image, marking the home board materials in the image in the form of a rectangular frame, and using the rectangular frame detected by YOLO to segment the area of ​​interest.

[0059] Furthermore, corner point detection is performed on the annotated home board image, including: obtaining a corner point response value for each pixel in the annotated home board image, performing non-maximum suppression on the corner point response value, and retaining local maximum points; setting a corner point response threshold, and marking the local maximum points with corner point response values ​​higher than the corner point response threshold as corner points; performing sub-pixel level refinement on the corner points, and obtaining the refined corner points.

[0060] Specifically, the annotated image is used for corner detection, including: using the Harris corner detection algorithm to calculate the corner response value of each pixel; performing non-maximum suppression on the corner response value, suppressing points with smaller response values ​​in the local neighborhood, and retaining local maximum points; setting a suitable threshold, marking points with corner response values ​​higher than the threshold as corner points; and using the Subpixel Interpolation method to refine the detected corner points at the sub-pixel level to obtain more accurate corner coordinates.

[0061] Furthermore, obtaining the geometric center of the home board includes:

[0062] Using the horizontal coordinates of the detected corner points, obtain the horizontal coordinates of the geometric center of the home board:

[0063]

[0064] Among them, x c is the abscissa of the geometric center of the home board, x1 is the abscissa of the first corner point, x2 is the abscissa of the second corner point, x3 is the abscissa of the third corner point, and x4 is the abscissa of the fourth corner point;

[0065] Using the ordinate of the detected corner point, obtain the ordinate of the geometric center of the home board:

[0066]

[0067] Among them, y c is the ordinate of the geometric center of the home board, y1 is the ordinate of the first corner point, y2 is the ordinate of the second corner point, y3 is the ordinate of the third corner point, and y4 is the ordinate of the fourth corner point.

[0068] Specifically, the geometric center of the furniture board is calculated using the corner points, including: summing the horizontal coordinates of the four corner points of the furniture board, dividing the sum of the horizontal coordinates of the four corner points of the furniture board by four, and obtaining the horizontal coordinate of the geometric center of the furniture board; summing the vertical coordinates of the four corner points of the furniture board, dividing the sum of the vertical coordinates of the four corner points of the furniture board by four, and obtaining the vertical coordinate of the geometric center of the furniture board.

[0069] Furthermore, obtaining the weight of the household board includes: determining the shooting focal length of the industrial camera, and using the distance between the corner points in combination with the shooting focal length to obtain the pixel length, pixel width, and pixel thickness of the household board; determining the distance between the industrial camera and the household board, and obtaining the actual length, actual width, and actual thickness of the household board based on the shooting focal length, pixel length, pixel width, pixel thickness, and distance; obtaining the volume of the household board based on the actual length, actual width, and actual thickness, and obtaining the weight of the household board based on the volume of the household board in combination with the density of the household board.

[0070] Furthermore, the method for obtaining the actual length, actual width and actual thickness of the furniture board is:

[0071]

[0072] Among them, l is the actual length, L is the pixel length, d is the distance between the object and the camera, f is the shooting focal length, w is the actual width, W is the pixel width, h is the actual thickness, and H is the pixel thickness.

[0073] Furthermore, the method for obtaining the weight of the home furnishing board is:

[0074] M=l*w*h*ρ

[0075] Among them, M is the weight, l is the actual length, w is the actual width, h is the actual thickness, and ρ is the density.

[0076] Specifically, the weight of the home furnishing board is calculated by combining the corner points with the density, including: determining the shooting focal length of the industrial camera, determining the pixel length, pixel width, and pixel thickness of the home furnishing board using the distance formula between two points, determining the distance between the object and the camera, and calculating the actual length, actual width, and actual thickness of the home furnishing board according to the shooting focal length, the distance between the object and the camera, the pixel length, pixel width, and pixel thickness; calculating the volume according to the actual length, actual width, and actual thickness; and obtaining the weight of the home furnishing board by multiplying the calculated volume by the density.

[0077] The actual length, actual width and actual thickness of the home furnishing board are calculated according to the shooting focal length, the distance between the object and the camera, the pixel length, the pixel width and the pixel thickness, including: multiplying the distance between the object and the camera and the pixel length, dividing the product of the distance between the object and the camera and the pixel length by the shooting focal length to obtain the actual length of the home furnishing board; dividing the product of the distance between the object and the camera and the pixel width by the shooting focal length to obtain the actual width of the home furnishing board; dividing the product of the distance between the object and the camera and the pixel thickness by the shooting focal length to obtain the actual thickness of the home furnishing board.

[0078] Furthermore, determining the handling strategy includes: determining the surface roughness of the home board based on the category of the home board, determining the number of working suction cups and the vacuum degree of the suction cups in the suction cup array in the robotic arm through the surface roughness and weight of the home board, and at the same time, using the robotic arm to grab the home board according to the geometric center of the home board.

[0079] Specifically, different handling strategies are adopted according to the category of home furnishing board, the weight of home furnishing board and the geometric center of home furnishing board, including: different categories of home furnishing board correspond to different surface roughness, which also affects the number of working suction cups and the vacuum degree of suction cups of the suction cup array; different weights of home furnishing board also affect the number of working suction cups and the vacuum degree of suction cups of the suction cup array; different geometric centers of home furnishing board correspond to different postures and sizes, and the robot arm performs precise grasping according to the geometric center of the home furnishing board; in some embodiments, the number of working suction cups of the suction cup array is achieved by a solenoid valve controlled by a programmable logic controller (PLC); in some embodiments, the vacuum degree of the suction cup of the suction cup array is achieved by a vacuum pressure regulating valve controlled by a PLC, and the suction cup array includes: a suction cup base 1, a crossbeam 2, a flange bottom plate 3, a flange 4, a vertical rod 5, and a suction cup 6, such as Figure 3 shown.

[0080] like Figure 2 As shown, the present embodiment also discloses a household board handling system based on machine vision, including: an image acquisition module, used to obtain household board images to be classified; a target detection module, used to input the household board images to be classified into a household board recognition model to identify the household board categories in the household board images to be classified, and mark the household boards in the image; wherein the household board recognition model is obtained by training a deep learning model using a training set; the training set includes: household board images; an image analysis module, used to perform corner point detection on the labeled household board images, use the detected corner points to obtain the geometric center of the household board, and further obtain the weight of the household board; an execution module, used to determine a handling strategy according to the category, geometric center and weight of the household board, and control the robotic arm to carry the household board based on the handling strategy.

[0081] Specifically, the model training module is used to perform deep learning model training through the home board image data set, so as to obtain the home board recognition model; the image acquisition module is used to take images of home board through an industrial camera; the target detection module is used to identify the home board category through the home board recognition model and mark the home board in the image; the image processing module is used to perform corner point detection using the marked image; the image analysis module is used to calculate the weight of the home board through the corner points combined with the density, and to calculate the geometric center of the home board using the corner points; the communication module is used for data transmission between the industrial camera, the robotic arm, and the PLC; the execution module is used to execute the control instructions from the robotic arm controller and the PLC, and is mainly composed of the robotic arm body, the vacuum suction cup, etc.

[0082] This embodiment provides a method for transporting furniture panels based on machine vision, and the specific process includes:

[0083] Use industrial cameras to obtain a home board image dataset, use the home board image dataset to train a deep learning model, and obtain a home board recognition model;

[0084] In this embodiment, the deep learning model is YOLOv5, and the industrial camera is a monocular industrial camera. The specific training process includes: data preparation, data preprocessing, configuration model, training model, model evaluation and adjustment. Data preparation includes data collection, data labeling, data partitioning, data enhancement, etc. Data preprocessing includes data format conversion, configuration file creation, etc. Configuration model includes editing configuration files, adjusting hyperparameters, etc. Training model includes importing pre-trained weights, monitoring training progress, etc. Model evaluation and adjustment includes evaluating model performance on the validation set, performing multiple iterations, etc.

[0085] The images of home furnishing panels are captured by industrial cameras deployed on the production line;

[0086] In this embodiment, before taking an image with an industrial camera, it also includes: using OpenCV to calibrate and correct the distortion of the industrial camera. The industrial camera is calibrated to obtain the internal and external parameters of the camera, thereby completing the conversion between the image pixel coordinate system, the camera coordinate system, and the world coordinate system.

[0087] Use the home board recognition model to identify the home board category and label the home board in the image;

[0088] In this embodiment, the furniture board recognition model is used to identify the furniture boards in the image and determine the category of each furniture board. The furniture boards in the image are marked in the form of rectangular frames. The rectangular frames detected by YOLO are used to segment the region of interest, thereby preparing for the subsequent corner point detection.

[0089] Perform corner detection on the annotated image;

[0090] In this embodiment, the Harris corner detection algorithm is used to calculate the corner point response value of each pixel; the corner point response value is non-maximum suppressed to suppress points with smaller response values ​​in the local neighborhood and retain local maximum points; a suitable threshold (128) is set to mark points with corner point response values ​​higher than the threshold as corner points; and the Subpixel Interpolation method is used to refine the detected corner points at the sub-pixel level to obtain more accurate corner point coordinates.

[0091] Calculate the weight of furniture panels by combining corner points with density;

[0092] In this embodiment, the shooting focal length f of the industrial camera is determined. The scale coordinates of the corner point coordinates are converted into pixel coordinates. Assuming that the image size is X*Y pixels, a certain point is (x, y), the pixel horizontal coordinate is X*x, and the pixel vertical coordinate is Y*y. Use the distance formula between two points:

[0093]

[0094] Determine the pixel length L, pixel width W, and pixel thickness H of the home board. Determine the distance d between the object and the camera. The distance between the object and the camera is fixed in the production line. Based on the principle of pinhole imaging:

[0095] S=s*(d / f)

[0096] Among them, S is the actual distance of the length, width and height of the plate, and s is the pixel distance of the length, width and height of the plate in the pixel coordinate system.

[0097] The actual length l, actual width w, and actual thickness h of the furniture board are calculated based on the shooting focal length f, the distance d between the object and the camera, the pixel length, pixel width, and pixel thickness. The volume V is calculated based on the actual length, actual width, and actual thickness. The weight M of the furniture board is obtained by multiplying the calculated volume by the density ρ:

[0098]

[0099] M=l*w*h*ρ

[0100] Among them, l is the actual length, L is the pixel length, d is the distance between the object and the camera, f is the shooting focal length, w is the actual width, W is the pixel width, h is the actual thickness, H is the pixel thickness, and M is the weight.

[0101] Calculate the product of the distance between the object and the camera and the pixel length; divide the product of the distance between the object and the camera and the pixel length by the shooting focal length to obtain the actual length of the home furnishing board; divide the product of the distance between the object and the camera and the pixel width by the shooting focal length to obtain the actual width of the home furnishing board; divide the product of the distance between the object and the camera and the pixel thickness by the shooting focal length to obtain the actual thickness of the home furnishing board.

[0102] Use corner points to calculate the geometric center of furniture panels;

[0103] In this embodiment, the abscissas of the four corner points of the furniture board are summed. The sum of the abscissas of the four corner points of the furniture board is divided by four to obtain the abscissa x of the geometric center of the furniture board. c . Sum the ordinates of the four corners of the furniture board. Divide the sum of the ordinates of the four corners of the furniture board by four to get the ordinate y of the geometric center of the furniture board. c .

[0104]

[0105] Different handling strategies are adopted according to the type of home board, weight of home board and geometric center of home board;

[0106] In this embodiment, different types of home furnishing boards correspond to different surface roughnesses. The surface roughness is evenly distributed on the surface of the home furnishing boards, which also affects the number of working suction cups and the vacuum degree of the suction cups of the suction cup array; the weight of different home furnishing boards also affects the number of working suction cups and the vacuum degree of the suction cups of the suction cup array. The most suitable suction force is distributed according to the weight to prevent damage to the board or failure to grasp due to excessive or insufficient suction. The geometric centers of different home furnishing boards correspond to different postures and sizes. The robotic arm performs precise grasping according to the geometric centers of the home furnishing boards. The number of working suction cups of the suction cup array is achieved by PLC-controlled solenoid valves, which are two-position two-way solenoid reversing valves. The vacuum degree of the suction cups of the suction cup array is achieved by PLC-controlled vacuum air-pressure valves. The entire vacuum system is mainly composed of an air source device, a vacuum generator, an electromagnetic reversing valve, a one-way valve, a vacuum air-pressure valve, pipelines, etc.

[0107] This embodiment provides a furniture board handling system based on machine vision, which specifically includes:

[0108] Model training module: used to train deep learning models using home board image datasets to obtain home board recognition models;

[0109] Image acquisition module: used to capture images of home furnishing panels using an industrial camera;

[0110] Object detection module: used to identify the category of home furnishing boards through the home furnishing board recognition model and mark the home furnishing boards in the image;

[0111] Image processing module: used to detect corner points using the annotated images;

[0112] Image analysis module: used to calculate the weight of furniture boards by combining corner points with density, and to calculate the geometric center of furniture boards by using corner points;

[0113] Communication module: used for data transmission between industrial cameras, robotic arms, and PLCs;

[0114] Execution module: used to execute the control instructions from the robot arm controller and PLC, mainly composed of the robot arm body, solenoid reversing valve, vacuum air pressure valve, vacuum suction cup, etc.

[0115] In this embodiment, the model training module is used to perform deep learning model training through a home board image data set, so as to obtain a home board recognition model, and the deep learning model is YOLOv5. The image acquisition module is used to take images of home board through an industrial camera, and the industrial camera is a monocular industrial camera, and the distance between the camera and the home board is fixed. The target detection module is used to identify the home board category through the home board recognition model and mark the home board in the image, and the marking form is the form of a rectangular frame. The image processing module is used to perform corner point detection using the marked image, and the corner point detection algorithm used is the Harris corner point detection algorithm. The image analysis module is used to calculate the weight of the home board by combining the corner point with the density, and to calculate the geometric center of the home board using the corner point. The communication module is used for data transmission between the industrial camera, the robot arm, and the PLC, and the communication method adopts industrial Ethernet communication. The execution module is used to execute the control instructions from the robot arm controller and the PLC, and is mainly composed of the robot arm body, the electromagnetic reversing valve, the vacuum air pressure valve, the vacuum suction cup, etc.

[0116] The embodiments described above are only descriptions of the preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Without departing from the design spirit of the present invention, various modifications and improvements made to the technical solutions of the present invention by ordinary technicians in this field should all fall within the protection scope determined by the claims of the present invention.

Claims

1. A method for transporting home furnishing panels based on machine vision, characterized in that: include: Acquire a furniture board image to be classified, input the furniture board image to be classified into a furniture board recognition model to identify the furniture board category in the furniture board image to be classified, and mark the furniture board in the image; wherein the furniture board recognition model is obtained by training a deep learning model using a training set; the training set includes: a furniture board image; Performing corner point detection on the annotated furniture board image, using the detected corner points to obtain the geometric center of the furniture board, and combining the density of the furniture board to obtain the weight of the furniture board; A transport strategy is determined according to the type, geometric center and weight of the home furnishing board, and the robotic arm is controlled to transport the home furnishing board based on the transport strategy.

2. The method for transporting furniture panels based on machine vision according to claim 1, characterized in that: Obtaining furniture board images to be classified includes: The industrial camera is calibrated and distortion corrected, and the image of the furniture board to be classified is obtained using the corrected industrial camera.

3. The method for transporting furniture panels based on machine vision according to claim 1, characterized in that: Corner point detection of the annotated furniture board image includes: Obtaining a corner point response value of each pixel in the annotated home board image, performing non-maximum suppression on the corner point response value, and retaining a local maximum point; Setting a corner point response threshold, marking the corner point response value of the local maximum point higher than the corner point response threshold as a corner point; Perform sub-pixel level refinement processing on the corner points to obtain the refined corner points.

4. The method for transporting furniture panels based on machine vision according to claim 1, characterized in that: Obtaining the geometric center of the home board includes: Using the horizontal coordinates of the detected corner points, obtain the horizontal coordinates of the geometric center of the home board: Among them, x c is the abscissa of the geometric center of the home board, x1 is the abscissa of the first corner point, x2 is the abscissa of the second corner point, x3 is the abscissa of the third corner point, and x4 is the abscissa of the fourth corner point; Using the ordinate of the detected corner point, obtain the ordinate of the geometric center of the home board: Among them, y c is the ordinate of the geometric center of the home board, y1 is the ordinate of the first corner point, y2 is the ordinate of the second corner point, y3 is the ordinate of the third corner point, and y4 is the ordinate of the fourth corner point.

5. The method for transporting furniture panels based on machine vision according to claim 2, characterized in that: Obtaining the weight of the home furnishing board includes: Determine the shooting focal length of the industrial camera, and use the distance between the corner points combined with the shooting focal length to obtain the pixel length, pixel width, and pixel thickness of the home board; Determine the distance between the industrial camera and the home furnishing board, and obtain the actual length, actual width and actual thickness of the home furnishing board according to the shooting focal length, the pixel length, the pixel width, the pixel thickness and the distance; The volume of the home furnishing board is obtained according to the actual length, the actual width and the actual thickness, and the weight of the home furnishing board is obtained based on the volume of the home furnishing board and the density of the home furnishing board.

6. The method for transporting furniture panels based on machine vision according to claim 5, characterized in that: The method for obtaining the actual length, actual width and actual thickness of the home furnishing board is: Among them, l is the actual length, L is the pixel length, d is the distance between the object and the camera, f is the shooting focal length, w is the actual width, W is the pixel width, h is the actual thickness, and H is the pixel thickness.

7. The method for transporting furniture panels based on machine vision according to claim 5, characterized in that: The method for obtaining the weight of the home furnishing board is: M=l*w*h*ρ Among them, M is the weight, l is the actual length, w is the actual width, h is the actual thickness, and ρ is the density.

8. The method for transporting furniture panels based on machine vision according to claim 1, characterized in that: Determining the handling strategy includes: Based on the category of the home furnishing board, the surface roughness of the home furnishing board is determined. According to the surface roughness and weight of the home furnishing board, the number of working suction cups and the vacuum degree of the suction cups in the suction cup array in the robotic arm are determined. At the same time, according to the geometric center of the home furnishing board, the robotic arm is used to grab the home furnishing board.

9. A furniture board handling system based on machine vision, characterized in that: include: An image acquisition module, used to acquire images of home furnishing panels to be classified; The target detection module is used to input the furniture board image to be classified into the furniture board recognition model to identify the furniture board category in the furniture board image to be classified, and mark the furniture board in the image; The home board recognition model is obtained by training a deep learning model using a training set; the training set includes: images of home board; An image analysis module is used to detect corner points of the annotated furniture board image, obtain the geometric center of the furniture board using the detected corner points, and obtain the weight of the furniture board in combination with the density of the furniture board; The execution module is used to determine a transport strategy according to the type, geometric center and weight of the home furnishing board, and control the robot arm to transport the home furnishing board based on the transport strategy.

Images