Method and system for achieving accurate positioning and grabbing of automobile side wall outer plate through machine vision

Abstract

The invention provides a method and system for achieving accurate positioning and grabbing of automobile side wall outer panels through machine vision, and the method comprises the steps that whether the number of panels in a storage location target material frame is empty or not is judged; if yes, a worker is dispatched to drive a forklift for feeding; if the number of the plates is not empty, shooting images of the plates in the target material frame through a 3D camera assembled on a gripper; based on a machine vision technology, identifying an identification hole site on the plate image; and based on the recognized hole site, the automobile side wall outer plate is grabbed into a clamp. According to the method and system for achieving accurate positioning and grabbing of the automobile side wall outer plate through machine vision, the automobile side wall outer plate is supplemented in time according to the judgment result of the number of the plates in the target material frame, when the number of the plates is not empty, the plate images shot by the 3D camera are obtained, the machine vision technology is introduced, the recognition hole positions in the plate images are recognized, and the accurate positioning and grabbing of the automobile side wall outer plate are achieved. And a proper grabbing strategy is determined, and the plate is grabbed into the clamp, so that automatic assembly of the automobile is realized, and the assembly efficiency is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of machine vision, and in particular to a method and system for realizing accurate positioning and grasping of a side outer panel of an automobile through machine vision. Background Art

[0002] The side panels of automobiles are the external covering parts of automobile bodies, mainly including engine hoods, doors, trunk lids, roof covers, etc. They are usually stamped from thin steel plates. As a component of automobiles, the side panels of automobiles also need welding, painting and assembly after stamping. Generally, manual work is used in each process to load and unload parts, but manual operation is difficult and the work efficiency is low.

[0003] In view of this, there is an urgent need for a method and system for accurately positioning and grasping the side outer panel of an automobile through machine vision to at least solve the above-mentioned shortcomings. Summary of the invention

[0004] One of the purposes of the present invention is to provide a method and system for accurately positioning and grasping the side outer panels of an automobile through machine vision. According to the judgment result of the number of panels in the target material frame placed in the storage location, it is judged whether the side outer panels of the automobile need to be replenished in time. When the number of panels is not empty, a 3D camera on the grasping device is introduced to shoot the image of the panels in the target material frame, and machine vision technology is introduced to identify the identification hole positions in the panel image, determine the appropriate grasping strategy and grasp the side outer panels of the automobile into the fixture, so as to realize the automatic assembly of the side outer panels of the automobile, improve the assembly efficiency, and be more intelligent.

[0005] The method for accurately positioning and grasping the side outer panel of an automobile provided by an embodiment of the present invention through machine vision includes: Determine whether the number of panels of the automobile side outer panel in the target material frame placed in the storage location is empty; If the number of panels is empty, the dispatcher will drive a forklift to deliver the materials; If the number of panels is not empty, the image of the panel of the automobile side outer panel that has been stamped in the target material frame is captured by a 3D camera pre-installed on the gripper; Based on machine vision technology, the target hole positions on the panel image are identified and the identified hole positions are obtained; Based on the hole position identification, the car side outer panel is grabbed into the fixture.

[0006] Preferably, determining whether the number of panels of the automobile side outer panel in the target material frame placed in the storage location is empty includes: Obtain the frame positioning signal of the storage location, analyze the frame positioning signal, and obtain the panel number recognition result; and / or, Obtain the forklift driver's panel number judgment result.

[0007] Preferably, the image of the stamped automobile side outer panel in the target material frame is captured by a 3D camera pre-assembled on the gripper, including: Get storage location environment data; Initialize 3D camera parameters according to storage environment data; Based on the target detection technology, detect the frame opening of the target material frame and calculate the frame opening coordinates; Calculate the best shooting coordinates according to the frame coordinates; Determine the 3D camera coordinates; According to the 3D camera coordinates and the best shooting coordinates, the 3D camera is adjusted to the best shooting position and the best shooting angle, and the panel image is captured by the adjusted 3D camera.

[0008] Preferably, the best shooting coordinates are calculated according to the frame opening coordinates, including: Get the dynamic coordinates of the 3D camera motion range; Obtain a virtual shooting situation; Construct a virtual shooting scene according to the frame coordinates, the dynamic coordinates of the 3D camera motion range and the virtual shooting situation; Construct a virtual camera object based on the initialized 3D camera parameters and the dynamic coordinates of the 3D camera motion range; Perform random image capture according to a virtual shooting situation and a virtual camera object to obtain a captured image; Extracting image features of the captured image, and inputting the image features into a preset image usability evaluation model to obtain an image usability value; The captured coordinates of the virtual camera object corresponding to the captured image with the largest image available value are obtained, and the captured coordinates are used as the optimal shooting coordinates.

[0009] Preferably, obtaining the virtual shooting situation includes: Obtaining the placement of the automobile side outer panel in the target material frame; Analyze the placement situation and obtain the relative position relationship between each car side outer panel and the target material frame; Get the material frame parameters of the target material frame; Construct a virtual material frame according to the material frame parameters; According to the relative position relationship and the parameters of the car side outer panel, a panel model is constructed in the virtual material frame; A virtual material frame in which a panel model is constructed corresponding to each placement situation is used as a virtual shooting situation.

[0010] Preferably, based on the identification of the hole positions, grabbing the side outer panel of the automobile into the fixture comprises: Based on the identification of the hole positions, the hole position coordinates are obtained; Get the gripper coordinates; According to the hole coordinates and gripper coordinates, the grasping strategy is determined to grasp the car side outer panel into the fixture.

[0011] Preferably, according to the hole position coordinates and the gripper coordinates, a gripping strategy is determined to grip the automobile side panel into a fixture, including: Get multiple gripper grasping directions; Projecting the hole coordinates and the grasping results of the grasping direction of the grasper based on the grasper coordinates into the same target space; Get the gripping mode of the gripper; Draw a grasping curve in the target space according to the grasping mode and the grasping result coordinates; Determine whether there is a grasping curve corresponding to the same grasper that passes through all the hole coordinates. If so, take the corresponding situation as a successful grasping situation, take the grasping direction of the corresponding grasper as the target grasping direction, and take the corresponding grasping mode as the target grasping mode; Get the first grasp cost of the target grasp direction; Get the second grasping cost of the target grasping mode; Sum the first grasping cost and the second grasping cost of the same successful grasping situation to obtain the grasping cost sum; The grasping cost and the target grasping direction and target grasping mode of the minimum successful grasping situation are used as the grasping strategy.

[0012] Preferably, according to the hole position coordinates and the gripper coordinates, determining the gripping strategy to grip the automobile side panel into the fixture also includes: Before taking the corresponding situation as a successful grasping situation, taking the corresponding situation as a successful grasping situation to be determined; Based on the hole coordinates, determining hole coordinate distribution characteristics, the hole coordinate distribution characteristics including: the side length of a polygon obtained by sequentially connecting the hole coordinates and the inner angle of a polygon obtained by sequentially connecting the hole coordinates; Based on the standard panel image, obtain the panel plane drawing template; The standard hole coordinate distribution feature in the panel plane drawing template and the hole coordinate distribution feature are matched. After the matching is successful, the panel plane is drawn based on the panel plane drawing template to obtain the panel plane; Try to obtain the intersection point between the plate plane and the grab curve; If the first moment of the intersection is after the second moment of the to-be-determined successful grasping situation, the corresponding to-be-determined successful grasping situation is regarded as the successful grasping situation; otherwise, the to-be-determined successful grasping situation is not regarded as the successful grasping situation.

[0013] The system for accurately positioning and grasping the side outer panel of an automobile provided by an embodiment of the present invention through machine vision includes: The panel number acquisition subsystem is used to determine whether the number of panels of the automobile side outer panel in the target material frame placed in the storage location is empty; The material delivery subsystem is used to dispatch staff to drive a forklift to deliver materials if the number of panels is empty; The plate image shooting subsystem is used to shoot the plate image of the automobile side outer panel that has been stamped in the target material frame through the 3D camera pre-installed on the gripper if the plate number is not empty; The hole position recognition subsystem is used to recognize the target hole position on the plate image based on machine vision technology and obtain the recognized hole position; The gripping subsystem is used to grip the vehicle side outer panel into the fixture based on the hole position identification.

[0014] Preferably, the board number acquisition subsystem includes: The module for obtaining the result of the recognition of the number of panels is used to obtain the frame positioning signal of the storage location, analyze the frame positioning signal, and obtain the result of the recognition of the number of panels; and / or, The panel number judgment result acquisition module is used to obtain the panel number judgment result of the forklift driver.

[0015] The beneficial effects of the present invention are: The present invention determines whether it is necessary to replenish the automobile side outer panel in time according to the judgment result of the number of panels in the target material frame placed in the storage location. When the number of panels is not empty, a 3D camera on the grasping device is introduced to shoot the panel image in the target material frame, and machine vision technology is introduced to identify the identification hole position in the panel image, determine the appropriate grasping strategy and grasp the automobile side outer panel into the fixture, so as to realize the automatic assembly of the automobile side outer panel, improve the assembly efficiency, and be more intelligent.

[0016] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures specifically pointed out in the present application documents.

[0017] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings: Figure 1 A schematic diagram of a method for accurately positioning and grasping a side outer panel of an automobile by machine vision in an embodiment of the present invention; Figure 2Schematic diagram of a system for accurately positioning and grasping a side outer panel of an automobile through machine vision in an embodiment of the present invention. DETAILED DESCRIPTION

[0019] The preferred embodiments of the present invention are described below in conjunction with the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0020] The embodiment of the present invention provides a method for accurately positioning and grasping the side outer panel of an automobile through machine vision, such as Figure 1 As shown, including: Step 1: Determine whether the number of panels of the automobile side outer panel in the target material frame placed in the storage location is empty; wherein, when determining whether the number of panels in the target material frame placed in the storage location is empty, the staff driving the forklift may determine it by observation, or may determine it by analyzing the frame positioning signal in the target material frame placed in the storage location (for example, analyzing the frame positioning signal to obtain the gravity sensing information of the gravity sensor in the positioned target material frame, if the gravity sensing value is greater than a preset threshold, the panel sensing signal is that a panel is sensed, and it is determined that the number of panels is not empty, otherwise it is determined that the number of panels is empty); Step 2: If the number of panels is empty, dispatch staff to drive a forklift to deliver the materials; Step 3: If the number of panels is not empty, the image of the panel of the automobile side outer panel that has been stamped in the target material frame is captured by a 3D camera pre-installed on the gripper; Step 4: Based on machine vision technology, identify the target hole position on the panel image and obtain the identified hole position; wherein the target hole position is a pre-designed hole on the panel for subsequent assembly or fixing operations; obtaining the identified hole position refers to identifying the hole position coordinates on the panel image; Step 5: Based on the hole position identification, grab the side panel of the car and put it into the fixture. The fixture is a device used to fix the side panel of the car to facilitate subsequent processing and assembly.

[0021] The working principle and beneficial effects of the above technical solution are: The present invention determines whether it is necessary to replenish the automobile side outer panel in time according to the judgment result of the number of panels in the target material frame placed in the storage location. When the number of panels is not empty, a 3D camera on the grasping device is introduced to shoot the panel image in the target material frame, and machine vision technology is introduced to identify the identification hole position in the panel image, determine the appropriate grasping strategy and grasp the automobile side outer panel into the fixture, so as to realize the automatic assembly of the automobile side outer panel, improve the assembly efficiency, and be more intelligent.

[0022] In one embodiment, determining whether the number of panels of the automobile side outer panel in the target material frame placed in the storage location is empty includes: Obtain the frame positioning signal of the storage location, analyze the frame positioning signal, and obtain the panel number recognition result; and / or, Obtain the forklift driver's panel number judgment result.

[0023] The working principle and beneficial effects of the above technical solution are: The present invention introduces two methods, frame positioning signal recognition and manual observation, to obtain the judgment result of the number of panels, thereby improving the timeliness of subsequent feeding.

[0024] In one embodiment, a 3D camera pre-assembled on a gripper is used to capture an image of a stamped automobile side panel in a target material frame, including: Obtaining storage location environment data; the storage location environment data refers to the environmental data around the storage location that affects the 3D camera shooting, such as the lighting conditions of the storage location; Initialize the 3D camera parameters according to the storage location environment data; Initializing the 3D camera parameters according to the storage location environment data means adjusting the 3D camera parameters (such as resolution, focal length, exposure and white balance, etc.) to a suitable storage location environment according to the storage location environment data; Based on the target detection technology, the target material frame is detected and the frame opening coordinates are calculated; wherein the frame opening coordinates are: the coordinates of the opening of the target material frame; Calculate the best shooting coordinates according to the frame opening coordinates; wherein the best shooting coordinates are: the best 3D camera position coordinates suitable for shooting the panels in the target material frame; Determine the 3D camera coordinates; wherein the 3D camera coordinates are: the real-time position coordinates of the 3D camera; According to the 3D camera coordinates and the best shooting coordinates, the 3D camera is adjusted to the best shooting position and the best shooting angle, and the panel image is captured by the adjusted 3D camera.

[0025] The working principle and beneficial effects of the above technical solution are: The present invention introduces the storage environment data to initialize the 3D camera parameters, introduces the target detection technology to first detect the frame opening coordinates of the target material frame, and after the frame opening coordinates are initially determined, it is necessary to further accurately determine the precise shooting coordinates of the 3D camera relative to the frame opening, that is, the optimal shooting coordinates. The frame opening coordinates are first determined, and then the optimal shooting coordinates are further determined, which improves the calculation efficiency of the optimal shooting coordinates. After determining the optimal shooting coordinates, the 3D camera is adjusted to the optimal shooting position and the optimal shooting angle to obtain the plate image, and the image quality of the plate image is higher, which further improves the subsequent grasping accuracy of the gripper.

[0026] In one embodiment, the optimal shooting coordinates are calculated according to the frame opening coordinates, including: Obtaining the dynamic coordinates of the 3D camera motion range; wherein the dynamic coordinates of the 3D camera motion range are: a set of position coordinates of a range within which the 3D camera can move in its working environment; Acquire a virtual shooting situation; wherein the virtual shooting situation is: a three-dimensional model of a virtually shot plate; Constructing a virtual shooting scene according to the frame opening coordinates, the dynamic coordinates of the 3D camera motion range and the virtual shooting situation; wherein, when constructing the virtual shooting scene, the virtual shooting situation is set according to the frame opening coordinates, and the virtual camera dynamic setting area is set according to the relative distance between the dynamic coordinates of the 3D camera motion range and the frame opening coordinates; Based on the initialized 3D camera parameters and the dynamic coordinates of the 3D camera motion range, a virtual camera object is constructed; wherein the virtual camera object is a virtual camera whose camera parameters conform to the initialized 3D camera parameters and whose set position coordinates conform to the dynamic coordinates of the 3D camera motion range; Randomly capture images according to the virtual shooting situation and the virtual camera object to obtain captured images; wherein the captured images are: the shot images of the virtual camera object; Extract the image features of the captured image, and input the image features into a preset image usability evaluation model to obtain the image usability value; wherein the image features are: the feature representation of the captured image, such as: the number of holes in the image, the image resolution, and the image contour; the image usability evaluation model is obtained by manually evaluating the usability of the image taken by the 3D camera for subsequent machine vision capture through training and convergence; the image usability value is: the usability of the captured image for subsequent machine vision capture judgment; The captured coordinates of the virtual camera object corresponding to the captured image with the largest image availability value are obtained, and the captured coordinates are used as the optimal shooting coordinates. The captured coordinates are: the virtual position of the virtual camera object that captures the captured image with the largest image availability value corresponds to the coordinates in the actual space when shooting.

[0027] The working principle and beneficial effects of the above technical solution are: The present invention introduces a virtual shooting situation, which is a three-dimensional model of a virtually shot plate, and is constructed according to the placement distribution form of an actual automobile side outer panel in a target material frame; a virtual shooting scene is constructed according to frame opening coordinates, 3D camera motion range dynamic coordinates and the virtual shooting situation. During the construction, the virtual coordinates of the frame opening coordinates in the virtual shooting scene are first determined, and the virtual shooting situation is mapped to the virtual shooting scene according to the position of the frame opening relative to the virtual shooting situation, and a virtual camera dynamic setting area in the virtual shooting scene is set according to the relative distance between the 3D camera motion range dynamic coordinates and the frame opening coordinates.

[0028] After the construction is completed, a virtual camera object is constructed based on the initialized 3D camera parameters and the dynamic coordinates of the 3D camera motion range, and random image capture is performed according to the virtual shooting situation and the virtual camera object to obtain a captured image; an image availability evaluation model is introduced, and the image availability value of the captured image is determined according to the image features of the captured image, and the virtual coordinates of the virtual camera object corresponding to the captured image with the largest image availability value are mapped to the capture coordinates in the actual space, and the captured coordinates are used as the optimal shooting coordinates. The determination of the optimal shooting position is more accurate and reasonable.

[0029] In one embodiment, obtaining a virtual shooting situation includes: Obtaining the placement of the automobile side panel in the target material frame; wherein the placement is: the actual placement distribution of the automobile side panel in the target material frame, obtained through the historical records of panel processing; Analyze the placement situation to obtain the relative position relationship between each automobile side panel and the target material frame; wherein the relative position relationship is: the relative position relationship between the automobile side panel and the target material frame; Obtaining material frame parameters of the target material frame; wherein the material frame parameters are: structural parameters of the material frame parameters, such as: length, width, height, structure, thickness, etc.; According to the material frame parameters, a virtual material frame is constructed; wherein the virtual material frame is: a three-dimensional model of the target material frame; According to the relative position relationship and the parameters of the automobile side outer panel, a panel model is constructed in the virtual material frame; wherein the panel model is: a three-dimensional model of the automobile side outer panel; A virtual material frame in which a panel model is constructed corresponding to each placement situation is used as a virtual shooting situation.

[0030] The working principle and beneficial effects of the above technical solution are: The present invention introduces the actual placement distribution pattern of the automobile side outer panel in the target material frame, analyzes the relative position relationship between each automobile side outer panel and the target material frame; constructs a virtual material frame based on the acquired material frame parameters of the target material frame; constructs a panel model in the virtual material frame based on the relative position relationship and the automobile side outer panel parameters, and uses the virtual material frame with the panel model constructed corresponding to each placement situation as a virtual shooting situation, thereby improving the comprehensiveness of the virtual shooting situation.

[0031] In one embodiment, based on the identification of the hole position, grabbing the side outer panel of the automobile into the fixture includes: Based on the identified hole position, the hole position coordinates are obtained; wherein the hole position coordinates are: the coordinates of the identified hole position in three-dimensional space; Obtaining the gripper coordinates; wherein the gripper coordinates are: the coordinates of the gripper used for gripping the vehicle side outer panel in three-dimensional space; According to the hole coordinates and gripper coordinates, the grasping strategy is determined to grasp the car side outer panel into the fixture.

[0032] The working principle and beneficial effects of the above technical solution are: The present invention determines the gripping strategy of the gripper based on the hole position coordinates and the gripper coordinates to clamp the plate into the fixture, thereby improving the gripping accuracy.

[0033] In one embodiment, according to the hole position coordinates and the gripper coordinates, a gripping strategy is determined to grip the side outer panel of the automobile into a fixture, including: Obtain multiple gripping directions of the gripper; wherein the gripping direction is: the gripper direction and gripper angle of the gripper when gripping the plate, the gripper direction is the direction of the gripper relative to the gripper plate, when determining the gripper angle, the initial gripper angle is set to 0 degrees when the gripper shaft has 0 torque, and the gripper angle is the rotation direction and rotation size of the gripper shaft relative to the initial gripper angle of 0 torque; Project the hole coordinates and the grasping results of the grasping direction of the grasper based on the grasper coordinates into the same target space; wherein the target space is: a simulated three-dimensional space for simulating the grasping of the plate by the grasper; Obtain the gripping mode of the gripper; wherein the gripping mode is: the gripping method of the gripper, such as: the gripper opening angle and gripping force; According to the grasping mode and the grasping result coordinates, a grasping curve is drawn in the target space; wherein the grasping result coordinates are: the position coordinates of the gripper claw tip during the grasping process of grasping the grasping direction of the gripper based on the gripper coordinates; the grasping curve is: a curve obtained by connecting the position coordinates of the same gripper claw tip; Determine whether there is a grasping curve corresponding to the same grasper that passes through all the hole coordinates. If so, take the corresponding situation as a successful grasping situation, take the grasping direction of the corresponding grasper as the target grasping direction, and take the corresponding grasping mode as the target grasping mode; Obtaining a first grasping cost of the target grasping direction; wherein the first grasping cost is determined by multiplying the driving power and the driving time of the target grasping direction; Obtaining a second grasping cost of the target grasping mode; wherein the second grasping cost is determined by multiplying the driving power and driving time of the target grasping mode; Sum the first grasping cost and the second grasping cost of the same successful grasping situation to obtain the grasping cost sum; The grasping cost and the target grasping direction and target grasping mode of the minimum successful grasping situation are used as the grasping strategy.

[0034] The working principle and beneficial effects of the above technical solution are: The present invention projects the hole coordinates and the grasping results of the grasping direction of the grasper based on the grasping coordinates to the same target space, draws a grasping curve on the target space according to the grasping mode and the grasping result coordinates, and determines whether there is a grasping curve corresponding to the same grasper that passes through all the hole coordinates. If so, it means that the corresponding grasping direction and grasping mode can be grasped successfully, and the corresponding situation is taken as a successful grasping situation, the grasping direction of the corresponding grasper is taken as the target grasping direction, and the corresponding grasping mode is taken as the target grasping mode. The driving power and driving time of the target grasping direction are multiplied to obtain the first grasping cost, and the driving power and driving time of the target grasping mode are multiplied to obtain the second grasping cost. The first grasping cost and the second grasping cost of the same successful grasping situation are summed to obtain the grasping cost sum, and the target grasping direction and target grasping mode of the successful grasping situation with the smallest grasping cost sum are taken as the grasping strategy, so that the formulation of the grasping strategy is more reasonable.

[0035] In one embodiment, according to the hole position coordinates and the gripper coordinates, a gripping strategy is determined to grip the side outer panel of the automobile into a fixture, and further includes: Before taking the corresponding situation as a successful grasping situation, taking the corresponding situation as a successful grasping situation to be determined; Based on the hole coordinates, determining hole coordinate distribution characteristics, the hole coordinate distribution characteristics including: the side length of a polygon obtained by sequentially connecting the hole coordinates and the inner angle of a polygon obtained by sequentially connecting the hole coordinates; Based on the standard plate image, a plate plane drawing template is obtained; wherein the plate plane drawing template includes: plate plane drawing methods corresponding to different standard hole coordinate distribution characteristics, for example: the standard hole coordinate distribution characteristic is that the side length of the polygon obtained by connecting the hole coordinates is 20 cm, then the corresponding plate plane drawing method is: a drawing method for a plate with a side length of 20 cm obtained by connecting the hole coordinates; The standard hole coordinate distribution feature in the panel plane drawing template and the hole coordinate distribution feature are matched. After the matching is successful, the panel plane is drawn based on the panel plane drawing template to obtain the panel plane; Try to obtain the intersection of the panel plane and the grab curve; the panel plane is: the three-dimensional model of the car side outer panel in the target space recognized in real time; If the first moment of the intersection is after the second moment of the to-be-determined successful grasping situation, the corresponding to-be-determined successful grasping situation is regarded as the successful grasping situation, otherwise, the to-be-determined successful grasping situation is not regarded as the successful grasping situation. The first moment is the simulated grasping moment when the grasping curve generated when the grasper performs simulated grasping of the grasping direction of the grasper based on the grasper coordinates and the simulated plate plane in the target space generate the intersection point; the second moment is the moment when the simulated to-be-determined successful grasping situation is generated.

[0036] The working principle and beneficial effects of the above technical solution are: Before determining a successful grasping situation, the present invention takes into account the situation that the gripper contacts the plate before passing through the hole, introduces a standard plate image, obtains a plate plane drawing template, and performs feature matching on the standard hole coordinate distribution characteristics and the hole coordinate distribution characteristics in the plate plane drawing template. After the matching is successful, the plate plane is drawn based on the plate plane drawing template to obtain the plate plane; the intersection of the plate plane and the grasping curve is obtained. If the first moment when the intersection is generated is after the second moment when the successful grasping situation is to be determined, it means that the gripper passes through the hole first and then contacts the plate, which does not hinder the stability of grasping and can be regarded as a successful grasping situation. Otherwise, the gripper contacts the plate first and then passes through the hole, which will cause displacement of the plate. At this time, it is inappropriate to judge whether to grasp based on the previous hole coordinates. Therefore, it is not regarded as a successful grasping situation, which improves the rationality of subsequent grasping strategy formulation.

[0037] The embodiment of the present invention provides a system for accurately positioning and grasping the side outer panel of an automobile through machine vision, such as Figure 2 As shown, including: The panel number acquisition subsystem 1 is used to determine whether the number of panels of the automobile side outer panel in the target material frame placed in the storage location is empty; Feeding subsystem 2 is used to dispatch staff to drive a forklift to deliver materials if the number of panels is empty; The plate image shooting subsystem 3 is used to shoot the plate image of the automobile side outer panel that has been stamped in the target material frame through a 3D camera pre-assembled on the gripper if the plate number is not empty; The hole position recognition subsystem 4 is used to recognize the target hole position on the plate image based on machine vision technology and obtain the recognized hole position; The grabbing subsystem 5 is used to grab the automobile side outer panel into the fixture based on the hole position identification.

[0038] In one embodiment, the board number acquisition subsystem includes: The module for obtaining the result of the recognition of the number of panels is used to obtain the frame positioning signal of the storage location, analyze the frame positioning signal, and obtain the result of the recognition of the number of panels; and / or, The panel number judgment result acquisition module is used to obtain the panel number judgment result of the forklift driver.

[0039] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A method for accurately positioning and grasping a side panel of an automobile by machine vision, characterized in that: include: Determine whether the number of panels of the automobile side outer panel in the target material frame placed in the storage location is empty; If the number of panels is empty, the dispatcher will drive a forklift to deliver the materials; If the number of panels is not empty, the image of the panel of the automobile side outer panel that has been stamped in the target material frame is captured by a 3D camera pre-installed on the gripper; Based on machine vision technology, the target hole positions on the panel image are identified and the identified hole positions are obtained; Based on the hole position identification, the car side outer panel is grabbed into the fixture.

2. The method for accurately positioning and grasping the side outer panel of an automobile by machine vision according to claim 1, characterized in that: Determine whether the number of panels of the automobile side outer panel in the target material frame placed in the storage location is empty, including: Obtain the frame positioning signal of the storage location, analyze the frame positioning signal, and obtain the panel number recognition result; and / or, Obtain the forklift driver's panel number judgment result.

3. The method for accurately positioning and grasping the side outer panel of an automobile by machine vision according to claim 1, characterized in that: The 3D camera pre-installed on the gripper captures the image of the stamped car side outer panel in the target frame, including: Get storage location environment data; Initialize 3D camera parameters according to storage environment data; Based on the target detection technology, detect the frame opening of the target material frame and calculate the frame opening coordinates; Calculate the best shooting coordinates according to the frame coordinates; Determine the 3D camera coordinates; According to the 3D camera coordinates and the best shooting coordinates, the 3D camera is adjusted to the best shooting position and the best shooting angle, and the panel image is captured by the adjusted 3D camera.

4. The method for accurately positioning and grasping the side outer panel of an automobile by machine vision according to claim 3, characterized in that: According to the frame coordinates, calculate the best shooting coordinates, including: Get the dynamic coordinates of the 3D camera motion range; Obtain a virtual shooting situation; Construct a virtual shooting scene according to the frame coordinates, the dynamic coordinates of the 3D camera motion range and the virtual shooting situation; Construct a virtual camera object based on the initialized 3D camera parameters and the dynamic coordinates of the 3D camera motion range; Perform random image capture according to a virtual shooting situation and a virtual camera object to obtain a captured image; Extracting image features of the captured image, and inputting the image features into a preset image usability evaluation model to obtain an image usability value; The captured coordinates of the virtual camera object corresponding to the captured image with the largest image available value are obtained, and the captured coordinates are used as the optimal shooting coordinates.

5. The method for accurately positioning and grasping the side outer panel of an automobile by machine vision according to claim 4, characterized in that: Get virtual shooting situations, including: Obtaining the placement of the automobile side outer panel in the target material frame; Analyze the placement situation and obtain the relative position relationship between each car side outer panel and the target material frame; Get the material frame parameters of the target material frame; Construct a virtual material frame according to the material frame parameters; According to the relative position relationship and the parameters of the car side outer panel, a panel model is constructed in the virtual material frame; A virtual material frame in which a panel model is constructed corresponding to each placement situation is used as a virtual shooting situation.

6. The method for accurately positioning and grasping the side outer panel of an automobile by machine vision according to claim 1, characterized in that: Based on the hole position identification, grab the car side outer panel into the fixture, including: Based on the identification of the hole positions, the hole position coordinates are obtained; Get the gripper coordinates; According to the hole coordinates and gripper coordinates, the grasping strategy is determined to grasp the car side outer panel into the fixture.

7. The method for accurately positioning and grasping the side outer panel of an automobile by machine vision according to claim 6, characterized in that: According to the hole coordinates and gripper coordinates, determine the grasping strategy to grasp the car side outer panel into the fixture, including: Get multiple gripper grasping directions; Projecting the hole coordinates and the grasping results of the grasping direction of the grasper based on the grasper coordinates into the same target space; Get the gripping mode of the gripper; Draw a grasping curve in the target space according to the grasping mode and the grasping result coordinates; Determine whether there is a grasping curve corresponding to the same grasper that passes through all the hole coordinates. If so, take the corresponding situation as a successful grasping situation, take the grasping direction of the corresponding grasper as the target grasping direction, and take the corresponding grasping mode as the target grasping mode; Get the first grasp cost of the target grasp direction; Get the second grasping cost of the target grasping mode; Sum the first grasping cost and the second grasping cost of the same successful grasping situation to obtain the grasping cost sum; The grasping cost and the target grasping direction and target grasping mode of the minimum successful grasping situation are used as the grasping strategy.

8. The method for accurately positioning and grasping the side outer panel of an automobile by machine vision as claimed in claim 7, characterized in that: According to the hole coordinates and gripper coordinates, the grasping strategy is determined to grasp the car side outer panel into the fixture, which also includes: Before taking the corresponding situation as a successful grasping situation, taking the corresponding situation as a successful grasping situation to be determined; Based on the hole coordinates, determining hole coordinate distribution characteristics, the hole coordinate distribution characteristics including: the side length of a polygon obtained by sequentially connecting the hole coordinates and the inner angle of a polygon obtained by sequentially connecting the hole coordinates; Based on the standard panel image, obtain the panel plane drawing template; The standard hole coordinate distribution feature in the panel plane drawing template and the hole coordinate distribution feature are feature matched. After the matching is successful, the panel plane is drawn based on the panel plane drawing template to obtain the panel plane; Try to obtain the intersection point between the plate plane and the grab curve; If the first moment of the intersection is after the second moment of the to-be-determined successful grasping situation, the corresponding to-be-determined successful grasping situation is regarded as the successful grasping situation; otherwise, the to-be-determined successful grasping situation is not regarded as the successful grasping situation.

9. A system for accurately positioning and grasping the side panels of automobiles through machine vision, characterized in that: include: The panel number acquisition subsystem is used to determine whether the number of panels of the automobile side outer panel in the target material frame placed in the storage location is empty; The material delivery subsystem is used to dispatch staff to drive a forklift to deliver materials if the number of panels is empty; The plate image shooting subsystem is used to shoot the plate image of the automobile side outer panel that has been stamped in the target material frame through the 3D camera pre-installed on the gripper if the plate number is not empty; The hole position recognition subsystem is used to recognize the target hole position on the plate image based on machine vision technology and obtain the recognized hole position; The gripping subsystem is used to grip the vehicle side outer panel into the fixture based on the hole position identification.

10. The system for accurately positioning and grasping the side outer panel of an automobile through machine vision as claimed in claim 9, characterized in that: The board quantity acquisition subsystem includes: The module for obtaining the result of the recognition of the number of panels is used to obtain the frame positioning signal of the storage location, analyze the frame positioning signal, and obtain the result of the recognition of the number of panels; and / or, The panel number judgment result acquisition module is used to obtain the panel number judgment result of the forklift driver.

Images