Flange positioning method and system applied to automobile automatic assembly

Abstract

The invention relates to the technical field of flange positioning, and provides a flange positioning method and system applied to automobile automatic assembly, and the method comprises the steps: obtaining a flange image of a to-be-assembled flange, and carrying out the preprocessing of the flange image, and obtaining a target flange image; identifying and positioning the target flange image to obtain a flange positioning result; and according to the flange positioning result, the assembly robot is controlled to complete automatic assembly of the automobile. According to the flange image of the to-be-assembled flange, recognition and positioning are carried out, the assembly robot is controlled to complete automatic assembly of the automobile according to the flange positioning result, the flange assembly precision and the assembly automation level can be improved, and the automobile assembly efficiency can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of flange positioning, and in particular to a flange positioning method and system applied to automatic assembly of automobiles. Background Art

[0002] In the process of automobile parts production, with the development of intelligence in the automobile industry, the demand for unmanned and intelligent production workshops is increasing. Among them, the assembly of flanges on the automobile automatic assembly line has a wide variety of flange parts. In order to ensure the accuracy of assembly, the flanges need to be accurately identified and positioned to ensure the accuracy of assembly implementation.

[0003] The flange positioning in the existing automobile automatic assembly process is not accurate enough, which affects the implementation accuracy and efficiency of automobile automatic assembly.

[0004] Therefore, it is necessary to provide a flange positioning method and system for automatic automobile assembly. Summary of the invention

[0005] The present invention provides a flange positioning method and system for automatic automobile assembly, which can improve the accuracy of flange assembly and the level of assembly automation by identifying and positioning the flange image of the flange to be assembled and controlling an assembly robot to complete the automatic assembly of the automobile according to the flange positioning result, thereby improving the efficiency of automobile assembly.

[0006] The present invention provides a flange positioning method for automobile automatic assembly, comprising:

[0007] Acquire a flange image of the flange to be assembled, and preprocess the flange image to obtain a target flange image;

[0008] Identify and locate the target flange image to obtain the flange positioning result;

[0009] According to the flange positioning results, the assembly robot is controlled to complete the automatic assembly of the car.

[0010] Further, a flange image of the flange to be assembled is obtained, and the flange image is preprocessed to obtain a target flange image, including:

[0011] Taking photos of the flange to be assembled based on the configured binocular camera to obtain the flange image;

[0012] The flange image is grayed, gray-scale transformed, and image enhanced to obtain a simplified flange image; wherein graying includes but is not limited to component method, mean method, maximum method, and weighted average method; gray-scale transformation includes but is not limited to logarithmic transformation, power law transformation, and linear transformation; image enhancement is histogram equalization;

[0013] The simplified flange image is subjected to hybrid filtering and denoising processing to obtain the target flange image.

[0014] Furthermore, the target flange image is identified and located to obtain the flange location result, including:

[0015] The target flange image is identified by using the template matching method to obtain the target area containing the flange;

[0016] Locate the target area and obtain the flange positioning result.

[0017] Furthermore, a template matching method is used to identify the target flange image to obtain a target area containing the flange, including:

[0018] Acquire the features or shape of the flange in the set template image;

[0019] Extract shape feature information and spatial position relationship from the set template image;

[0020] Based on the shape feature information and the spatial position relationship, a matching first shape feature and a first spatial position are searched in the target flange image, and a target area containing the flange is obtained after marking.

[0021] Further, based on the shape feature information and the spatial position relationship, searching for a matching first shape feature and a first spatial position in the target flange image, and obtaining a target area containing the flange after marking, including:

[0022] Preprocessing the set template image to obtain a preprocessed template image;

[0023] Obtain the outermost contour and holes representing shape feature information and spatial position relationship in the preprocessed template image;

[0024] Get the entire outline of the target flange image;

[0025] The contour-based Hu moment feature is used for matching to obtain the holes in the target flange image;

[0026] The holes are extracted, and SIFT local feature matching is performed between the holes in the template image and the holes in the target flange image to obtain matching feature points according to the steps of detecting scale space extreme values, locating key points, matching key points, and screening the results of SIFT preliminary matching using a random sampling consistency estimation algorithm;

[0027] Based on the matching feature points, obtaining a corresponding first shape feature and a first spatial position;

[0028] The first shape feature and the first spatial position are marked to obtain a target area including the flange.

[0029] Furthermore, the contour-based Hu moment feature is used for matching to obtain the holes of the target flange image, including:

[0030] Acquire the template image and the target flange image, rotate the template image by 0°-360°, and select the rotated template images when they are rotated to four angles of 45°, 90°, 225° and 270° respectively;

[0031] Preprocessing the template image, the rotated template image and the target flange image, and performing threshold segmentation and contour extraction to obtain the outer contour of the template image, the outer contour of the rotated template image and the entire contour of the target flange image;

[0032] The outer contour of the template image, the outer contour of the rotated template image and the entire contour of the target flange image are calculated to obtain the geometric moment, center distance and normalized moment;

[0033] According to the geometric moment, the center distance and the normalized moment, the Hu moment of the template image, the Hu moment of the rotated template image and the Hu moment of each contour contained in all contours of the target flange image are calculated;

[0034] The Hu moment of the template image and the Hu moment of the rotated template image are stored in the template image Hu moment list, and the difference value is obtained by traversing the list and subtracting the Hu moment values ​​of each contour contained in all contours of the target flange image;

[0035] According to the set difference threshold, the difference value is judged by the threshold value. If the difference value is less than the set difference threshold, it is judged that the outer contour of the template image is similar to the contour of the target flange image.

[0036] According to the determination result that the outer contour of the template image is similar to the contour of the target flange image, the holes of the target flange image are obtained.

[0037] Further, the target area is positioned to obtain flange positioning results, including:

[0038] The Bouguet algorithm is used to perform stereo correction on the target flange image to obtain a corrected target flange image;

[0039] Based on the matching feature points, the homography matrix is ​​calculated, and the coordinates of the center of the left image of the corrected target flange image are calculated according to the homography matrix and the center of the template image;

[0040] Using a regional stereo matching algorithm, the center coordinates of the right image of the correction target flange image that is stereo matched with the center coordinates of the left image are obtained;

[0041] Based on the coordinates of the center of the left image and the center of the right image, a disparity map is obtained using the disparity ranging method, and the flange positioning result is obtained based on the disparity map.

[0042] Further, according to the flange positioning result, the assembly robot is controlled to complete the automatic assembly of the automobile, including:

[0043] According to the flange positioning result, the position of the flange in the storage space is obtained;

[0044] Using the set automobile automatic assembly platform, control the drone to take pictures of the flange in position and obtain the flange position image;

[0045] Based on the flange position image, draw a distribution map of the flange position in the storage space;

[0046] According to the process of automatic assembly of automobiles and the position distribution map, the assembly and use sequence of flanges is obtained;

[0047] According to the assembly and retrieval sequence and the initial operation position of the assembly robot, the movement trajectory of the assembly robot for retrieval of the flange is designed;

[0048] Generate programming instructions for the assembly robot based on the initial operation position of the assembly robot and the moving trajectory of the assembly robot taking the flange;

[0049] Utilize the automobile automatic assembly platform to control the assembly robot based on programming instructions to complete the automatic assembly of the automobile.

[0050] Furthermore, according to the flange positioning result, the assembly robot is controlled to complete the automatic assembly of the automobile, and the flange is replenished in time according to the assembly and use status of the flange; the specific steps are:

[0051] Using the automobile automatic assembly platform, obtaining assembly record data of the assembly robot implementing the automatic assembly of the automobile;

[0052] According to the assembly record data, the flange inventory prediction model is used to predict the remaining assembly quantity of the flange to obtain prediction data; wherein the flange inventory prediction model is generated based on long short-term memory network training;

[0053] Generate a product replenishment strategy for flanges based on forecast data and combined with the automobile automatic assembly production plan;

[0054] According to the set evaluation model, the degree of adaptation between the product replenishment strategy and the automatic assembly production plan is evaluated. If the degree of adaptation is less than the set adaptation threshold, the automatic assembly production plan and the product replenishment strategy of the flange are adjusted according to the production needs of the automobile.

[0055] A flange positioning system for automobile automatic assembly, comprising:

[0056] A target flange image acquisition module is used to acquire a flange image of the flange to be assembled, and pre-process the flange image to obtain a target flange image;

[0057] The flange positioning result acquisition module is used to identify and locate the target flange image and obtain the flange positioning result;

[0058] The automatic assembly implementation module is used to control the assembly robot to complete the automatic assembly of the car according to the flange positioning result.

[0059] Compared with the prior art, the present invention has the following advantages and beneficial effects: by identifying and locating the flange image of the flange to be assembled, and controlling the assembly robot to complete the automatic assembly of the automobile according to the flange positioning result, the accuracy of flange assembly and the level of assembly automation can be improved, which is beneficial to improving the efficiency of automobile assembly.

[0060] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures particularly pointed out in the written description and the accompanying drawings.

[0061] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0063] Figure 1 A schematic diagram of the steps of a flange positioning method used in automatic assembly of automobiles;

[0064] Figure 2 A schematic diagram of the steps for obtaining flange positioning results;

[0065] Figure 3 The figure is a schematic diagram of the structure of a flange positioning system used in automatic automobile assembly. DETAILED DESCRIPTION

[0066] The preferred embodiments of the present invention are described below in conjunction with the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0067] The present invention provides a flange positioning method for automobile automatic assembly, such as Figure 1 As shown, including:

[0068] Acquire a flange image of the flange to be assembled, and preprocess the flange image to obtain a target flange image;

[0069] Identify and locate the target flange image to obtain the flange positioning result;

[0070] According to the flange positioning results, the assembly robot is controlled to complete the automatic assembly of the car.

[0071] The working principle of the above technical solution is: in order to realize the flange positioning method applied to the automatic assembly of automobiles, the present invention first obtains the flange image of the flange to be assembled, and pre-processes the flange image to obtain the target flange image; then identifies and locates the target flange image to obtain the flange positioning result; finally, according to the flange positioning result, controls the assembly robot to complete the automatic assembly of the automobile.

[0072] The beneficial effect of the above technical solution is: by adopting the solution provided in this embodiment, by identifying and locating the flange image of the flange to be assembled, and controlling the assembly robot to complete the automatic assembly of the automobile based on the flange positioning result, the accuracy of the flange assembly and the level of assembly automation can be improved, which is beneficial to improving the efficiency of automobile assembly.

[0073] In one embodiment, a flange image of a flange to be assembled is acquired, and the flange image is preprocessed to obtain a target flange image, including:

[0074] Taking photos of the flange to be assembled based on the configured binocular camera to obtain the flange image;

[0075] The flange image is grayed, gray-scale transformed, and image enhanced to obtain a simplified flange image; wherein graying includes but is not limited to component method, mean method, maximum method, and weighted average method; gray-scale transformation includes but is not limited to logarithmic transformation, power law transformation, and linear transformation; image enhancement is histogram equalization;

[0076] The simplified flange image is subjected to hybrid filtering and denoising processing to obtain the target flange image.

[0077] The working principle of the above technical solution is: in order to obtain the target flange image, the present invention takes a picture of the flange to be assembled based on the configured binocular camera to obtain the flange image; then a preprocessing method is used to grayscale, grayscale transform and image enhance the flange image to obtain a simplified flange image; wherein, grayscale includes but is not limited to component method, mean method, maximum method and weighted average method; grayscale transformation includes but is not limited to logarithmic transformation, power law transformation and linear transformation; image enhancement is histogram equalization; the simplified flange image is subjected to hybrid filtering and denoising processing to obtain the target flange image.

[0078] The beneficial effect of the above technical solution is: by adopting the solution provided in this embodiment, by acquiring the flange image, and by graying, graying, image enhancing and hybrid filtering and denoising the flange image, it is possible to ensure that a target flange image that meets the requirements is obtained.

[0079] In one embodiment, Figure 2 As shown, the target flange image is identified and located to obtain the flange positioning result, including:

[0080] The target flange image is identified by using the template matching method to obtain the target area containing the flange;

[0081] Locate the target area and obtain the flange positioning result.

[0082] The working principle of the above technical solution is: in order to realize the positioning of the flange, the present invention adopts a template matching method to identify the target flange image and obtain a target area containing the flange; then the target area is positioned to obtain the flange positioning result.

[0083] The beneficial effect of the above technical solution is: by adopting the solution provided in this embodiment, the flange is identified and positioned, providing a basis and conditions for subsequent flange assembly.

[0084] In one embodiment, a template matching method is used to identify the target flange image to obtain a target area containing the flange, including:

[0085] Acquire the features or shape of the flange in the set template image;

[0086] Extract shape feature information and spatial position relationship from the set template image;

[0087] Based on the shape feature information and the spatial position relationship, a matching first shape feature and a first spatial position are searched in the target flange image, and a target area containing the flange is obtained after marking.

[0088] The working principle of the above technical solution is as follows: in order to obtain the target area containing the flange, the present invention adopts a template matching method to first obtain the characteristics or shape of the flange in the set template image; then extract the shape feature information and the spatial position relationship from the set template image; finally, based on the shape feature information and the spatial position relationship, search for the matching first shape feature and the first spatial position in the target flange image, and obtain the target area containing the flange after marking.

[0089] The beneficial effect of the above technical solution is: by adopting the solution provided in this embodiment, the target area including the flange is obtained by adopting the template matching method, so that the target area can be accurately obtained.

[0090] In one embodiment, based on the shape feature information and the spatial position relationship, searching for a matching first shape feature and a first spatial position in the target flange image, and obtaining a target area including the flange after marking, includes:

[0091] Preprocessing the set template image to obtain a preprocessed template image;

[0092] Obtain the outermost contour and holes representing shape feature information and spatial position relationship in the preprocessed template image;

[0093] Get the entire outline of the target flange image;

[0094] The contour-based Hu moment feature is used for matching to obtain the holes in the target flange image;

[0095] The holes are extracted, and SIFT local feature matching is performed between the holes in the template image and the holes in the target flange image to obtain matching feature points according to the steps of detecting scale space extreme values, locating key points, matching key points, and screening the results of SIFT preliminary matching using a random sampling consistency estimation algorithm;

[0096] Based on the matching feature points, obtaining a corresponding first shape feature and a first spatial position;

[0097] The first shape feature and the first spatial position are marked to obtain a target area including the flange.

[0098] The working principle of the above technical solution is as follows: SIFT is a local feature point descriptor, which essentially constructs different scale spaces, finds key points in them, and calculates the direction of key points. It has invariance to rotation, scale, illumination, viewpoint, etc. The present invention first preprocesses the set template image to obtain a preprocessed template image; obtains the outermost contour and holes representing the shape feature information and spatial position relationship in the preprocessed template image; obtains the entire contour of the target flange image; then uses the contour-based Hu moment feature for matching to obtain the holes in the target flange image; since the flange is a weak texture shape, its surface does not allow it to have a good matching point, but will cause interference, therefore, the S of the flange holes is selected. IFT features are matched and combined with Hu moments; specifically, the holes are extracted, and the holes in the template image are matched with the holes in the target flange image according to the steps of detecting scale space extreme values, locating key points, matching key points, and using a random sampling consistency estimation algorithm to screen the results of SIFT preliminary matching to obtain matching feature points; due to the large number of mismatched points, the introduction of a random sampling consistency estimation algorithm to screen the results of SIFT preliminary matching can effectively solve this problem; finally, based on the matching feature points, the corresponding first shape feature and first spatial position are obtained; the first shape feature and the first spatial position are marked to obtain the target area containing the flange.

[0099] The beneficial effect of the above technical solution is: by adopting the solution provided in this embodiment, by selecting the SIFT features of the flange holes for matching and combining it with the Hu moment, the accuracy of obtaining the target area of ​​the flange can be guaranteed.

[0100] In one embodiment, matching is performed using contour-based Hu moment features to obtain holes in the target flange image, including:

[0101] Acquire the template image and the target flange image, rotate the template image by 0°-360°, and select the rotated template images when they are rotated to four angles of 45°, 90°, 225° and 270° respectively;

[0102] Preprocessing the template image, the rotated template image and the target flange image, and performing threshold segmentation and contour extraction to obtain the outer contour of the template image, the outer contour of the rotated template image and the entire contour of the target flange image;

[0103] The outer contour of the template image, the outer contour of the rotated template image and the entire contour of the target flange image are calculated to obtain the geometric moment, center distance and normalized moment;

[0104] According to the geometric moment, the center distance and the normalized moment, the Hu moment of the template image, the Hu moment of the rotated template image and the Hu moment of each contour contained in all contours of the target flange image are calculated;

[0105] The Hu moment of the template image and the Hu moment of the rotated template image are stored in the template image Hu moment list, and the difference value is obtained by traversing the list and subtracting the Hu moment values ​​of each contour contained in all contours of the target flange image;

[0106] According to the set difference threshold, the difference value is judged by the threshold value. If the difference value is less than the set difference threshold, it is judged that the outer contour of the template image is similar to the contour of the target flange image.

[0107] According to the determination result that the outer contour of the template image is similar to the contour of the target flange image, the holes of the target flange image are obtained.

[0108] The working principle of the above technical solution is: the interior of the flange is composed of two identical ellipses and circles, and their positions, inner diameters, major and minor axes, etc. all need to be identified. The procedure is complicated and the time required is much longer than that of the outer contour. The contour-based Hu moment feature is used for matching to reduce the amount of calculation; at the same time, the Hu moment is invariant to rotation, and it has good anti-interference ability to the lighting environment. However, when the image rotation angle is too large, the image function will change, and its Hu moment value will change. In order to obtain a high degree of matching, the present invention first obtains the template image and the target flange image, and rotates the template image 0o-360o, and selects the rotated template images when rotated to four angles of 45°, 90°, 225° and 270° respectively; then the template image, the rotated template image and the target flange image are preprocessed, and threshold segmentation and contour extraction are performed to obtain the outer contour of the template image, the outer contour of the rotated template image and the target flange image. all contours; then, the moments of the outer contour of the template image, the outer contour of the rotated template image and the entire contour of the target flange image are calculated to obtain the geometric moment, the center distance and the normalized moment; then, according to the geometric moment, the center distance and the normalized moment, the Hu moment of the template image, the Hu moment of the rotated template image and the Hu moment of each contour contained in the entire contour of the target flange image are calculated; the Hu moment of the template image and the Hu moment of the rotated template image are stored in the template image Hu moment list, and the difference value is obtained by traversing the list and subtracting the Hu moment value of each contour contained in the entire contour of the target flange image; finally, according to the set difference threshold, the difference value is threshold judged, if the difference value is less than the set difference threshold, it is judged that the outer contour of the template image is similar to the contour of the target flange image; according to the judgment result that the outer contour of the template image is similar to the contour of the target flange image, the hole of the target flange image is obtained.

[0109] The beneficial effect of the above technical solution is: by adopting the solution provided in this embodiment, the holes of the target flange image can be accurately obtained by performing multiple steps of calculation on the Hu moments of the contours of the template image and the target flange image and judging the threshold of the difference value.

[0110] In one embodiment, positioning the target area to obtain the flange positioning result includes:

[0111] The Bouguet algorithm is used to perform stereo correction on the target flange image to obtain a corrected target flange image;

[0112] Based on the matching feature points, the homography matrix is ​​calculated, and the coordinates of the center of the left image of the corrected target flange image are calculated according to the homography matrix and the center of the template image;

[0113] Using a regional stereo matching algorithm, the center coordinates of the right image of the correction target flange image that is stereo matched with the center coordinates of the left image are obtained;

[0114] Based on the coordinates of the center of the left image and the center of the right image, a disparity map is obtained using the disparity ranging method, and the flange positioning result is obtained based on the disparity map.

[0115] The working principle of the above technical solution is: in order to obtain the flange positioning result, it is necessary to perform stereo matching on the collected left and right images, obtain the parallax, and then calculate the depth information of the point based on the parallax; the present invention adopts the Bouguet algorithm to perform stereo correction on the target flange image to obtain the corrected target flange image; the purpose of stereo correction is to correct the two non-coplanar and non-row aligned images taken by the binocular camera to coplanar row alignment, so that the points on the left image and the corresponding points on the right image are in the same row, and then a one-dimensional search in the same row can match each other to find the corresponding points; based on the matching feature points, the homography matrix is ​​calculated, and the coordinates of the center of the left image of the corrected target flange image are calculated based on the homography matrix and the center of the template image; the homography is a mapping between two plane projections of the image; and then regional stereo matching is used The algorithm obtains the center coordinates of the right image of the corrected target flange image that is stereo matched with the center coordinates of the left image; the normalized cross-correlation matching method is used in the regional stereo matching algorithm, which is to calculate the target area size of the image to be matched as the window size, slide this window in the image to be matched, calculate the normalized cross-correlation value between each window area and the target area, and determine the target area of ​​the image to be matched according to the size of the normalized cross-correlation value to complete the stereo matching, which can effectively reduce the influence of illumination on the image comparison result; then based on the center coordinates of the left image and the center coordinates of the right image, the disparity ranging method is used to obtain the disparity map, and the flange positioning result is obtained based on the disparity map; wherein, the disparity ranging method uses a binocular camera to shoot images of the same scene, and calculates the distance of the object by comparing the pixel information in the two images.

[0116] The beneficial effect of the above technical solution is: by adopting the solution provided in this embodiment, by stereo correcting the target flange image and performing regional stereo matching, the center coordinates of the left image and the center coordinates of the right image can be obtained, and the disparity map can be further obtained, and the accurate flange positioning result can be obtained based on the disparity map.

[0117] In one embodiment, according to the flange positioning result, controlling the assembly robot to complete the automatic assembly of the automobile includes:

[0118] According to the flange positioning result, the position of the flange in the storage space is obtained;

[0119] Using the set automobile automatic assembly platform, control the drone to take pictures of the flange in position and obtain the flange position image;

[0120] Based on the flange position image, draw a distribution map of the flange position in the storage space;

[0121] According to the process of automatic assembly of automobiles and the position distribution map, the assembly and use sequence of flanges is obtained;

[0122] According to the assembly and retrieval sequence and the initial operation position of the assembly robot, the movement trajectory of the assembly robot for retrieval of the flange is designed;

[0123] Generate programming instructions for the assembly robot based on the initial operation position of the assembly robot and the moving trajectory of the assembly robot taking the flange;

[0124] Utilize the automobile automatic assembly platform to control the assembly robot based on programming instructions to complete the automatic assembly of the automobile.

[0125] The working principle of the above technical solution is as follows: in order to ensure the quality of the automatic assembly of the automobile by the assembly robot, the present invention first obtains the position of the flange in the storage space according to the flange positioning result; then uses the set automobile automatic assembly platform to control the drone to take pictures of the flange in position to obtain the flange position image; based on the flange position image, draws a position distribution map of the flange in the storage space; according to the process of automatic assembly of the automobile and the position distribution map, obtains the assembly and retrieval sequence of the flange; according to the assembly and retrieval sequence and the initial operating position of the assembly robot, designs the movement trajectory route of the assembly robot to take the flange; based on the initial operating position of the assembly robot and the movement trajectory route of the assembly robot to take the flange, generates the programming instructions of the assembly robot; finally, uses the automobile automatic assembly platform to control the assembly robot to complete the automatic assembly of the automobile based on the programming instructions.

[0126] The beneficial effect of the above technical solution is: by adopting the solution provided in this embodiment, the movement trajectory route of the assembly robot to pick up the flange is designed according to the position of the storage space of the flange, and based on the initial operating position of the assembly robot and the movement trajectory route of the assembly robot to pick up the flange, the programming instructions of the assembly robot are generated, thereby controlling the assembly robot to complete the automatic assembly of the car, thereby ensuring the effect of the automatic assembly of the car.

[0127] In one embodiment, according to the flange positioning result, the assembly robot is controlled to complete the automatic assembly of the automobile, and the flange is replenished in time according to the assembly and use status of the flange; the specific steps are:

[0128] Using the automobile automatic assembly platform, the assembly record data of the assembly robot implementing the automatic assembly of the automobile is obtained;

[0129] According to the assembly record data, the flange inventory prediction model is used to predict the remaining assembly quantity of the flange to obtain prediction data; wherein the flange inventory prediction model is generated based on long short-term memory network training;

[0130] Generate a product replenishment strategy for flanges based on forecast data and combined with the automobile automatic assembly production plan;

[0131] According to the set evaluation model, the degree of adaptation between the product replenishment strategy and the automatic assembly production plan is evaluated. If the degree of adaptation is less than the set adaptation threshold, the automatic assembly production plan and the product replenishment strategy of the flange are adjusted according to the production needs of the automobile.

[0132] The working principle of the above technical solution is as follows: in order to ensure the orderly and smooth implementation of flange assembly, it is necessary to monitor the flange inventory in real time and plan the replenishment of flange goods. The present invention utilizes the automobile automatic assembly platform to obtain the assembly record data of the assembly robot implementing the automatic assembly of the automobile; according to the assembly record data, the flange inventory prediction model is used to predict the remaining assembly quantity of the flange to obtain the prediction data; wherein, the flange inventory prediction model is generated based on the long short-term memory network training; the long short-term memory network is a time recursive neural network, which is suitable for processing and predicting important events with relatively long intervals and delays in time series; according to the prediction data, combined with the automatic assembly production plan of the automobile, a flange product replenishment strategy is generated; according to the set evaluation model, the degree of adaptation of the product replenishment strategy and the automatic assembly production plan is evaluated. If the degree of adaptation is less than the set degree of adaptation threshold, the automatic assembly production plan and the flange product replenishment strategy are adjusted according to the production needs of the automobile.

[0133] The beneficial effect of the above technical solution is: by adopting the solution provided in this embodiment, by using the prediction model to predict the flange inventory and formulate a product replenishment strategy, the automatic assembly production plan and the flange product replenishment strategy can be adjusted in time according to the production needs of the automobile, thereby providing a guarantee for the smooth and efficient implementation of the flange assembly.

[0134] A flange positioning system for automatic assembly of automobiles, such as Figure 3 As shown, including:

[0135] A target flange image acquisition module is used to acquire a flange image of the flange to be assembled, and pre-process the flange image to obtain a target flange image;

[0136] The flange positioning result acquisition module is used to identify and locate the target flange image and obtain the flange positioning result;

[0137] The automatic assembly implementation module is used to control the assembly robot to complete the automatic assembly of the car according to the flange positioning result.

[0138] The working principle of the above technical solution is: in order to realize the flange positioning system applied to the automatic assembly of automobiles, the present invention sets a target flange image acquisition module for acquiring the flange image of the flange to be assembled, and pre-processes the flange image to obtain the target flange image; sets a flange positioning result acquisition module for identifying and positioning the target flange image to obtain the flange positioning result; sets an automatic assembly implementation module for controlling the assembly robot to complete the automatic assembly of the automobile according to the flange positioning result.

[0139] The beneficial effect of the above technical solution is: by adopting the solution provided in this embodiment, by identifying and locating the flange image of the flange to be assembled, and controlling the assembly robot to complete the automatic assembly of the automobile based on the flange positioning result, the accuracy of the flange assembly and the level of assembly automation can be improved, which is beneficial to improving the efficiency of automobile assembly.

[0140] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A flange positioning method for automatic assembly of automobiles, characterized in that: include: Acquire a flange image of the flange to be assembled, and preprocess the flange image to obtain a target flange image; Identify and locate the target flange image to obtain the flange positioning result; According to the flange positioning results, the assembly robot is controlled to complete the automatic assembly of the car.

2. A flange positioning method for automobile automatic assembly according to claim 1, characterized in that: Acquire the flange image of the flange to be assembled, and preprocess the flange image to obtain the target flange image, including: Taking photos of the flange to be assembled based on the configured binocular camera to obtain the flange image; The flange image is grayed, gray-scale transformed, and image enhanced to obtain a simplified flange image; wherein graying includes but is not limited to component method, mean method, maximum method, and weighted average method; gray-scale transformation includes but is not limited to logarithmic transformation, power law transformation, and linear transformation; image enhancement is histogram equalization; The simplified flange image is subjected to hybrid filtering and denoising processing to obtain the target flange image.

3. A flange positioning method for automobile automatic assembly according to claim 1, characterized in that: Identify and locate the target flange image to obtain flange positioning results, including: The target flange image is identified by using the template matching method to obtain the target area containing the flange; Locate the target area and obtain the flange positioning result.

4. A flange positioning method for automobile automatic assembly according to claim 3, characterized in that: The target flange image is identified by using the template matching method to obtain the target area containing the flange, including: Acquire the features or shape of the flange in the set template image; Extract shape feature information and spatial position relationship from the set template image; Based on the shape feature information and the spatial position relationship, a matching first shape feature and a first spatial position are searched in the target flange image, and a target area containing the flange is obtained after marking.

5. A flange positioning method for automobile automatic assembly according to claim 4, characterized in that: Based on the shape feature information and the spatial position relationship, a matching first shape feature and a first spatial position are searched in the target flange image, and a target area containing the flange is obtained after marking, including: Preprocessing the set template image to obtain a preprocessed template image; Obtain the outermost contour and holes representing shape feature information and spatial position relationship in the preprocessed template image; Get the entire outline of the target flange image; The contour-based Hu moment feature is used for matching to obtain the holes in the target flange image; The holes are extracted, and SIFT local feature matching is performed between the holes in the template image and the holes in the target flange image to obtain matching feature points according to the steps of detecting scale space extreme values, locating key points, matching key points, and screening the results of SIFT preliminary matching using a random sampling consistency estimation algorithm; Based on the matching feature points, obtaining a corresponding first shape feature and a first spatial position; The first shape feature and the first spatial position are marked to obtain a target area including the flange.

6. A flange positioning method for automobile automatic assembly according to claim 5, characterized in that: The contour-based Hu moment feature is used for matching to obtain the holes of the target flange image, including: Acquire the template image and the target flange image, rotate the template image by 0°-360°, and select the rotated template images when they are rotated to four angles of 45°, 90°, 225° and 270° respectively; Preprocessing the template image, the rotated template image and the target flange image, and performing threshold segmentation and contour extraction to obtain the outer contour of the template image, the outer contour of the rotated template image and the entire contour of the target flange image; The outer contour of the template image, the outer contour of the rotated template image and the entire contour of the target flange image are calculated to obtain the geometric moment, center distance and normalized moment; According to the geometric moment, the center distance and the normalized moment, the Hu moment of the template image, the Hu moment of the rotated template image and the Hu moment of each contour contained in all contours of the target flange image are calculated; The Hu moment of the template image and the Hu moment of the rotated template image are stored in the template image Hu moment list, and the difference value is obtained by traversing the list and subtracting the Hu moment values ​​of each contour contained in all contours of the target flange image; According to the set difference threshold, the difference value is judged by the threshold value. If the difference value is less than the set difference threshold, it is judged that the outer contour of the template image is similar to the contour of the target flange image. According to the determination result that the outer contour of the template image is similar to the contour of the target flange image, the holes of the target flange image are obtained.

7. A flange positioning method for automobile automatic assembly according to claim 5, characterized in that: Locate the target area and obtain flange positioning results, including: The Bouguet algorithm is used to perform stereo correction on the target flange image to obtain a corrected target flange image; Based on the matching feature points, the homography matrix is ​​calculated, and the coordinates of the center of the left image of the corrected target flange image are calculated according to the homography matrix and the center of the template image; Using a regional stereo matching algorithm, the center coordinates of the right image of the correction target flange image that is stereo matched with the center coordinates of the left image are obtained; Based on the coordinates of the center of the left image and the center of the right image, a disparity map is obtained using the disparity ranging method, and the flange positioning result is obtained based on the disparity map.

8. The flange positioning method for automobile automatic assembly according to claim 1, characterized in that: According to the flange positioning results, the assembly robot is controlled to complete the automatic assembly of the car, including: According to the flange positioning result, the position of the flange in the storage space is obtained; Using the set automobile automatic assembly platform, control the drone to take pictures of the flange in position and obtain the flange position image; Based on the flange position image, draw a distribution map of the flange position in the storage space; According to the process of automatic assembly of automobiles and the position distribution map, the assembly and use sequence of flanges is obtained; According to the assembly and retrieval sequence and the initial operation position of the assembly robot, the movement trajectory of the assembly robot for retrieval of the flange is designed; Generate programming instructions for the assembly robot based on the initial operation position of the assembly robot and the moving trajectory of the assembly robot taking the flange; Utilize the automobile automatic assembly platform to control the assembly robot based on programming instructions to complete the automatic assembly of the automobile.

9. A flange positioning method for automobile automatic assembly according to claim 8, characterized in that: According to the flange positioning result, the assembly robot is controlled to complete the automatic assembly of the car, and the flange is replenished in time according to the assembly and use status of the flange. The specific steps are: Using the automobile automatic assembly platform, obtaining assembly record data of the assembly robot implementing the automatic assembly of the automobile; According to the assembly record data, the flange inventory prediction model is used to predict the remaining assembly quantity of the flange to obtain prediction data; wherein the flange inventory prediction model is generated based on long short-term memory network training; Generate a product replenishment strategy for flanges based on forecast data and combined with the automobile automatic assembly production plan; According to the set evaluation model, the degree of adaptation between the product replenishment strategy and the automatic assembly production plan is evaluated. If the degree of adaptation is less than the set adaptation threshold, the automatic assembly production plan and the product replenishment strategy of the flange are adjusted according to the production needs of the automobile.

10. A flange positioning system for automobile automatic assembly, characterized in that: include: A target flange image acquisition module is used to acquire a flange image of the flange to be assembled, and pre-process the flange image to obtain a target flange image; The flange positioning result acquisition module is used to identify and locate the target flange image and obtain the flange positioning result; The automatic assembly implementation module is used to control the assembly robot to complete the automatic assembly of the car according to the flange positioning result.

Images