Profile end position detection method for aluminum profile stretching and straightening based on depth camera

Abstract

The present invention provides a method for detecting the end position of aluminum profiles during stretching and straightening based on a depth camera. It comprises: two depth cameras are respectively installed on a mobile stretching trolley and a fixed stretching trolley, and are located above a cooling bed, with the lenses vertically downward and the field of view facing the profiles on the cooling bed; a proximity switch is installed on the cooling bed to detect whether the profiles have been loaded, and when the proximity switch detects that the profiles are in place, the depth camera starts to collect data. The method first performs regional interception, selects the area to be detected, and uses the depth information to eliminate the background and remove interference at the same time; the depth cameras on both sides identify and locate the ends of the profiles on both sides. The method of the present invention has the characteristics of simple implementation, wide applicability and high robustness.

Description

Technical Field

[0001] The invention relates to a profile position detection method in the field of stretching and straightening, and in particular to a profile end detection method when fully automated stretching and straightening is realized for aluminum alloy or other metal profiles. Background Art

[0002] As my country's industry is increasingly moving towards green, lightweight, high-speed and modernization, there is an increasing tendency to replace steel with aluminum due to the advantages of aluminum and aluminum alloys such as low density, high strength, corrosion resistance and good formability. Aluminum alloys are now widely used in aviation, aerospace, weapons, automobiles, ships, ships, machinery manufacturing, home appliances and electronic communications industries.

[0003] Aluminum profiles have strict requirements on size and mechanical properties, and related fields also have very demanding requirements on flatness and straightness. Extrusion molding is a very important link in the production process of aluminum profiles. After extrusion molding, the aluminum profiles need to be cooled by air cooling, mist cooling or water cooling. After being pulled by the traction machine and rapidly cooled, the aluminum profiles generally produce varying degrees of bending and distortion, and also produce uneven stress in the cross-sectional direction. Therefore, in the post-processing area of ​​aluminum profile extrusion production, the aluminum profiles need to be stretched and straightened to eliminate the bending and distortion of the aluminum profiles, and at the same time eliminate the uneven stress of the aluminum profiles along the cross-sectional direction. Therefore, the straightening process is also a process that cannot be ignored.

[0004] At present, the stretching straightening technology and equipment have achieved semi-automation, but the key processes are still completed manually. For example, the position of the stretching trolley needs to be constantly adjusted to adapt to profiles of different lengths, and automatic adjustment is currently not possible. The main reasons are as follows:

[0005] Users have high requirements for the surface quality of profiles, such as no scratches, etc. Therefore, in order to achieve automatic loading and unloading, only the ends of the profiles can be grabbed, and the grabbed parts can only be cut off as waste. In order to accurately grab and reduce waste, the position of the profile ends must be identified with high precision. However, there are the following difficulties:

[0006] 1. Due to the limited degree of automation of the corresponding previous process (extrusion molding), the extrusion length of the profile is not consistent and will fluctuate within a certain range. In addition, the length of the profile extruded from the tail of an aluminum rod may be only half of the length of the normal extruded profile. If an ordinary camera in a fixed position is used for recognition, the image resolution and field of view cannot be taken into account at the same time, and it is difficult to achieve target recognition in such a large range.

[0007] 2. If an ordinary camera is fixed on the stretching trolley for identification, it is difficult to remove the dynamic background of the video frame because the trolley is constantly moving and the target background is quite complex. In addition, some profile waste will be left in the on-site environment, which will greatly affect the robustness of the identification system.

[0008] 3. The profile specifications are diverse, and there are many customized products, which adds considerable difficulty to the model training for target recognition using ordinary cameras.

[0009] 4. Aluminum alloy profiles are relatively soft when they are just extruded. If the end of the profile is suspended in the air, it will sag. The amount of sag is related to the cantilever length and profile specifications of the profile, and is not a fixed value. This creates certain difficulties for automated clamping and feeding. Summary of the invention

[0010] The present invention aims to develop a simple, highly versatile (for a variety of products) and robust profile end detection method to achieve full automation of the stretch straightening production line. To this end, the present invention provides a profile end detection method based on a depth camera.

[0011] The technical solution of the present invention mainly includes the following contents:

[0012] 1. A profile end position detection device for automatic stretching and straightening of aluminum profiles based on a depth camera:

[0013] The device includes a mobile stretching trolley, a trolley track, a proximity switch, a cooling bed, a mobile depth camera, a fixed stretching trolley, and a fixed depth camera; the mobile stretching trolley can be movably installed on the trolley track along the trolley track, the mobile depth camera is installed on the mobile stretching trolley, the fixed stretching trolley is fixedly installed on the trolley track, and the fixed depth camera is installed on the fixed stretching trolley; a cooling bed is arranged on the side of the trolley track, multiple profiles are transported on the cooling bed, and multiple profiles are transported across multiple cooling beds, a single profile is arranged parallel to the trolley track, and the profile moves horizontally on the cooling bed from away from the trolley track to close to the trolley track; the mobile depth camera and the fixed depth camera are both facing the profile on the cooling bed.

[0014] A proximity switch is installed on the cooling bed at the side close to the trolley track, and the proximity switch is used to detect whether the profile is loaded in place.

[0015] The fixed depth camera is located above the end of the profile close to one end of the fixed stretching trolley.

[0016] 2. A method for detecting the end position of aluminum profiles in automatic stretching and straightening based on a depth camera:

[0017] Both the mobile depth camera and the fixed depth camera shoot downward images of multiple profiles near one end of themselves, and the images taken by the fixed depth camera are subjected to image analysis and processing to locate the end of the profile. The mobile depth camera moves with the mobile stretching trolley, and moves on the trolley track in a direction parallel to the length of the profile while shooting images in real time and performing real-time image analysis and processing to track and locate the end of the profile.

[0018] The image analysis and processing process is specifically as follows:

[0019] First, align the depth frame and the RGB frame;

[0020] Secondly, the area in the depth frame that contains the profile and its two sides parallel to the profile direction is intercepted to obtain the detection area A corresponding to the profile to be straightened, such as Figure 3 and Figure 5 Area A in the middle to avoid interference from other profiles;

[0021] Then, the depth frame information is used to perform background subtraction and then remove interference;

[0022] Finally, the end of the profile is identified and located, and the pixel coordinates of the end of the profile in the image are converted into camera coordinates with the camera as the spatial origin using the distortion parameters and coordinate conversion parameters corresponding to the RGB inside the camera.

[0023] When the background of the image taken by the mobile depth camera is removed, specifically:

[0024] In the depth frame of the mobile depth camera, only pixels whose pixel values ​​are in the range of (D-δ, D+β) are retained, where D represents the distance between the camera lens and the cooling bed surface, δ is a parameter value greater than the maximum thickness of the profile, and β is a reference threshold for profile sagging, which is a threshold set considering the sagging of the end of the profile, and can ensure that the end of the profile can be detected; based on this, the depth frame is binarized, and the binarization result is dot-multiplied with the RGB frame to remove most of the background.

[0025] When the background of the image taken by the fixed depth camera is removed, specifically:

[0026] First, invalid pixels in the detection area A of the depth frame are removed, that is, the multiple N is pre-set, and the pixel values ​​of all pixels with original pixel values ​​of 0 are traversed to a value N times greater than the distance D between the camera lens and the cooling bed surface, so as to highlight the minimum value among the valid pixel values.

[0027] Traverse the pixels again and find the minimum value S of the valid pixel values;

[0028] The pixels are traversed again, and the pixels with pixel values ​​in (S, S+δ+β) are filtered and retained, and the pixel values ​​of the remaining pixels are all changed to 0. δ is a parameter value greater than the maximum thickness of the profile, and β is a reference threshold for profile sagging, which is a threshold set considering the sagging of the end of the profile, and can ensure that the end of the profile can be detected. Based on this, the depth frame is binarized, and the binarization result is dot-multiplied with the RGB frame to remove most of the background.

[0029] The interference removal is to continue to remove interference from the depth frame, specifically by removing the interference of intruding objects and noise interference by removing smaller connected domains and Gaussian and mean filtering methods.

[0030] The mobile depth camera removes interference specifically as follows:

[0031] First, the connected areas with an area smaller than the preset threshold area are removed, that is, the image is segmented first, and then the connected areas with an area smaller than the preset threshold area in a part of the image are removed by using geometric features (area), and at the same time, the circumscribed rectangle of each remaining connected area is obtained to obtain the pixel width of the connected area in the y direction parallel to the length direction of the profile;

[0032] Then, the pixel width is converted into camera coordinates through the distortion parameters and coordinate transformation parameters of the RGB frame to obtain the actual width of the connected area, and the connected area whose actual width is less than ε times the width of the cooling bed is removed, where ε is a safety factor to ensure that the interference can be completely removed, and the value of ε can be between (1.5, 2).

[0033] This can remove the interference caused by the cooling bed when the profile is not in the field of view of the mobile depth camera.

[0034] If there are still connected areas after the above operations are completed, it means that the profile is already in the field of view of the mobile depth camera. At this time, it is very likely that the cooling bed 4 and the profile 5 form a large connected area, which is not conducive to the reading of the pixel coordinates of the end position of the profile. Therefore, the pixel width of the connected domain in the x direction is first detected. The side length of the circumscribed rectangle in the x direction can be calculated. If it is less than the width of the A area in the x direction, it means that there is only the profile in the field of view. If it is equal, it means that the profile and the cooling bed are in the field of view at the same time, and the interference of the cooling bed needs to be removed. Since the aluminum profile is silver-white and the cooling bed surface is dark gray, the RGB image in claim 6 is converted into a grayscale image and histogram equalized, and the maximum inter-class variance method is used for binarization. At this time, the interference of the cooling bed can be removed according to the brightness feature.

[0035] The images captured by the mobile depth camera are processed by image analysis to track and locate the end of the profile:

[0036] If there is no profile in the detection area A of the image at the beginning, that is, all pixels are 0, the mobile stretching trolley is controlled to move along the trolley track in a direction parallel to the profile and close to the rear end of the fixed depth camera ( Figure 2 -y direction) until the pixel point of the profile appears in the detection area A, then control the mobile stretching car to stop moving, start traversing the pixels in the image taken by the mobile depth camera, obtain the pixel coordinates of the end of the profile and convert them into camera coordinates, so as to obtain the position of the end of the profile relative to the depth camera to complete the positioning;

[0037] If there is a profile in the detection area A of the image at the beginning, but the end of the profile coincides with the boundary of the detection area A of the depth frame ( Figure 3 ), which means that the profile has covered the entire monitoring area in the y direction, then the mobile stretching trolley is controlled to move along the trolley track parallel to the profile and in the direction of the rear end of the principle fixed depth camera ( Figure 2 +y direction) until the end of the profile is within the boundary of the detection area A, which means that the end of the profile is already in the detection area. Then the mobile stretching car is controlled to stop moving and start traversing the pixels in the image taken by the mobile depth camera to obtain the pixel coordinates of the end of the profile and convert them into camera coordinates, thereby obtaining the position of the end of the profile relative to the depth camera to complete the positioning.

[0038] In the mobile depth camera:

[0039] First, preprocessing is performed. The data is real-time video stream data. Since the infrared lens and RGB lens fields of the mobile depth camera are inconsistent, the computer first performs pixel alignment on the depth image and the RGB image so that each pixel of the depth image can be correctly matched to the corresponding RGB image pixel.

[0040] After alignment, the background is removed using image area capture and depth information. Since there are multiple profiles in the field of view, this will affect the detection of the earliest extruded profile on the cooling bed. Therefore, it is necessary to perform regional capture of the image and only identify the area where the earliest extruded profile may exist. In addition, since the cooling bed surface is horizontal, the height is consistent with the track surface of the moving stretching trolley, and there is a large height difference between the cooling bed and other interferences in the captured field of view, it is very suitable to use depth information for background removal. Then, noise removal is performed to remove smaller connected areas. At the same time, the interference of the cooling bed is removed.

[0041] After preprocessing, theoretically only the pixel information of the profile remains. There are two situations:

[0042] (1) If there is no profile in the field of view, the mobile cart will move at a constant speed toward the rear end of the profile. The computer will continue to perform the above operations on each frame of depth data until the profile is detected to appear in the field of view. The mobile cart will stop moving, and then the end position will be obtained by converting the pixel coordinates with the camera coordinates, and the front end of the profile will begin to be grasped.

[0043] (2) If there is a profile in the field of view, it will first detect whether the end of the profile is beyond the field of view. If so, the mobile stretching carriage will move at a constant speed toward the front end of the profile until it detects that the front end of the profile is within the field of view. The mobile stretching carriage will then stop moving and then obtain the position of the end relative to the camera through the conversion between pixel coordinates and camera coordinates, and begin to grab the front end of the profile.

[0044] After the front end of the profile is detected, data points are collected within a distance s from the end. At this time, the end of the profile 5 is similar to a cantilever beam structure, and the distance s from the end can be approximately regarded as no bending caused by gravity. Assume that the pixel coordinates of a point on the profile within a distance s correspond to the camera coordinates (x i ,y i ,z i ). Then traverse the pixels in the x direction and find the coordinates of each row that are not 0. Assume that in the depth frame (parallel to the xoy plane in the camera coordinates) the yth i There are n pixel values ​​in the row whose coordinates are within the range of (D-δ, D+β), and the number of pixels in the x direction is 640, so the calculation is as follows

[0045]

[0046] Then you can get a series of pixels Assume that there are N such points within distance s.

[0047] Assume that the fitting line is z=cy+e, and use the least squares fitting parameters c and e.

[0048]

[0049] Wherein, z represents the spatial coordinate z axis with the camera as the origin, i.e., the height; c represents the slope of the end straight line fitted within a distance s from the end of the profile after sagging; y represents the spatial coordinate y axis with the camera as the origin, i.e., the coordinate in the axial direction of the profile; e represents the z-axis intercept of the fitting straight line; and i represents the sequence number of the pixel point;

[0050] The thickness of a certain specification profile being straightened is r. Now, by substituting the end coordinate y, we can get the sag h:

[0051] h=cy+e-D+r

[0052] Wherein, D represents the distance between the camera lens and the cooling bed surface.

[0053] The present invention utilizes two depth cameras to collect image information and depth information in real time, eliminates the background through the depth information, detects the end of the profile in real time and guides the movement of the stretching trolley, and at the same time detects the sagging amount of the profile after detecting the end of the profile, so as to guide the subsequent clamping and feeding process.

[0054] The beneficial effects of the present invention are:

[0055] 1. The present invention adopts a depth camera to more robustly detect the ends of the profiles, thereby realizing adaptive positioning of the stretching trolley, and realizing a simple, highly versatile (for a variety of products) and highly robust detection method for the ends of the profiles, which can be used in the fully automated design of the stretching and straightening production line.

[0056] 2. At the same time, it can detect the sagging amount of the profile end and provide data for subsequent clamping and feeding, which is of great significance for the research and development of the profile stretching and straightening automatic production line.

[0057] 3. The method of the present invention has the characteristics of simple implementation, wide applicability and high robustness. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 This is a diagram of the location of related equipment and components at one end of the mobile stretching trolley for the stretching and straightening production line;

[0059] Figure 2 This is a top view of one end of the mobile stretching trolley of the stretching straightening production line;

[0060] Figure 3 This is a schematic diagram of capturing the image area of ​​the moving depth camera at one end of the moving stretching trolley;

[0061] Figure 4 Position diagram of related equipment and components for fixing one end of the stretching trolley for the stretching straightening production line;

[0062] Figure 5 This is a schematic diagram of the image area of ​​the fixed depth camera at one end of the fixed stretching trolley;

[0063] Figure 6 This is a flow chart of image analysis processing of the detection method of the present invention.

[0064] Figure 7 Schematic diagram of the drooping of the profile end.

[0065] In the figure: 1 mobile stretching trolley, 2 trolley track, 3 proximity switch, 4 cooling bed, 5 profile, 6 mobile depth camera, 7 fixed stretching trolley, 8 fixed depth camera. DETAILED DESCRIPTION

[0066] The present invention will be further described below in conjunction with the accompanying drawings and specific implementations.

[0067] like Figure 1As shown, the device used for the specific implementation of the method includes a mobile stretching trolley 1, a trolley track 2, a proximity switch 3, a cooling bed 4, a mobile depth camera 6, a fixed stretching trolley 7, and a fixed depth camera 8; the mobile stretching trolley 1 and the fixed stretching trolley 7 are respectively arranged at both ends of the trolley track 2, the mobile stretching trolley 1 can be movably installed on the trolley track 2 along the trolley track 2, the mobile depth camera 6 is installed on the mobile stretching trolley 1, the fixed stretching trolley 7 is fixedly installed on the trolley track 2, and the fixed depth camera 8 is installed on the fixed stretching trolley 7; a plurality of cooling beds 4 are arranged on the side of the trolley track 2, the cooling bed 4 is provided with a conveyor belt, a plurality of profiles 5 are transported on the cooling bed 4, and the plurality of profiles 5 are transported across the plurality of cooling beds 4, a single profile 5 is arranged parallel to the trolley track 2, and the profile 5 is translated on the cooling bed 4 from away from the trolley track 2 to close to the trolley track 2; the mobile depth camera 6 and the fixed depth camera 8 are both facing the profile 5 on the cooling bed 4, and the two camera lenses are vertically downward and located at a certain height above the cooling bed.

[0068] A proximity switch 3 is installed on the side of the cooling bed 4 close to the trolley track 2. The proximity switch 3 is used to detect whether the profile 5 is loaded in place, that is, whether it reaches the position on the cooling bed 4 close to the trolley track 2. When it is detected that the profile 5 is loaded in place, the two cameras start to collect data simultaneously.

[0069] The fixed depth camera 8 is located above the end of the profile 5 close to one end of the fixed stretching trolley 7 .

[0070] Multiple profiles 5 are arranged in parallel, and are flush or nearly flush at one end near the fixed stretching trolley 7. Multiple profiles 4 have completely different lengths due to different required characteristics obtained by stretching processing, and the end positions at one end near the mobile stretching trolley 1 are very different due to different required characteristics obtained by stretching processing.

[0071] The present invention detects one end of the profile 4 which is relatively evenly positioned by fixing the stretching trolley 7, and detects the other end of the profile 4 which is relatively unevenly positioned by moving the stretching trolley 1, thereby realizing identification and positioning of both ends of the profile 4.

[0072] When a single extruded profile is extruded, since it is not extruded at the same time, the length of the profile will be inconsistent, which will cause the front end of the profile (the end that is extruded first) to be misaligned in the axial direction, and there will be a large float. In addition, the front end position of the tail of a rod when it is extruded is not certain, so there will be a length difference between the tail and the end position of other profiles.

[0073] The main function of the mobile stretching trolley is to clamp the front end of the profile, so before clamping, the position of the front end of the profile must be identified first. The front end of the profile is close to the fixed stretching trolley. The mobile depth camera is installed on the mobile stretching trolley, above the conveyor belt group used to place and load the profile on the cooling bed. The lens is vertically downward, and the field of view is facing the profile on the cooling bed. As the cooling bed moves, the extruded profiles can be arranged on the cooling bed surface with a certain interval between each other, and the cooling bed surface is horizontal.

[0074] The rear end of the profile is close to the fixed stretching trolley. When the profile is extruded, it needs to be cut off after one extrusion to facilitate the next extrusion. The cutting position is basically fixed, which makes the rear end of the profile relatively even after being transported to the cooling bed. According to statistics, it will fluctuate within a certain length (such as 15-20mm). Therefore, on one side of the fixed stretching trolley, the depth camera is not installed on the fixed stretching trolley, but is directly fixed on the ground. The installation height is basically the same as the other side. The lens is vertically downward, facing the cooling bed, and its field of view just covers the rear end of the profile.

[0075] A proximity switch is placed on the side of the cooling bed close to the rear end of the profile (the end after extrusion) to detect whether the earliest profile extruded on the cooling bed moves to the position of the proximity switch for straightening. When the proximity switch detects that the profile is in place, the mobile depth cameras on the mobile stretching trolley and the fixed stretching trolley begin to collect data, including depth information and RGB image information.

[0076] The mobile depth camera 6 and the fixed depth camera 8 both shoot downward images of multiple profiles 5 close to one end thereof. The mobile depth camera 6 and the fixed depth camera 8 contain RGB lenses and depth lenses. Each frame of the captured image includes a depth frame and an RGB frame. The image captured by the fixed depth camera 8 is subjected to image analysis and processing to locate the end of the profile 5. The mobile depth camera 6 moves with the mobile stretching trolley 1, and while moving on the trolley track 2 in a direction parallel to the length of the profile 5, it shoots images in real time and performs real-time image analysis and processing to track and locate the end of the profile 5.

[0077] The image taken by the fixed depth camera 8 is traversed after the background and interference are removed. The pixel coordinates of the end of the profile are found, and the pixel coordinates are converted into camera coordinates using the distortion parameters and conversion parameters of the RGB lens to complete the positioning.

[0078] The implementation process of the present invention is as follows:

[0079] 1. Inspection of the front end of the profile (extrude one end first)

[0080] like Figure 1As shown, the mobile depth camera 6 is installed on the mobile stretching trolley 1, with the lens facing vertically downward, directly facing the cooling bed 4. The cooling bed 4 moves the profile 5, and by setting the rotation of the cooling bed 4, the minimum spacing of the profile 5 on the cooling bed 4 can be controlled, and the distance between its components can be measured in advance.

[0081] Step 1, when the earliest extruded profile moves to the vicinity of the proximity switch 3, the cooling bed 4 stops moving, and the mobile depth camera 6 starts to collect data. The collected data is uploaded to the computer, and the image is pre-processed.

[0082] Step 2: The computer first performs pixel alignment on the depth frame and the RGB frame (some depth cameras can directly output aligned depth frames). After alignment, the image region is captured.

[0083] Step 3, such as Figure 3 As shown, the rectangular range is the field of view of the infrared lens of the mobile depth camera 6, and the field of view will include multiple profiles 5. The installation height of the mobile depth camera 6 is fixed, and the relative position of the proximity switch 3 in the radial direction of the profile 5 is fixed, and the earliest extruded profile will stop at the proximity switch 3. Therefore, with the proximity switch 3 as the center, the area width is set along the radial direction of the profile in the image and the detection area A is obtained by intercepting. The detection area A is the area to be detected at the end of the earliest extruded profile. In this detection area A, there are many background interferences distributed in the height direction. At this time, the depth information needs to be used to remove the background. The removal steps are as follows:

[0084] (1) After the mobile depth camera 6 is installed, the height difference D between the cooling bed 4 and the mobile depth camera 6 is obtained.

[0085] (2) Since the pixel value of the depth frame is the vertical distance between the spatial object and the camera lens plane, we only retain the pixel values ​​within the range of (D-δ, D+β) and modify the other depth frame pixels to 0, where δ is a value greater than the maximum thickness of the profile 5, and β is a threshold set to ensure that the profile end can be detected considering that the profile end is hanging in the air. At this point, most of the background can be removed. Then the depth frame is binarized.

[0086] Step 4, after background removal, interference must be removed. One is to remove noise, that is, use a correlation filter algorithm to remove Gaussian, peak or shot noise, and remove the noise of intrusive objects by removing smaller connected areas. Second, the cooling bed 4 is within the range of (D-δ, D+β), which will interfere with the identification of the profile end, so this interference needs to be removed. The width of the cooling bed 4 in the axial direction of the profile is d, and the specific method of removal is as follows:

[0087] (1) After background and noise removal, the depth frame is counted for connected regions and segmented.

[0088] (2) Obtain a circumscribed rectangle for each segmented connected area to obtain the pixel width of the connected area in the axial direction (y direction) of the profile 5.

[0089] (3) Obtain the distortion parameters of the infrared lens and RGB lens that have been burned into the mobile depth camera 6 and the conversion parameters of the pixel coordinates and the camera coordinates. Since the aligned depth pixel coordinates are not accurately converted into the camera coordinates, it is necessary to use the parameters of the RGB lens to convert the pixel coordinates into the camera coordinates. Finally, the pixel width in (2) can be converted to the width d in the real world. * .

[0090] (4) Just convert a series of d * Satisfy * The connected areas with a value < εd can be deleted, where ε is a safety factor to ensure that the interference can be completely removed, and the value of ε is between (1.5, 2).

[0091] (5) If there are still connected areas after the above operations are completed, it means that the profile 5 is already in the field of view of the mobile depth camera 6. At this time, it is very likely that the cooling bed 4 and the profile 5 are both in the field of view of the camera and merged to form a large connected area, which is not conducive to the coordinate reading of the end position of the profile. Therefore, the pixel width of the connected domain in the x direction is first detected, that is, by calculating the side length of the circumscribed rectangle in the x direction. If it is less than the width of the A area in the x direction, it means that there is only the profile in the field of view. If it is equal, it means that the profile and the cooling bed are in the field of view at the same time. At this time, the interference of the cooling bed needs to be removed. Since the aluminum profile is silver-white and the cooling bed surface is dark gray, the binarized depth frame processed in step 3 (2) is multiplied by the corresponding pixel of the RGB image, and then converted into a grayscale image and histogram equalization. The maximum inter-class variance method is used for binarization. At this time, the difference in brightness can be used to remove the interference of the cooling bed.

[0092] Step 5: After the image is preprocessed, only the pixel information of the profile remains in the depth frame. At this time, the end of profile 2 is located. There are two situations:

[0093] (1) All pixels are 0, indicating that there is no profile 5 in the detection area. The mobile stretching carriage 1 starts to move toward the rear end of the profile 5 ( Figure 2-y direction), and the above preprocessing is continuously performed until a pixel point appears in the detection area, which means that the profile 5 has entered the detection area by a length of about εd. At this time, the mobile stretching trolley 1 stops moving, and the computer starts to traverse the pixels, obtains the pixel coordinates of the end of the profile 5, and converts them into camera coordinates through coordinate conversion, thereby obtaining the y-direction distance L of the end of the profile 5 relative to the camera, and the distance of the end of the profile 5 from the mobile depth camera 6 in the x direction can be obtained through the relative position of the proximity switch 3 and the mobile depth camera 6. Thus, the relevant components can be guided to grab the end of the profile 5.

[0094] (2) There is a profile, but the end of the profile 5 coincides with the boundary of the depth frame detection area A ( Figure 3 ), which means that the profile has covered the entire monitoring area in the y direction, so the mobile stretching trolley 1 starts to move towards the front end of the profile 5 ( Figure 2 +y direction) until the end of the profile 5 is within the boundary of the depth frame detection area A, which means that the end of the profile is already in the detection area. Similarly, after pixel traversal and coordinate transformation, the position of the end of the profile 5 relative to the mobile depth camera 6 can be obtained, so that it can be accurately positioned for precise grasping.

[0095] 2. Inspection of the rear end of the profile (the last extruded end)

[0096] like Figure 4 As shown, the fixed depth camera 8 is installed on the fixed stretching trolley 7 and is fixed to the ground. The camera lens is vertically downward, facing the cooling bed 4. The installation height is consistent with the mobile stretching trolley on the other side. When the earliest extruded profile moves to the proximity switch 3, the cooling bed 4 stops moving, and the fixed depth camera 8 also starts to collect data.

[0097] After the collected data is uploaded to the computer, it is necessary to preprocess the data.

[0098] The detection of step 1, step 2 and the front end of the profile (extrude one end first) is consistent.

[0099] Step 3, such as Figure 4 As shown, the rear end of the profile 5 extends out of the cooling bed for a long time, so the field of view of the fixed depth camera ( Figure 5 ) can exclude the cooling bed 4, and only the profile 5 is in the field of view. Similarly, the detection area A is intercepted, and only the profile 5 is in the field of view. And the object closest to the lens must be the end of the rear end of the profile, and the next step is to remove the background. Since there is always the end of the profile in the detection area when collecting data, the method of removing the background is different from the detection of the front end of the profile (extrude one end first). The specific implementation method is as follows:

[0100] (1) Remove invalid pixels in the depth frame detection area A. Invalid pixels refer to the dead angles that cannot be illuminated by structured light. The pixel values ​​corresponding to these positions will be set to 0 by the camera. Then, the pixels with a pixel value of 0 will be changed to a value much larger than D, so as to highlight the minimum value among the valid pixel values.

[0101] (2) Traverse the pixels again and find the effective minimum pixel value S.

[0102] (3) The pixels are traversed again, and the pixel values ​​of (S, S+δ+β) are screened and retained, and the rest are all changed to 0. δ is a value greater than the maximum thickness of the profile 5, and β is a threshold value set to ensure that the end of the profile can be detected in consideration of the drooping of the end of the profile.

[0103] (4) The processed depth frame is binarized and multiplied with the RGB frame to complete the background removal.

[0104] Step 4: denoising after background removal, that is, removing smaller connected areas.

[0105] Step 5, locate the rear end of the profile 5. Traverse the pixels again to find the pixel coordinates of the end of the profile 5, and use the distortion parameters and conversion parameters of the RGB lens to convert the pixel coordinates into camera coordinates. In this way, the position of the rear end of the profile 5 relative to the fixed depth camera 8 is obtained, thereby guiding the fixed stretching trolley 7 to grab the rear end of the profile.

[0106] 3. Detection of sagging amount at the end of profile

[0107] The purpose of detecting the ends of the profile 5 on both sides is to realize the automatic clamping and feeding of the profile 5 in the subsequent process. Since the profile 5 is relatively soft after extrusion, and since the conveyor belt interval of the cooling bed 4 is 1.2 meters, which is relatively large, for the profile 5 with smaller specifications, the front end of the profile (one end is extruded first) may be suspended too long, which will cause sagging. Therefore, in order to make the subsequent process of grabbing the profile 5 proceed smoothly, it is necessary to measure the sagging amount of the end of the profile 5. It is known that the ends of the profile 5 on both sides can be detected in parts 1 and 2 respectively, and the position of the ends relative to the depth camera, that is, the camera coordinates, can be calculated, and the profile 5 is basically in a plane parallel to the plane yoz in the camera coordinates. Therefore, in order to avoid interference such as noise and improve measurement stability and accuracy, the depth image data is used to achieve straight line fitting in the camera coordinates.

[0108] Figure 7The figure is a schematic diagram of the end of the profile 5 sagging. After the front end of the profile is detected, data points are collected within a distance s from the end. At this time, the end of the profile 5 is similar to a cantilever beam structure, and the distance s from the end can be approximately regarded as no bending caused by gravity. Assume that the pixel coordinates of a point on the profile within a distance s correspond to the camera coordinates (x i ,y i ,z i ). Then traverse the pixels in the x direction and find the coordinates of each row that are not 0. Assume that in the depth frame (parallel to the xoy plane in the camera coordinates) the yth i There are n pixel values ​​in the row whose coordinates are within the range of (D-δ, D+β), and the number of pixels in the x direction is 640, so the calculation is as follows

[0109]

[0110] Then you can get a series of pixels Assume that there are N points within distance s.

[0111] Assume that the fitting line is z=cy+e, and use the least squares fitting parameters c and e.

[0112]

[0113] Among them, z represents the spatial coordinate z-axis with the camera as the origin, that is, the height, c represents the slope of the terminal straight line fitted within a distance s from the end after the profile sags, y represents the spatial coordinate y-axis with the camera as the origin, that is, the coordinate in the axial direction of the profile, e represents the z-axis intercept of the fitting straight line, and i represents the serial number of the pixel point.

[0114] Assuming that the thickness of a certain specification profile being straightened is r, then by inputting the end coordinate y, the sag h can be obtained:

[0115] h=cy+e-D+r

[0116] h can be used to guide the subsequent clamping and loading process.

Claims

1. A profile end position detection device for aluminum profile automatic stretching and straightening based on a depth camera, characterized in that: The invention comprises a mobile stretching trolley (1), a trolley track (2), a proximity switch (3), a cooling bed (4), a mobile depth camera (6), a fixed stretching trolley (7), and a fixed depth camera (8); the mobile stretching trolley (1) can be movably installed on the trolley track (2), the mobile depth camera (6) is installed on the mobile stretching trolley (1), the fixed stretching trolley (7) is fixedly installed on the trolley track (2), and the fixed depth camera (8) is installed on the fixed stretching trolley (7); a cooling bed (4) is arranged on the side of the trolley track (2), a plurality of profiles (5) are transported on the cooling bed (4), and the plurality of profiles (5) are transported across a plurality of cooling beds (4), a single profile (5) is arranged parallel to the trolley track (2), and the profile (5) on the cooling bed (4) moves horizontally from away from the trolley track (2) to close to the trolley track (2); the mobile depth camera (6) and the fixed depth camera (8) are both facing the profile (5) on the cooling bed (4); A method for detecting the end position of an aluminum profile in automatic stretching and straightening of an aluminum profile in the device for detecting the end position of an aluminum profile in automatic stretching and straightening of an aluminum profile comprises: The mobile depth camera (6) and the fixed depth camera (8) both shoot downward images of the ends of the plurality of profiles (5) close to their own ends, and perform image analysis and processing on the images shot by the fixed depth camera (8) to locate the ends of the profiles (5); the mobile depth camera (6) moves with the mobile stretching trolley (1), and while moving on the trolley track (2) in a direction parallel to the length of the profiles (5), it shoots images in real time and performs real-time image analysis and processing to track and locate the ends of the profiles (5); The image analysis and processing process is specifically as follows: First, align the depth frame and the RGB frame; Secondly, the area in the depth frame containing the profile (5) and the area on both sides thereof parallel to the direction of the profile (5) is intercepted to obtain a detection area A corresponding to the profile to be straightened; Then, the depth frame information is used to perform background subtraction and then remove interference; Finally, the end of the profile is identified and located, and the pixel coordinates of the end of the profile in the image are converted into camera coordinates with the camera as the spatial origin using the distortion parameters and coordinate conversion parameters corresponding to the RGB inside the camera.

2. The device for detecting the end position of aluminum profiles in automatic stretching and straightening based on a depth camera according to claim 1, characterized in that: A proximity switch (3) is installed on the cooling bed (4) at the side close to the trolley track (2), and the proximity switch (3) is used to detect whether the profile (5) is loaded in place.

3. The device for detecting the end position of aluminum profiles in automatic stretching and straightening based on a depth camera according to claim 1, characterized in that: The fixed depth camera (8) is located above one end of the profile (5) close to the fixed stretching trolley (7).

4. The device for detecting the end position of aluminum profiles in automatic stretching and straightening based on a depth camera according to claim 1, characterized in that: When the background of the image captured by the mobile depth camera (6) is removed, specifically: In the depth frame of the mobile depth camera (6), only pixels whose pixel values ​​are in the range of (D-δ, D+β) are retained, where D represents the distance between the camera lens and the cooling bed surface, δ is a parameter value greater than the maximum thickness of the profile (5), and β is a reference threshold for the profile sagging. Based on this, the depth frame is binarized, and the binarization result is multiplied by the RGB frame to remove the background.

5. The device for detecting the end position of aluminum profiles in automatic stretching and straightening based on a depth camera according to claim 1, characterized in that: When background removal is performed on the image captured by the fixed depth camera (8), specifically: (1) First, invalid pixels in the detection area A of the depth frame are removed, that is, a multiple N is preset, and the pixel values ​​of all pixels with original pixel values ​​of 0 are changed to a value N times greater than the distance D between the camera lens and the cooling bed surface; Traverse the pixels again and find the minimum value S of the valid pixel values; The pixels are traversed again, and the pixels with pixel values ​​in (S, S+δ+β) are filtered and retained, and the pixel values ​​of the remaining pixels are all changed to 0. δ is a parameter value greater than the maximum thickness of the profile (5), and β is the reference threshold for the profile sagging. The depth frame is binarized based on this, and the binarization result is multiplied with the RGB frame to remove the background.

6. The device for detecting the end position of aluminum profiles in automatic stretching and straightening based on a depth camera according to claim 1, characterized in that: The interference removal is to continue to remove interference from the depth frame, specifically to remove the interference of intruding objects and noise interference by removing small connected domains and Gaussian and mean filtering methods.

7. The device for detecting the end position of aluminum profiles in automatic stretching and straightening based on a depth camera according to claim 1, characterized in that: The mobile depth camera (6) removes interference specifically by: First, the connected areas with an area smaller than the preset threshold area are removed, that is, the image is segmented first, and then the connected areas with an area smaller than the preset threshold area in a part of the image are removed by using geometric features, and at the same time, the circumscribed rectangle of each remaining connected area is obtained to obtain the pixel width of the connected area in the y direction parallel to the length direction of the profile; Then, the pixel width is converted into camera coordinates through the distortion parameters and coordinate transformation parameters of the RGB frame to obtain the actual width of the connected area, and the connected areas whose actual width is less than ε times the cooling bed width are removed.

8. The device for detecting the end position of aluminum profiles in automatic stretching and straightening based on a depth camera according to claim 1, characterized in that: The image captured by the mobile depth camera (6) is subjected to image analysis and processing to track and locate the end of the profile (5): If there is no profile in the detection area A of the image at the beginning, the mobile stretching trolley (1) is controlled to move on the trolley track (2) in a direction parallel to the profile (5) and close to the fixed depth camera (8) until the pixel point of the profile appears in the detection area A, and then the mobile stretching trolley (1) is controlled to stop moving and start traversing the pixels in the image taken by the mobile depth camera (6), and the pixel coordinates of the end of the profile (5) are obtained and converted into camera coordinates, so as to obtain the position of the end of the profile (5) relative to the depth camera to complete the positioning; If there is a profile in the detection area A of the image at the beginning, but the end of the profile coincides with the boundary of the detection area A of the depth frame, the mobile stretching trolley (1) is controlled to move on the trolley track (2) in a direction parallel to the profile (5) and the fixed depth camera (8) until the end of the profile is within the boundary of the detection area A. The mobile stretching trolley (1) is then controlled to stop moving and start traversing the pixels in the image taken by the mobile depth camera (6), obtaining the pixel coordinates of the end of the profile (4) and converting them into camera coordinates, thereby obtaining the position of the end of the profile (5) relative to the depth camera to complete the positioning.

Images