A loading and unloading operation scheduling method for CNC machining

Abstract

The application discloses a kind of loading and unloading operation scheduling methods for CNC machining, it is related to industrial control field, comprising: obtaining the multiple polarizing angle images collected under the acquisition of polarizing angle switching and lens transmittance value, generate fusion image by image fusion and transmittance enhancement;Based on the edge distribution value and brightness gradient value generated by fusion image, the occlusion intensity map is constructed by edge distribution value and brightness gradient value and the occlusion area is removed, to generate unoccluded image;Based on unoccluded image, and the interference image sample generated by the cooling liquid pattern and metal scrap pattern collected, the recognition structure is trained by interference image sample and the workpiece edge value and workpiece category value are identified;The application realizes the stable generation of fusion image and the structure maintenance of unoccluded image under the CNC machining environment of metal reflection and oil mist interference coexistence, and the positioning accuracy and scheduling stability of robot loading and unloading operation under complex optical conditions are maintained in combination with lens pollution result.

Description

Technical Field

[0001] This invention relates to the field of industrial control technology, and more specifically to a loading and unloading operation scheduling method for CNC machining. Background Technology

[0002] With the continuous development of CNC machining technology and automated production lines, CNC machining centers are gradually forming a linkage structure with robotic loading and unloading units to realize automatic loading and unloading and continuous processing of workpieces. In such application scenarios, visual recognition usually undertakes key tasks such as workpiece positioning, category differentiation and posture determination, and is directly related to robot motion control parameters. By driving the grasping posture and handling path through visual data, the cycle stability and human-machine collaboration efficiency can be improved. Therefore, maintaining the spatial consistency, brightness stability and structural boundary clarity of image information during the processing has become the basic premise for ensuring the accuracy of loading and unloading operation scheduling.

[0003] Currently, during the workpiece loading and unloading recognition process inside a CNC machining center, the images are prone to phenomena such as high reflectivity areas, local brightness attenuation, and debris occlusion due to the superposition of complex optical factors such as metal surface reflection, cutting fluid splashing, and oil mist dispersion. This causes distortion of grayscale variation features and edge structure information in the image, resulting in incomplete edge contour extraction or category judgment deviation. When the recognition results are used as the input basis for grasping pose and motion path, it can easily lead to the accumulation of posture matching errors or grasping position offset, affecting the continuity and positioning accuracy of loading and unloading operations.

[0004] Secondly, during continuous operation of the processing cycle, the lens surface may experience a decrease in light transmittance due to oil mist deposition. The average brightness distribution value of the continuous image will drift over time. When stable brightness sections are not distinguished, the visual recognition confidence state is difficult to match with the changes in the real optical environment. Consequently, even under conditions of image contrast decay, the grabbing pose and category judgment results are still output, causing the loading and unloading operation scheduling instructions to become disconnected from the actual visual quality. As the degree of contamination gradually increases, it is easy to cause inappropriate selection of grabbing torque parameters and handling path parameters, reducing the operational stability and execution consistency of the robot during loading and unloading. Summary of the Invention

[0005] To address the aforementioned technical problems, this invention provides a loading and unloading operation scheduling method for CNC machining, comprising: A loading and unloading operation scheduling method for CNC machining, the method comprising: Acquire multi-polarization angle images and lens transmittance values ​​under polarization angle switching, and generate a fused image through image fusion and transmittance enhancement; Based on the fused image, edge distribution values ​​and brightness gradient values ​​are generated. An occlusion intensity map is constructed using the edge distribution values ​​and brightness gradient values, and occlusion areas are removed to generate an occlusion-free image. Interference image samples are generated based on unobstructed images, as well as collected coolant and metal debris patterns. The recognition structure is trained using interference image samples to identify workpiece edge values ​​and workpiece category values. The lens brightness mean distribution value is obtained and compared with the set contamination threshold to generate lens contamination results. Based on the lens contamination results and the workpiece category value, the target pose recognition result is generated. Based on the final target pose recognition result, workpiece category value and lens contamination result, a loading and unloading operation scheduling instruction is generated, and the loading and unloading operation execution strategy is branched according to the contamination level corresponding to the lens contamination result.

[0006] Furthermore, the steps for generating the fused image are as follows: By switching the polarization angle, images from multiple polarization angles are acquired, and these images are aligned according to the acquisition time sequence to generate a multi-angle image group. Spot suppression processing is performed on the pixel brightness distribution of each image in the multi-angle image group to obtain a spot-suppressed image group; Obtain the lens transmittance value and perform transmittance enhancement processing on the splatter suppression image group to generate a transmittance enhancement image group; A pixel-level image fusion operation is performed on the light-enhancing image group to generate a fused image.

[0007] Furthermore, the steps for performing the transmittance enhancement treatment are as follows: A transmittance matching calculation is performed based on the lens transmittance value and the average brightness of each image in the bokeh suppression image group to generate an image brightness compensation factor; The pixel brightness values ​​of each image in the light spot suppression image group are adjusted by the image brightness compensation factor to obtain the brightness compensation image group, which is then output as the light transmission enhancement image group.

[0008] Furthermore, the steps for constructing the occlusion intensity map are as follows: Based on the grayscale changes of each pixel in the fused image, the grayscale difference between adjacent pixels is calculated to obtain the brightness gradient value. Edge detection is performed on the fused image to extract all edge pixels, count the number of edge pixels in a unit area, and generate edge distribution values. An occlusion intensity map is generated by weighting and superimposing edge distribution values ​​and brightness gradient values.

[0009] Furthermore, the steps for performing the weighted summation calculation are as follows: Normalize the brightness gradient values ​​to obtain normalized brightness gradient values; The edge distribution values ​​are used to perform high-response region extraction operations to filter out high-density edge region values; An occlusion intensity map is generated by performing a weighted fusion calculation on the normalized brightness gradient value and the edge high-density region value.

[0010] Furthermore, the steps for removing obstructed areas are as follows: Based on the pixel value distribution in the occlusion intensity map, set the occlusion determination threshold; Based on the occlusion determination threshold, the occlusion intensity map is processed to separate regions and mark the set of pixels in the high occlusion region. Remove the set of pixels with high occlusion from the fused image to output an unoccluded image.

[0011] Furthermore, the steps for generating interfering image samples are as follows: Collect images of coolant patterns and metal debris patterns, and establish a set of images of interfering elements based on the image content; Based on the location coordinates and edge contour range of the target region in the unobstructed image, the area of ​​interference image overlay is determined; Perturbation fusion processing is performed on the superimposed regions of interfering images in an unoccluded image using a set of interfering element images to generate interfering image samples.

[0012] Furthermore, the steps for training the recognition structure and identifying workpiece edge values ​​and workpiece category values ​​are as follows: The interference image samples are paired with the unoccluded images to form training image pairs, and a training image pair set is constructed. Based on the training images, perform multiple rounds of iterative training on the images and corresponding annotations in the set to generate a recognition structure; By recognizing the structure, feature extraction and recognition processing are performed on the unobstructed image to generate workpiece edge values ​​and workpiece category values, respectively.

[0013] Furthermore, the steps for generating lens contamination results are as follows: Based on the fused images of multiple consecutive frames in the image acquisition sequence, the average brightness value of each frame is calculated to generate a brightness mean sequence. Perform multi-scale trend analysis based on the mean brightness value sequence to extract the low-frequency stable segment in the mean brightness value change curve; The difference between the average brightness of the current frame and the average value of the low-frequency stable section is calculated and compared with the set pollution threshold to generate the lens pollution result.

[0014] Compared with the prior art, the beneficial effects of the present invention are as follows: This invention achieves stable generation of fused images in CNC machining environments with high metal reflectivity and oil mist by co-correcting multi-polarization angle images and lens transmittance values. It also constructs an occlusion intensity map by combining edge distribution values ​​and brightness gradient values ​​to form an unoccluded image. This allows visual data to maintain structural integrity and brightness consistency under complex optical interference conditions, avoiding misjudgments caused by cutting fluid splashes and metal chips in workpiece contour recognition. At the same time, it provides a stable input basis for the subsequent extraction of workpiece edge values ​​and workpiece category values, thereby ensuring the spatial accuracy and attitude matching reliability of the final target pose recognition result. Furthermore, this invention also dynamically determines the lens contamination result and contamination level, and associates the final target pose recognition result, workpiece category value and loading and unloading operation scheduling instructions for control. This enables the grasping pose parameters, grasping torque parameters and transport path parameters to be adjusted according to the changes in the average distribution value of lens brightness. Combined with the review shooting strategy parameters, the image recognition confidence state is filtered. Thus, the execution sequence stability and execution parameter consistency of the robot loading and unloading operation are maintained under the conditions of continuous changes in processing cycle and fluctuations in visual conditions. This reduces the risk of grasping deviation caused by visual inaccuracy and forms a closed-loop control relationship between visual recognition, posture matching and loading and unloading operation scheduling. In summary, this invention achieves stable generation of fused images and preservation of the structure of unobstructed images in a CNC machining environment where metal reflection and oil mist interference coexist. Furthermore, it incorporates lens contamination results to correlate and control the final target pose recognition results with loading and unloading operation scheduling instructions. This ensures the reliability of workpiece edge values ​​and workpiece category values ​​recognition, as well as the consistency of the execution of grasping pose parameters and transport path parameters, thereby maintaining the positioning accuracy and scheduling stability of robot loading and unloading operations under complex optical conditions. Attached Figure Description

[0015] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this invention. For those skilled in the art, other drawings can be obtained based on these drawings.

[0016] Figure 1 A flowchart of a loading and unloading operation scheduling method for CNC machining provided in an embodiment of the present invention. Detailed Implementation

[0017] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0018] Please see Figure 1 As shown in the figure, this embodiment discloses a loading and unloading operation scheduling method for CNC machining, including: S11: Acquire multi-polarization angle images and lens transmittance values ​​under polarization angle switching, and generate a fused image through image fusion and transmittance enhancement; In one specific embodiment, an electronic rotating polarizer installed at the front end of an industrial camera is first used to switch between multiple preset polarization angles, and the shutter is triggered at each polarization angle to acquire multi-polarization angle images under the switching polarization angle.

[0019] The lens transmittance value is obtained by measuring the proportion of light transmitted through the lens surface using a light intensity sensor deployed on the side of the lens. A fused image is generated through image fusion and transmittance enhancement to eliminate the impact of reflections and oil mist scattering from metal surfaces on visual recognition within the CNC machining center.

[0020] It should be noted that the multi-polarization angle image refers to spatial domain image data acquired under multiple polarization direction conditions, including but not limited to image data acquired under polarization directions of 0°, 45°, 90° and 135°. The multi-polarization angle image does not represent a multi-frequency image in the frequency domain or multispectral sense.

[0021] Specifically, the steps for generating the fused image are as follows: S111: Acquire images from multiple polarization angles by switching polarization angles, align the images from multiple polarization angles according to the acquisition time sequence, and generate a multi-angle image group; In one specific embodiment, multi-polarization angle images are acquired by switching polarization angles. These multi-polarization angle images include original images captured under four polarization directions: 0°, 45°, 90°, and 135°. The multi-polarization angle images are aligned according to the acquisition time sequence. Using the first acquired image as the reference coordinate system, common feature points of each frame are extracted through feature matching.

[0022] Specifically, the calculation logic for image alignment of multi-polarization angle images is as follows: calculate the homography transformation matrix between the current frame image and the reference coordinate system image; use the homography transformation matrix to perform perspective transformation on the current frame image to complete pixel coordinate alignment; resample all transformed images into the same pixel grid; index and mark the resampled images according to the polarization angle to generate a multi-angle image group.

[0023] It should be noted that the establishment of the homography transformation matrix is ​​based on the premise that the processing platform where the workpiece is located is an approximate plane and the industrial camera is installed in a fixed posture. This is to adapt to the small pose offset alignment requirements caused by the robot arm. Image alignment can eliminate the small pose offset caused by the robot arm's movement and ensure that spatial pixels under different polarization angles can correspond one-to-one.

[0024] S112: Perform spot suppression processing based on the pixel brightness distribution of each image in the multi-angle image group to obtain a spot-suppressed image group; In one specific embodiment, spot suppression processing is performed based on the pixel brightness distribution of each image in the multi-angle image group.

[0025] Specifically, the calculation logic for performing spot suppression processing is as follows: extract the pixel brightness value of each coordinate position in the multi-angle image group under different polarization angles, and calculate the variance of all pixel brightness values ​​at that coordinate position; A light spot determination threshold is set, which is obtained by collecting the brightness fluctuation range of a standard stainless steel workpiece under high light environment.

[0026] It should be noted that the light spot determination threshold is determined as follows: under standard studio lighting conditions, multi-frame brightness variance statistics are performed on standard metal samples with reflectivity greater than or equal to 85%, and the maximum brightness standard deviation obtained from the statistics is defined as the light spot determination threshold.

[0027] If the variance of the pixel brightness value at a certain coordinate position is greater than or equal to the spot determination threshold, then that position is determined to be a highly reflective area.

[0028] In highly reflective areas, the minimum pixel brightness value among multiple pixel brightness values ​​corresponding to that coordinate position is selected as the output pixel value for that position. If the variance of the pixel brightness values ​​is less than the spot determination threshold, the original pixel brightness distribution remains unchanged. When performing the above processing on each frame of the multi-angle image group, the output pixel value calculated with the polarization angle at each coordinate position is used to replace the pixel brightness at that coordinate position, and the number of frames of the output image is kept consistent with the input multi-angle image group, thereby obtaining the spot suppression image group.

[0029] It should be noted that by utilizing the intensity difference of reflected light from a metal surface at different polarization angles, extremely bright specular reflection spots can be effectively filtered out.

[0030] S113: Obtain the lens transmittance value and perform transmittance enhancement processing on the splatter suppression image group to generate a transmittance enhancement image group; In one specific embodiment, a lens transmittance value is obtained, which reflects the degree to which oil mist accumulated in the CNC machining center attenuates the light transmission of the lens. Transmittance enhancement processing is then performed on the light spot suppression image group to generate a transmittance enhanced image group.

[0031] Specifically, the steps for performing transmittance enhancement treatment are as follows: S113.1: Perform transmittance matching calculation based on the lens transmittance value and the average brightness of each image in the spot suppression image group to generate an image brightness compensation factor; In one specific embodiment, a transmittance matching calculation is performed based on the lens transmittance value and the average brightness of each image in the spot suppression image group to generate an image brightness compensation factor.

[0032] Specifically, the calculation logic for generating the image brightness compensation factor is as follows: A standard brightness reference value is obtained by photographing an 18% neutral grayscale card conforming to the ISO 12233 standard in a clean environment (free from oil fog and with a constant illumination of 500 lux ± 10%), and then taking the average grayscale value of the central area of ​​the grayscale card image. By clearly defining the physical specifications of the reference source, a highly consistent benchmark is ensured for different batches of equipment during calibration, thereby avoiding algorithm failure due to an unclear reference source. The initial gain value is obtained by dividing the standard brightness reference value by the average brightness value of each image in the spot-suppressed image group.

[0033] Dividing the initial gain value by the lens transmittance value yields the image brightness compensation factor.

[0034] It should be noted that the range of lens transmittance value is 0 to 1 after normalization. The image brightness compensation factor is a frame-level compensation factor. For each frame in the spot suppression image group, a corresponding compensation factor is generated based on its average brightness value and used to adjust the pixel brightness value of the corresponding frame image.

[0035] The normalization method involves calculating the ratio of the currently measured light intensity value to the light intensity value in a standard accessible environment.

[0036] The larger the image brightness compensation factor, the darker the current lens pollution or the darker the lighting environment.

[0037] S113.2: Adjust the pixel brightness values ​​of each image in the spot suppression image group by using the image brightness compensation factor to obtain the brightness compensation image group, and output the brightness compensation image group as the light transmission enhancement image group.

[0038] In one specific embodiment, the pixel brightness values ​​of each image in the spot suppression image group are adjusted by an image brightness compensation factor.

[0039] Specifically, the calculation logic for adjusting pixel brightness values ​​is as follows: multiply the pixel brightness value of each pixel in the spot suppression image group with the image brightness compensation factor. Check if the calculation result is greater than the maximum pixel size of 255. If the result is greater than or equal to 255, reset the brightness value of that pixel to 255. If the result of the calculation is less than 255, the result is retained as the new pixel value; The brightness-compensated image group is obtained by iterating through all pixels in the image group. The brightness-compensated image group is then output as a light-enhancing image group.

[0040] S114: Perform pixel-level image fusion operation through the light-enhancing image group to generate a fused image.

[0041] In one specific embodiment, pixel-level image fusion is performed using a group of light-enhancing images.

[0042] Specifically, the calculation logic for performing pixel-level image fusion is as follows: decompose multiple frames of images in the light-enhancing image group based on discrete wavelet transform to obtain the low-frequency and high-frequency components of each frame.

[0043] For low-frequency components, an average fusion method is used, which calculates the arithmetic mean of the low-frequency coefficients at corresponding positions across all frames. For high-frequency components, an absolute value maximization fusion method is used, which selects the high-frequency coefficients with the most prominent texture features from multiple frames. The fused low-frequency and high-frequency components are then subjected to inverse wavelet transform to form a single-frame fused image.

[0044] It should be noted that wavelet transform fusion can preserve edge details under different polarization angles while maintaining the overall brightness consistency of the image. The wavelet basis can be either Haar wavelet or Daubechies wavelet.

[0045] S12: Generate edge distribution values ​​and brightness gradient values ​​based on the fused image, construct an occlusion intensity map using the edge distribution values ​​and brightness gradient values, remove occluded areas, and generate an occluded image; In one specific embodiment, the grayscale variation features of the fused image in the spatial domain are first extracted. The brightness gradient value is calculated using these grayscale variation features. Edge distribution values ​​are statistically generated using structured edge information in the fused image. The edge distribution values ​​and brightness gradient values ​​are spatially mapped and superimposed to construct an occlusion intensity map. Based on the occlusion intensity map, visual occlusions caused by cutting fluid splashes or large debris in the fused image are identified and removed.

[0046] Specifically, the steps for constructing an occlusion intensity map are as follows: S121: Based on the gray level change of each pixel in the fused image, calculate the gray level difference between adjacent pixels to obtain the brightness gradient value; In one specific embodiment, the brightness gradient value is calculated based on the grayscale changes of each pixel in the fused image.

[0047] Specifically, the calculation logic for the brightness gradient value is as follows: Select a 3×3 pixel neighborhood centered on the current pixel; The horizontal grayscale difference is obtained by performing convolution operation with the pixel neighborhood using the horizontal operator, and the vertical grayscale difference is obtained by performing convolution operation with the pixel neighborhood using the vertical operator. The sum of the squares of the horizontal grayscale difference and the vertical grayscale difference is then calculated. The square root of the above sum of squares is taken, and the result is used as the brightness gradient value of the pixel.

[0048] It should be noted that the brightness gradient value can reflect the intensity of the outline of objects in the image and is an important basis for distinguishing the workpiece edge from background interference.

[0049] S122: Perform edge detection based on the fused image, extract all edge pixels, count the number of edge pixels in a unit area, and generate edge distribution values; In one specific embodiment, edge detection is performed based on the fused image. Specifically, the calculation logic for generating edge distribution values ​​is as follows: the fused image is subjected to multi-level filtering and suppression processing using the Canny edge detection operator. Pixels identified as edges are marked as 1. Non-edge pixels are marked as 0. A statistical window of size 16×16 is slid across the image.

[0050] Accumulate the number of all edge pixels marked as 1 within the statistical window; The number of records within each statistical window is used as the marginal distribution value at the center of that window. It should be noted that the edge distribution value represents the local texture complexity of the image. Metal debris in CNC machining usually exhibits the characteristic of abnormally high edge distribution values. The 3×3 pixel neighborhood is used to obtain local contour intensity information of pixel-level grayscale changes, and the 16×16 statistical window is used to obtain regional edge density information, so as to distinguish the abnormal texture area caused by workpiece contour edges and debris accumulation through feature descriptions at different spatial scales.

[0051] S123: Generate an occlusion intensity map by performing weighted superposition calculations using edge distribution values ​​and brightness gradient values.

[0052] In one specific embodiment, an occlusion intensity map is generated by performing a weighted superposition calculation using edge distribution values ​​and brightness gradient values.

[0053] Specifically, the steps for performing the weighted summation calculation are as follows: S123.1: Perform normalization on the brightness gradient value to obtain the normalized brightness gradient value; In one specific embodiment, the brightness gradient values ​​are normalized.

[0054] Specifically, the calculation logic for performing normalization is as follows: retrieve the maximum and minimum values ​​of the brightness gradient values ​​across the entire image, subtract the minimum value from the brightness gradient value of each pixel to obtain the brightness difference, calculate the difference between the maximum and minimum values, and divide the brightness difference by the above difference to obtain the normalized brightness gradient value with a value between 0 and 1.

[0055] S123.2: Perform high-response region extraction operation on the edge distribution values ​​to filter out the high-density edge region values; In one specific embodiment, a high-response region extraction operation is performed on the edge distribution values.

[0056] Specifically, the computational logic for performing the high-response region extraction operation is as follows: A high-density response threshold is set, which is obtained by statistically analyzing the average highest pixel density at the edge of a standard workpiece.

[0057] Traverse the edge distribution values ​​of the entire graph.

[0058] If the edge distribution value is greater than or equal to the high-density response threshold, then the value is retained as the edge high-density region value.

[0059] If the edge distribution value is less than the high-density response threshold, then the edge high-density region value at that location is assigned a value of 0.

[0060] It should be noted that this step uses nonlinear suppression to filter out the normal edges of the workpiece itself and lock the abnormal high-frequency region caused by debris accumulation.

[0061] S123.3: Generate an occlusion intensity map by performing a weighted fusion calculation on the normalized brightness gradient value and the edge high-density region value.

[0062] In one specific embodiment, a weighted fusion calculation is performed by combining the normalized brightness gradient value with the edge high-density region value.

[0063] Specifically, the calculation logic for performing the weighted fusion calculation is as follows: obtain a weight adjustment coefficient, which is preset according to the cutting load of the current machining process; The cutting load is obtained in real time by accessing the PLC status register of the CNC system through the MTConnect protocol or OPCUA communication interface; the system reads the ratio of the spindle load current value to the rated current value in real time as the cutting load percentage. The load-weight mapping table divides the cutting load into three intervals: low, medium, and high, and each interval corresponds to a preset weight adjustment coefficient. The normalized brightness gradient value at each pixel location is multiplied by the value of the high-density edge region. The result of the multiplication operation is then multiplied by a weight adjustment coefficient, and the final product is used as the occlusion intensity of the corresponding pixel. The occlusion intensities of all pixels constitute an occlusion intensity map.

[0064] It should be noted that the weight adjustment coefficient is positively correlated with the cutting load and is obtained by querying a preset load-weight mapping table. The load-weight mapping table is established as follows: Beforehand, fused images under different cutting load conditions are acquired and corresponding occlusion intensity maps are constructed. The false detection rate or false negative rate caused by occlusion removal under each load condition is statistically analyzed. The cutting load is divided into intervals based on the trend of false detection rate or false detection rate with load variation; A weight adjustment coefficient is configured for each load range, so that the high load range corresponds to a larger weight adjustment coefficient, in order to form a correspondence table between cutting load and weight adjustment coefficient.

[0065] Specifically, the steps for removing obstructed areas are as follows: S124: Set the occlusion determination threshold based on the pixel value distribution in the occlusion intensity map; In one specific embodiment, an occlusion determination threshold is set based on the pixel value distribution in the occlusion intensity map.

[0066] Specifically, the logic for setting the occlusion determination threshold is as follows: A histogram of the probability distribution of all pixel values ​​in the occlusion intensity map is plotted. The inter-class variance of the histogram is calculated using the Otsu algorithm. The pixel grayscale level that maximizes the inter-class variance is selected as the occlusion determination threshold.

[0067] S125: Perform region separation processing on the occlusion intensity map based on the occlusion determination threshold, and mark the set of pixels in the high occlusion region; In one specific embodiment, region separation processing is performed on the occlusion intensity map based on the occlusion determination threshold.

[0068] Specifically, the logic for performing region separation processing is as follows: compare the pixel values ​​of each point in the occlusion intensity map with the occlusion determination threshold. If the pixel value is greater than or equal to the occlusion determination threshold, then the pixel coordinates are assigned to the set of pixels in the high-occlusion region. If the pixel value is less than the occlusion determination threshold, then the pixel coordinates are marked as a valid region.

[0069] S126: Remove the set of pixels in the highly occluded region from the fused image and output an unoccluded image.

[0070] In one specific embodiment, a set of pixels in highly occluded regions is removed from the fused image.

[0071] Specifically, the logic for the removal operation is as follows: Locate all coordinates of pixels belonging to the highly occluded region in the fused image. Replace the pixel color value at the corresponding coordinates with the average pixel value of the surrounding valid region. Smooth the replaced region using spatial interpolation. Output the processed, occlude-free image.

[0072] It should be noted that by removing highly occluded areas, the robot vision algorithm can be prevented from mistaking floating cutting fluid foam or metal wires for workpiece contours.

[0073] S13: Based on the unobstructed image, as well as the collected coolant pattern and metal debris pattern, generate interference image samples, train the recognition structure through interference image samples, and identify the workpiece edge value and workpiece category value. In one specific embodiment, firstly, patterns of coolant under spray conditions and patterns of metal shavings generated during machining in an industrial setting are acquired. These noise data, characteristic of the industry, are used to augment the unobstructed image. Interference image samples are generated by simulating complex visual interference in a real machining environment. These interference image samples are then input into a pre-defined convolutional neural network for learning, resulting in a recognition structure capable of resisting noise interference. Features are extracted using the recognition structure, and pose reference data for robot grasping is output.

[0074] Specifically, the steps for generating interfering image samples are as follows: S131: Collect images of coolant patterns and metal debris patterns, and establish a set of images of interfering elements based on the image content; In one specific embodiment, a high-speed camera captures dynamic images of a CNC machining center in operation, and a partial frame containing coolant splashes and metal shavings splashes is extracted.

[0075] Specifically, the logic for establishing the image set of interfering elements is as follows: perform background subtraction on the captured local frames; extract isolated connected regions where pixel brightness changes abruptly; The width and height of the connected region are compared with a preset granularity threshold. If the size of the connected region is within the granularity threshold range, it is determined to be a coolant pattern or a metal debris pattern. The selected coolant patterns and metal debris patterns are stored in a unified image container to form a set of interference element images.

[0076] It should be noted that the particle size threshold is preset in the following way: for cutting experiments of different materials (such as aluminum alloy and 45# steel) at different spindle speeds, the projection pixel size of the cutting chips in the image is statistically analyzed, and a 'speed-pixel size' mapping table is established. This setting logic ensures that the interference recognition structure can accurately lock the real chips, rather than the tiny hole features of the workpiece itself.

[0077] S132: Determine the area of ​​interference image overlay based on the position coordinates and edge contour range of the target region in the unobstructed image; In one specific embodiment, the interference image overlay area is determined based on the position coordinates of the target region in the unobstructed image and the edge contour range.

[0078] Specifically, the calculation logic for determining the interference image overlay region is as follows: Obtain the position coordinates of the target region, i.e., the coordinates of the center point of the workpiece in the image. Obtain the edge contour range, i.e., the smallest bounding rectangle surrounding the edge of the workpiece. Set an offset coefficient, which is obtained by measuring the maximum clearance of the fixture on the worktable. Extend the edge contour range outward by a width determined by the product of the position coordinates and the offset coefficient. Mark the extended rectangular closed region as the interference image overlay region.

[0079] It should be noted that the setting of the interference image overlay area ensures that the synthesized sample can simulate the real situation of interference appearing at and around the edge of the workpiece.

[0080] S133: Perform perturbation fusion processing on the superimposed region of the interfering images in the unobstructed image using the set of interfering element images to generate interfering image samples.

[0081] Specifically, the computational logic for performing perturbation fusion processing is as follows: Obtain a transparency weight value, which is preset based on the refractive index and color depth of the coolant.

[0082] The interference pixel values ​​in the interference element image set are superimposed with the original pixel values ​​at the corresponding positions in the unobstructed image.

[0083] The generated pixel value is equal to the product of the transparency weight value and the interference pixel value, plus 1 minus the product of the difference between the transparency weight value and the original pixel value.

[0084] Iterate through the superimposed areas of the interfering images to recalculate all pixels.

[0085] The recalculated image is output as a sample of the interference image.

[0086] Specifically, the steps for training the recognition structure and identifying workpiece edge values ​​and workpiece category values ​​are as follows: S134: Combine the interfering image samples with the unoccluded image to form training image pairs, and construct a training image pair set; In one specific embodiment, interference image samples are paired with unobstructed images to form training image pairs.

[0087] Specifically, the computational logic for constructing the training image pair set is as follows: retrieve the original unoccluded image corresponding to each interference image sample during the generation process; use the interference image sample as the input source data of the neural network; use the unoccluded image as the learning target reference data of the neural network; pairwise associate the input source data and the learning target reference data to form training image pairs; repeat the above association process until all sampled samples are covered to generate the training image pair set.

[0088] It should be noted that by pairing images with noise interference with clear images, it is possible to guide the recognition structure to learn how to extract core workpiece features from complex backgrounds.

[0089] S135: Perform multiple rounds of iterative training on the images and corresponding annotations in the set based on the training images to generate a recognition structure; In one specific embodiment, multiple rounds of iterative training are performed on the images in the set and their corresponding annotations based on the training images.

[0090] Specifically, the computational logic for generating the recognition structure is as follows: inputting interference image samples from the training image pair set into the convolutional neural network model to be trained; extracting the predicted feature distribution of the interference image samples; and calculating the loss function value for a single iteration using the predicted feature distribution and corresponding annotations.

[0091] Specifically, the calculation logic for the loss function value is as follows: the loss function value is equal to the weighted sum of the edge extraction loss value and the category classification loss value; whereby the edge extraction loss value is obtained by calculating the cross-entropy between the predicted pixel gradient distribution and the corresponding true edge mask in the annotation; the category classification loss value is obtained by calculating the Euclidean distance between the predicted category probability and the corresponding true category label in the annotation; a preset loss weight allocation coefficient is obtained, which is determined according to the priority experimental ratio between edge localization accuracy and classification accuracy; the calculation expression for the loss function value is:

[0092] In the formula, The value of the loss function. Extract loss values ​​for the edges. For category classification loss value, Assign coefficients to the loss weights; The cross-entropy is a pixel-by-pixel binary classification cross-entropy loss form, which is calculated by averaging the log-likelihood between the predicted edge probability and the true edge mask.

[0093] The Euclidean distance corresponding to the category classification loss value is the L2 norm distance between the predicted category probability vector and the true category one-hot encoding vector.

[0094] The internal weight matrix of the convolutional neural network model is updated based on the loss function value using the backpropagation algorithm; training stops when the loss function value is less than or equal to the preset convergence threshold; and the trained convolutional neural network model is output as the recognition structure.

[0095] It should be noted that: the corresponding annotations are the workpiece pixel edge coordinates and their type labels that are manually annotated on the unobstructed image in advance; the convergence judgment threshold is obtained by the error stability interval of the statistical model on the validation set.

[0096] S136: Perform feature extraction and recognition processing on the unobstructed image by recognizing the structure, and generate workpiece edge value and workpiece category value respectively.

[0097] In one specific embodiment, feature extraction and recognition processing are performed on an unobstructed image by recognizing the structure.

[0098] Specifically, the calculation logic for generating workpiece edge values ​​and workpiece category values ​​is as follows: An unobstructed image is input into the recognition structure; the recognition structure extracts spatial geometric feature information of the image through its multiple convolutional layers; after outputting an edge probability map, the recognition structure performs threshold filtering and connectivity aggregation processing on the edge probability map to obtain the set of edge pixel coordinates corresponding to the workpiece contour, and defines the set of edge pixel coordinates or its closed boundary coordinate set sorted by connectivity as the workpiece edge value; the recognition structure simultaneously outputs a category probability vector, and by retrieving the category label corresponding to the item with the highest probability in the category probability vector, it outputs the workpiece name code in conjunction with a pre-established category label-workpiece name mapping table, and defines the workpiece name code as the workpiece category value.

[0099] It should be noted that the workpiece edge value directly determines the contact point position when the robot grasps, while the workpiece category value is related to the grasping torque and the transport path.

[0100] S14: Obtain the average distribution value of lens brightness and compare it with the set contamination threshold to generate lens contamination results. Generate target pose recognition results based on lens contamination results and workpiece category values.

[0101] In one specific embodiment, the system monitors the optical channel status of an industrial camera in real time. It obtains the average brightness distribution value of the lens by analyzing the brightness stability of consecutive images and compares it with a set contamination threshold to generate a lens contamination result. Combined with the workpiece category value identified in previous steps, it generates a target pose recognition result to guide the robot's end effector for precise grasping.

[0102] Specifically, the steps to generate lens contamination results are as follows: S141: Based on the fused images of multiple consecutive frames in the image acquisition sequence, calculate the average brightness value of each frame and generate a brightness mean sequence. In one specific embodiment, the average brightness value of each frame is calculated based on multiple consecutive fused images from the image acquisition sequence. Specifically, the calculation logic for generating the average brightness value sequence is as follows: extract N consecutive fused images from the image acquisition sequence. For each fused image frame, accumulate the grayscale values ​​of all pixels within that frame. Divide the accumulated total grayscale value by the total number of pixels in that frame to obtain the average brightness value of that frame. Arrange the N average brightness values ​​according to the order in which the images were acquired. Use the arranged data group as the average brightness value sequence. It should be noted that the number of sampling frames N is set according to the working pulse frequency of the CNC machining center, and is usually set between 10 and 50 frames.

[0103] S142: Perform multi-scale trend analysis based on the mean brightness sequence to extract the low-frequency stable segment in the mean brightness change curve; In one specific embodiment, multi-scale trend analysis is performed based on the mean brightness sequence.

[0104] Specifically, the calculation logic for extracting the low-frequency stable segment is as follows: slide time windows of different lengths on the brightness mean sequence.

[0105] Calculate the slope of brightness data change within each time window. Obtain a stability discrimination coefficient, which is obtained by measuring the brightness fluctuation range of the clean lens under standard illumination.

[0106] If the absolute value of the slope value within multiple consecutive time windows is less than or equal to the stability discrimination coefficient, then the curve segment is determined to be in a stable fluctuation state.

[0107] Extract the average brightness sequence segment that is in a stable fluctuation state. Output it as a low-frequency stable segment.

[0108] It should be noted that the low-frequency stable range represents the baseline level of ambient light in the environment, excluding instantaneous brightness jumps caused by splashing metal debris.

[0109] S143: Perform a difference calculation between the average brightness value of the current frame and the average value of the low-frequency stable segment, and compare it with the set contamination threshold to generate the lens contamination result.

[0110] In one specific embodiment, the difference between the average brightness of the current frame and the average value of the low-frequency stable segment is calculated.

[0111] Specifically, the calculation logic for generating the lens contamination result is as follows: Calculate the arithmetic mean of all sample points within the low-frequency stable region to obtain the average value of the low-frequency stable region.

[0112] Obtain the average brightness of the latest frame in the image acquisition sequence as the average brightness of the current frame. Calculate the absolute value of the difference between the average brightness of the current frame and the average value of the low-frequency stable region.

[0113] A set contamination threshold is obtained, which is set by simulating the percentage of light intensity attenuation caused by cutting fluid mist obscuring the lens; It should be noted that the pollution threshold is specifically calibrated as follows: the average value of the low-frequency stable range is obtained under standard fog-free conditions as the reference brightness. Set the attenuation ratio coefficient Then set a pollution threshold. 。

[0114] If the absolute value of the difference is greater than or equal to the set contamination threshold, a lens contamination result representing lens damage or severe contamination will be generated.

[0115] If the absolute value of the difference is less than the set contamination threshold, a lens contamination result characterizing lens cleanliness is generated.

[0116] In one specific embodiment, the image recognition confidence state is determined based on the lens contamination results, and the workpiece recognition results are then filtered.

[0117] Specifically, the calculation logic for the screening and identification results is as follows: obtain the absolute value of the difference between the average brightness of the current frame recorded in the lens contamination results and the average value of the low-frequency stable segment.

[0118] Calculate the ratio of the absolute value of the difference to the average value of the low-frequency stable section, and define this ratio as the brightness offset ratio.

[0119] Multiple gradient ranges are set. If the brightness offset ratio is between 0 and 0.1, the pollution level is marked as level 1. If the brightness offset ratio is between 0.1 and 0.25, the pollution level is marked as Level 2. If the brightness offset ratio is between 0.25 and 0.4, the pollution level is marked as Level 3. If the brightness offset ratio is greater than or equal to 0.4, the pollution level is marked as level 4.

[0120] Obtain a preset pollution level determination threshold.

[0121] The current pollution level is compared with the pollution level determination threshold. If the current pollution level is greater than or equal to the pollution level determination threshold, the image recognition confidence status is determined to be unreliable, and the corresponding workpiece recognition result is deleted from the current data stream.

[0122] If the current pollution level is less than the pollution level determination threshold, the image recognition confidence status is determined to be reliable, and the corresponding workpiece recognition result is retained.

[0123] It should be noted that the pollution level determination threshold was obtained by statistically analyzing the correlation curve between the robot's successful capture rate and the image contrast decay, in order to ensure the reliability of the visual data.

[0124] The workpiece category value of the retained recognition result is input into the pose model recognition function to perform spatial pose matching processing, and the pose matching processing result is output as the target pose recognition result.

[0125] In one specific embodiment, the workpiece category value and workpiece edge value of the retained recognition results are associated, and spatial posture matching is performed by establishing a mapping relationship between the virtual three-dimensional space and the robot's physical space.

[0126] Specifically, the calculation logic for performing spatial posture matching is as follows: based on the workpiece category value, the corresponding 3D CAD geometric model is retrieved from the preset feature library; Obtain the camera's intrinsic parameter matrix, which includes the principal point coordinates and focal length parameters; use the intrinsic parameter matrix to convert the pixel coordinates in the workpiece edge values ​​into three-dimensional spatial point coordinates with the camera's optical center as the origin; The extrinsic parameter matrix of the camera is obtained in advance through hand-eye calibration experiments between the robot end effector and the camera. It is used to describe the rotation and translation transformation relationship between the camera coordinate system and the robot base coordinate system. The coordinates of three-dimensional points are transformed to the robot base coordinate system using the extrinsic parameter matrix to obtain the robot's spatial coordinates. Calculate the Euclidean distance from each robot spatial coordinate point to the nearest point on the surface of the 3D CAD geometric model; calculate the sum of the squares of the Euclidean distances of all robot spatial coordinate points; The six-degree-of-freedom parameters of the 3D CAD geometric model in the robot base coordinate system are adjusted iteratively using the least squares method. The six-degree-of-freedom parameters include translation in three axes and rotation in three axes. It should be noted that before performing the least squares iteration, the geometric center coordinates and principal axis direction of the workpiece are calculated using the workpiece edge values, and these are used as the initial values ​​for the six degrees of freedom parameters to ensure the convergence of the iteration process.

[0127] Determine whether the sum of squares of the Euclidean distances is less than or equal to a preset error convergence threshold; When the sum of squares of the Euclidean distances is less than or equal to the error convergence threshold, the displacement of the 3D CAD geometric model relative to the robot base coordinate system in the three axes is extracted as the translation offset. Extract the rotation angles of the three axes of the 3D CAD geometric model relative to the robot base coordinate system, and use them as axis rotation angles; The translation offset and axis rotation angle are encapsulated to generate target pose recognition results for output.

[0128] It should be noted that the error convergence threshold is set based on the mechanical fit tolerance of the robot fixture within the CNC machining center. The extrinsic parameter matrix includes the physical scale transformation from the vision sensor space to the robot motion space, ensuring that the recognition results can be directly converted into robot motion commands. The translation offset reflects the three-dimensional absolute position of the workpiece center in the robot base coordinate system, and the axis rotation angle reflects the deflection posture of the workpiece relative to the robot axis. S15: Generate loading and unloading operation scheduling instructions based on the final target pose recognition result, workpiece category value and lens contamination result, and perform branch control on the loading and unloading operation execution strategy according to the contamination level corresponding to the lens contamination result; In one specific embodiment, the loading and unloading operation scheduling instruction includes the loading and unloading operation task number, target workstation number, grasping posture parameters, grasping torque parameters, transport path parameters, and verification shooting strategy parameters; The logic for generating the loading and unloading operation scheduling instruction is as follows: obtain the current loading and unloading operation task queue to be executed, and read the workpiece category value and the final target pose recognition result associated with each task; query the preset workpiece category-grabbing parameter mapping table according to the workpiece category value to generate the corresponding task's gripping torque parameter and clamping strategy parameter. The arrival path of the robot end effector is calculated based on the final target pose recognition result, and the transport path parameters are generated. The above parameters are combined to form the loading and unloading operation scheduling instruction and output to the robot controller.

[0129] The logic for branch control is as follows: when the pollution level is level 1, the standard loading and unloading strategy is executed and the loading and unloading operation scheduling instructions are output in the order of the task queue. When the pollution level is level 2, a review shooting strategy is added before the loading and unloading operation scheduling instruction is output. If the pose deviation between the final target pose recognition result obtained by the review shooting and the previous final target pose recognition result is less than the preset pose deviation threshold, the loading and unloading operation scheduling instruction is output; otherwise, the task is marked as a reshoot task and inserted at the beginning of the task queue. When the pollution level is level 3, the current loading and unloading operation is suspended and a lens cleaning operation instruction is output. After the lens cleaning operation is completed, S11 to S14 are re-executed to generate the final target pose recognition result, and then the loading and unloading operation scheduling instruction is output. When the pollution level is level 4, the system sends a "feed pause" signal to the CNC system, interlocks and suspends the current machining operation, and outputs a fault maintenance command. This step constitutes a closed-loop safety mechanism for vision-based machine tool operation control, which is an essential technical feature of this method, ensuring that collisions or mis-grabbing will not occur in extremely polluted environments.

[0130] It should be noted that the loading and unloading operation scheduling instructions are transmitted to the task buffer of the robot controller via the industrial Ethernet protocol; when executing lens cleaning operation instructions or fault maintenance operation instructions, the system simultaneously suspends the current feed instructions of the CNC machining center until a feedback signal indicating that lens cleaning is complete is returned.

[0131] The above embodiments are only used to illustrate the technical methods of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical methods of the present invention without departing from the spirit and scope of the technical methods of the present invention.

Claims

1. A loading and unloading operation scheduling method for CNC machining, characterized in that, The method includes: Acquire multi-polarization angle images and lens transmittance values ​​under polarization angle switching, and generate a fused image through image fusion and transmittance enhancement; Based on the fused image, edge distribution values ​​and brightness gradient values ​​are generated. An occlusion intensity map is constructed using the edge distribution values ​​and brightness gradient values, and occlusion areas are removed to generate an occlusion-free image. Interference image samples are generated based on unobstructed images, as well as collected coolant and metal debris patterns. The recognition structure is trained using interference image samples to identify workpiece edge values ​​and workpiece category values. The lens brightness mean distribution value is obtained and compared with the set contamination threshold to generate lens contamination results. Based on the lens contamination results and the workpiece category value, the target pose recognition result is generated. Based on the final target pose recognition result, workpiece category value and lens contamination result, a loading and unloading operation scheduling instruction is generated, and the loading and unloading operation execution strategy is branched according to the contamination level corresponding to the lens contamination result.

2. The loading and unloading operation scheduling method for CNC machining according to claim 1, characterized in that, The steps to generate the fused image are as follows: By switching the polarization angle, images from multiple polarization angles are acquired, and these images are aligned according to the acquisition time sequence to generate a multi-angle image group. Spot suppression processing is performed on the pixel brightness distribution of each image in the multi-angle image group to obtain a spot-suppressed image group; Obtain the lens transmittance value and perform transmittance enhancement processing on the splatter suppression image group to generate a transmittance enhancement image group; A pixel-level image fusion operation is performed on the light-enhancing image group to generate a fused image.

3. The loading and unloading operation scheduling method for CNC machining according to claim 2, characterized in that, The steps for performing transmittance enhancement treatment are as follows: A transmittance matching calculation is performed based on the lens transmittance value and the average brightness of each image in the bokeh suppression image group to generate an image brightness compensation factor; The pixel brightness values ​​of each image in the light spot suppression image group are adjusted by the image brightness compensation factor to obtain the brightness compensation image group, which is then output as the light transmission enhancement image group.

4. The loading and unloading operation scheduling method for CNC machining according to claim 3, characterized in that, The steps to construct an occlusion intensity map are as follows: Based on the grayscale changes of each pixel in the fused image, the grayscale difference between adjacent pixels is calculated to obtain the brightness gradient value. Edge detection is performed on the fused image to extract all edge pixels, count the number of edge pixels in a unit area, and generate edge distribution values. An occlusion intensity map is generated by weighting and superimposing edge distribution values ​​and brightness gradient values.

5. A loading and unloading operation scheduling method for CNC machining according to claim 4, characterized in that, The steps for performing weighted summation calculation are as follows: Normalize the brightness gradient values ​​to obtain normalized brightness gradient values; The edge distribution values ​​are used to perform high-response region extraction operations to filter out high-density edge region values; An occlusion intensity map is generated by performing a weighted fusion calculation on the normalized brightness gradient value and the edge high-density region value.

6. A loading and unloading operation scheduling method for CNC machining according to claim 5, characterized in that, The steps to remove obstructed areas are as follows: Based on the pixel value distribution in the occlusion intensity map, set the occlusion determination threshold; Based on the occlusion determination threshold, the occlusion intensity map is processed to separate regions and mark the set of pixels in the high occlusion region. Remove the set of pixels with high occlusion from the fused image to output an unoccluded image.

7. A loading and unloading operation scheduling method for CNC machining according to claim 6, characterized in that, The steps for generating interfering image samples are as follows: Collect images of coolant patterns and metal debris patterns, and establish a set of images of interfering elements based on the image content; Based on the location coordinates and edge contour range of the target region in the unobstructed image, the area of ​​interference image overlay is determined; Perturbation fusion processing is performed on the superimposed regions of interfering images in an unoccluded image using a set of interfering element images to generate interfering image samples.

8. A loading and unloading operation scheduling method for CNC machining according to claim 7, characterized in that, The steps for training the recognition structure and identifying workpiece edge values ​​and workpiece category values ​​are as follows: The interference image samples are paired with the unoccluded images to form training image pairs, and a training image pair set is constructed. Based on the training images, perform multiple rounds of iterative training on the images and corresponding annotations in the set to generate a recognition structure; By recognizing the structure, feature extraction and recognition processing are performed on the unobstructed image to generate workpiece edge values ​​and workpiece category values, respectively.

9. A loading and unloading operation scheduling method for CNC machining according to claim 8, characterized in that, The steps to generate lens contamination results are as follows: Based on the fused images of multiple consecutive frames in the image acquisition sequence, the average brightness value of each frame is calculated to generate a brightness mean sequence. Perform multi-scale trend analysis based on the mean brightness sequence to extract the low-frequency stable segment from the mean brightness change curve; The difference between the average brightness of the current frame and the average value of the low-frequency stable section is calculated and compared with the set pollution threshold to generate the lens pollution result.

Images