Material status visual perception and recognition methods and related equipment for machine tool loading and unloading

Abstract

This application relates to the fields of visual perception and robotic loading / unloading technology, and particularly to a method and related equipment for visual perception and recognition of material status during machine tool loading / unloading. The method includes: acquiring real-time images of materials within a loading box; processing these images using a pre-trained hybrid recognition model that integrates deep learning and traditional visual feature extraction; branching processing according to classification results; optimizing local images of normal materials for scene adaptation; extracting contour features and key geometric parameters; comparing these parameters with a standard parameter library to determine the posture; and integrating the results to output a comprehensive recognition result. It also includes steps such as model pre-training and parameter library construction, enabling accurate identification and precise positioning of material status, adapting to multiple working conditions and dynamically adjusting to changes, providing effective information for the machine tool control system.

Description

Technical Field

[0001] This application relates to the fields of visual perception and robotic loading and unloading technology, and in particular to a visual perception and recognition method and related equipment for material status during machine tool loading and unloading. Background Technology

[0002] In modern industrial production systems centered on high-precision machining, the widespread adoption of CNC machine tool automated production lines has played a crucial role. It has significantly improved the efficiency and accuracy of precision machining, propelling industrial production towards greater efficiency and precision, resulting in substantial improvements in product quality and production scale, and thus occupying a vital position in the industrial manufacturing sector. However, the material handling process, as a key link connecting the entire process of "material input - machine tool processing - finished product output," faces numerous challenges in achieving a high level of automation. Because this process involves manual operation, which is inherently uncertain, and the production environment is also relatively complex, the material handling process has become a critical bottleneck restricting further advancements in the automation of the entire production line.

[0003] Previous technologies have employed various methods to address material handling issues. Regarding material state recognition, traditional robotic material handling methods without visual perception lack the ability to judge the material's state; they simply perform grasping actions according to a fixed program. Traditional visual recognition technologies typically use template matching and threshold segmentation to identify material state. For material positioning, limiting blocks are added to the material frame to restrict the material's movement, aiming to keep the material in a relatively fixed position for easier subsequent grasping and processing.

[0004] However, existing technologies have significant drawbacks. Traditional robotic material handling without visual perception continues to grasp materials even when the material box is empty, increasing the time spent on ineffective grasping and wasting production line time. When the material is raw (inverted, reversed) or processed, it can cause processing logic errors, resulting in collisions with the fixture's positioning surface. Traditional visual recognition technology is limited by changes in ambient lighting and the varying characteristics of materials in different states, leading to algorithmic feature extraction failures and low recognition accuracy. This makes it difficult to meet the accuracy requirements for full-state material recognition and precise positioning in complex industrial scenarios, significantly reducing production line automation efficiency and causing subsequent processing anomalies due to inaccurate detection, resulting in batch quality defects. Furthermore, while adding limit blocks in the material box can restrict the material's movement, the box shakes during trolley transport, causing dynamic shifts in the material's position relative to its initial location. Simultaneously, multimodal interference sources in industrial environments significantly reduce the robustness of the visual positioning system. Summary of the Invention

[0005] The purpose of this application is to provide a method and related equipment for visual perception and recognition of material status for machine tool loading and unloading.

[0006] The purpose of this application is to provide a visual perception and recognition method for material status during machine tool loading and unloading, including:

[0007] Real-time images of materials in the material box during machine tool loading and unloading are collected, and the real-time images include the shape of the materials and scene environment information;

[0008] The pre-trained hybrid recognition model is invoked to process the real-time image. The hybrid recognition model integrates deep learning object detection and traditional visual feature extraction algorithms, and outputs the initial classification results and feature region identifiers of the materials. The initial classification results include normal materials to be processed and materials in abnormal state.

[0009] Based on the initial classification results, branch processing is performed: if the material is in an abnormal state, the status information containing the abnormal type is directly output; if the material is normal and awaiting processing, the local image region corresponding to the feature region identifier is extracted.

[0010] Scene adaptation optimization processing is performed on the local image region, including adaptive noise reduction and contrast enhancement based on material surface characteristics;

[0011] Extract the contour features of the material and at least two types of key geometric parameters from the optimized local image region. The key geometric parameters include the relative positional relationship between features and the feature size.

[0012] The key geometric parameters are compared with a preset standard parameter library in multiple dimensions, and the spatial orientation of the material is determined by combining the comparison deviation value.

[0013] Integrate the attitude determination results with the coordinate information of the feature region to output a comprehensive recognition result that includes material status and precise positioning.

[0014] By adopting the above technical solution, multiple states of materials within the loading frame in machine tool loading and unloading scenarios can be accurately identified, effectively solving the problems of low recognition accuracy and poor robustness of traditional vision technologies without visual perception or based on manually designed features. Even in the face of complex industrial interferences such as changes in lighting, metal reflection, residual cutting fluid, and dynamic material displacement caused by frame shaking, it can still output stable and highly accurate recognition results. At the same time, through the synergistic effect of hybrid recognition models and scene adaptation optimization processing, high-precision spatial positioning of normal materials to be processed is achieved, accurately obtaining the key geometric parameters and coordinate information required for material grasping, completely avoiding problems such as invalid grasping, processing logic errors, and jig collisions, significantly optimizing the production line operation process, and improving the efficiency and stability of automated production line operation. In addition, this method does not require frequent manual intervention to adjust parameters or processing conditions, which can effectively reduce labor input costs, reduce batch quality defects caused by inaccurate detection, reduce raw material waste and rework losses, and fully meet the stringent requirements of precision machining for "zero-defect and fully continuous production", providing strong support for the intelligent upgrade of machine tool loading and unloading processes.

[0015] In one possible implementation of this application, the pre-training process of the hybrid recognition model includes:

[0016] Construct a sample library containing multiple working conditions of machine tool loading and unloading scenarios. The samples cover labeled images with different light intensities, material surface conditions, and material frame stacking states.

[0017] A deep learning object detection network was initialized using transfer learning, and trained using the categories and feature regions of materials in the sample library as labeled targets.

[0018] The traditional visual feature extraction module is trained synchronously, and the classification loss of the deep learning network and the feature localization loss of the traditional algorithm are dynamically balanced by the loss function.

[0019] By adopting the above technical solutions, the hybrid recognition model can fully learn the material characteristics under different working conditions in machine tool loading and unloading scenarios, effectively adapting to the recognition challenges brought about by changes in lighting, differences in material surface conditions, and different stacking methods of material frames, significantly improving the model's adaptability to complex industrial environments. By using transfer learning to initialize the deep learning object detection network, the sample size and training time required for model training can be reduced, improving training efficiency and reducing training costs, while avoiding overfitting or slow convergence problems that may occur when training the model from scratch. By dynamically balancing the classification loss of the deep learning network and the feature localization loss of traditional algorithms through the loss function, the model can achieve excellent performance in both material state classification and recognition and accurate feature region localization. This ensures the accuracy of the initial classification results and lays a reliable foundation for subsequent local image region processing and key parameter extraction, thereby improving the stability and efficiency of the entire material state visual perception and recognition process, and providing strong support for the intelligent operation of the machine tool loading and unloading process.

[0020] In one possible implementation of this application, scene adaptation optimization processing is performed on the local image region. This processing includes adaptive noise reduction and contrast enhancement steps based on material surface characteristics, comprising:

[0021] Pixel brightness analysis is performed on the local image area to identify the highlight area and non-highlight area on the material surface, wherein the highlight area is a cluster of high-brightness pixels formed by metal reflection or oil reflection.

[0022] A bilateral filtering algorithm is used to denoise the highlight areas, which suppresses noise while preserving feature edge information by maintaining the edge weight factor.

[0023] For non-highlight areas, a Gaussian filtering algorithm is used for noise reduction. The pixel values ​​are weighted and averaged using a preset Gaussian kernel function to suppress random noise.

[0024] The local image regions after noise reduction are subjected to contrast enhancement processing using local histogram equalization or multiplicative image fusion algorithms to improve the grayscale difference between the feature regions and the background regions. The feature regions are image sub-regions containing preset geometric features.

[0025] By adopting the above technical solutions, the characteristics of different areas on the material surface can be specifically adapted, and the high-light interference caused by metal reflection and oil stain reflection, as well as random environmental noise, can be accurately processed. While effectively suppressing various noises and improving image quality, the edge information of key material features can be preserved to the greatest extent, avoiding the feature blurring problem caused by traditional single noise reduction methods. The subsequent contrast enhancement processing further widens the grayscale difference between the feature area and the background area, making the material contour, positioning holes and other key features clearer and more distinguishable. This provides a high-quality image foundation for subsequent contour feature extraction and key geometric parameter calculation, significantly improving the accuracy and stability of feature extraction. It effectively resists the impact of multimodal interference such as light fluctuations and surface contamination in industrial scenarios on the recognition effect, thereby ensuring the robustness of the entire material state recognition and positioning process and providing reliable support for the robot to accurately perform loading and unloading operations.

[0026] In one possible implementation of this application, the step of extracting the contour features of a material and at least two types of key geometric parameters from an optimized local image region, wherein the key geometric parameters include the relative positional relationship between features and the feature size, includes:

[0027] The Sobel operator is used to perform convolution operations on the optimized local image region to obtain the gradient components in the x and y directions;

[0028] Based on the gradient components, the gradient intensity and gradient direction of the pixel are calculated, and the edge is thinned to a single pixel width through non-maximum suppression processing.

[0029] A dual-threshold filtering mechanism is used to retain effective edges. Pixels with gradient strength greater than the high threshold are judged as strong edges, while weak edges with gradient strength between the high and low thresholds and connected to the strong edges are retained.

[0030] Subpixel contours are extracted from materials based on effective edges, and key geometric parameters are extracted from the contours.

[0031] By adopting the above technical solution, pixel gradient information of the optimized local image region can be accurately captured. Non-maximum suppression processing refines the edges to a single pixel width, effectively avoiding the problems of excessively wide edges and blurred contours in traditional edge extraction. Then, a dual threshold screening mechanism accurately distinguishes strong edges from weak edges, eliminating false edges while retaining effective weak edges connected to strong edges, ensuring the integrity and purity of the material edge contour. The sub-pixel contour based on effective edge extraction further improves the fineness of contour features, providing a high-precision foundation for the extraction of key geometric parameters. This makes the calculation of parameters such as the relative positional relationship between features and feature size more accurate and reliable, effectively avoiding the problem of misjudgment of material posture caused by coarse contour extraction and parameter calculation deviation. It significantly enhances the stability and robustness of material feature extraction in complex industrial scenarios, providing solid feature support for subsequent material spatial posture determination and precise positioning, and ensuring the accuracy of robot loading and unloading operations.

[0032] In one possible implementation of this application, the step of comparing the key geometric parameters with a preset standard parameter library in multiple dimensions includes:

[0033] Based on the sub-pixel contour, at least two types of key geometric parameters of the material are extracted, wherein:

[0034] To determine the relative positional relationship between features, the relative distance and angle between material surface features are calculated using the coordinates of contour feature points, thereby generating positional relationship parameters.

[0035] For feature dimensions, the length, width, diameter and other dimensional parameters of the feature are calculated by fitting contour edge segments or curves.

[0036] The extracted key geometric parameters are compared item by item with the preset standard parameter library, and the deviation value of each parameter is calculated.

[0037] The deviation value is calculated comprehensively based on the weight of the parameters in determining the material posture.

[0038] Based on the comparison between the comprehensive deviation value and the preset threshold, it is determined whether the material is within the preset spatial orientation range.

[0039] By adopting the above technical solution, key geometric parameters such as the relative positional relationship and feature size between material features can be comprehensively extracted based on high-precision sub-pixel contours, ensuring the completeness and relevance of parameter extraction and providing comprehensive and reliable data support for posture determination. Through item-by-item comparison with a preset standard parameter library and deviation value calculation, accurate verification of each parameter is achieved. Then, combined with the weighted impact of parameters on posture determination, a comprehensive calculation is performed, avoiding the one-sidedness of single-parameter comparisons and making deviation assessment more aligned with actual working conditions. Based on the comparison results of the comprehensive deviation value and a preset threshold, it is possible to accurately determine whether the material is within the preset spatial posture range, effectively avoiding posture misjudgment caused by single parameter evaluation and weight imbalance. This significantly improves the accuracy and rationality of material spatial posture determination, enhances the robustness of posture recognition under complex working conditions, provides crucial assurance for robots to accurately execute loading and unloading operations and avoid processing logic errors, and further consolidates the reliability of the entire visual perception and recognition process.

[0040] In one possible implementation of this application, the steps for constructing the preset standard parameter library include:

[0041] Collect reference images of the same type of material under standard working conditions. The reference images cover the feature data of the material's normal posture and typical abnormal posture.

[0042] Feature extraction is performed on the reference image to obtain the contour features and key geometric parameters under standard conditions, which are then used as reference parameters.

[0043] By statistically analyzing multiple sets of benchmark parameters, the allowable fluctuation range of each parameter is determined, and standard parameter thresholds are formed.

[0044] Classify and store the benchmark parameters and thresholds corresponding to different material types and different working conditions to establish an expandable standard parameter library;

[0045] It supports updating the standard parameter library by adding new benchmark image samples and dynamically adjusting parameter thresholds to adapt to changes in operating conditions.

[0046] By adopting the above technical solutions, the constructed standard parameter library can determine a scientifically reasonable allowable fluctuation range based on comprehensive benchmark data under standard working conditions of the same material model, combined with statistical analysis of multiple sets of parameters. This ensures the accuracy and authority of benchmark parameters and thresholds, providing a reliable basis for multi-dimensional comparison of key geometric parameters. Simultaneously, the standard parameter library supports the categorized storage of benchmark parameters and thresholds for different material models and working conditions, possessing good compatibility and scalability, and adapting to diverse machine tool loading and unloading scenarios. Furthermore, dynamic updates of parameter thresholds can be achieved by adding new benchmark image samples, enabling the standard parameter library to flexibly respond to actual needs such as changes in working conditions and adjustments to material characteristics. This effectively avoids the problem of insufficient recognition adaptability caused by a fixed parameter library, significantly improving the adaptability and robustness of the entire material state visual perception and recognition method in complex and ever-changing industrial scenarios, providing a solid foundation for continuous and stable attitude determination and precise positioning.

[0047] In one possible implementation of this application, the step of integrating the attitude determination result with the coordinate information of the feature region to output a comprehensive identification result including material state and precise positioning includes:

[0048] The material status is divided into two categories: normal and awaiting capture, and abnormal. A unique identifier code is assigned to each status.

[0049] Calculate the three-dimensional coordinates of the material grabbing point based on the coordinate information of the feature area identifier;

[0050] Output digital signals containing status identification codes, three-dimensional coordinates of the gripping point, and attitude angles to the machine tool control system;

[0051] Visual information, including bounding boxes of feature regions, values ​​of key geometric parameters, and textual annotations of state determination results, is overlaid on real-time images.

[0052] If the condition is determined to be abnormal, an explanation of the abnormality type and corresponding handling suggestions will be output synchronously.

[0053] By adopting the above technical solutions, the machine tool control system can quickly and accurately identify the material status through a clear classification method and unique identification code. The three-dimensional coordinates and posture angles of the gripping point calculated by combining the coordinate information of the feature area provide precise operational basis for the robot, ensuring the accurate execution of loading and unloading actions. The standardized digital signals output to the machine tool control system realize the efficient docking of the identification results with the automated production system, while the visual information superimposed on the real-time image allows the staff to intuitively grasp the material status, key parameters and judgment results, reducing the difficulty of manual monitoring and debugging. The type description and handling suggestions output synchronously for abnormal status can help the staff quickly locate the problem and take targeted measures to avoid the production stoppage or quality defects caused by the abnormality escalating. Overall, it not only improves the automation collaboration efficiency and operational accuracy of the machine tool loading and unloading process, but also enhances the monitorability of the production process and the timeliness of problem handling, providing a strong guarantee for the stable and continuous operation of the production line.

[0054] The second objective of this application is to provide a visual perception and recognition system for material status during machine tool loading and unloading, the system comprising:

[0055] Image acquisition module: used to acquire real-time images of materials in the material box during machine tool loading and unloading scenarios. The real-time images include the shape of the materials and scene environment information.

[0056] Hybrid recognition model processing module: used to call a pre-trained hybrid recognition model to process the real-time image. The hybrid recognition model integrates deep learning object detection and traditional visual feature extraction algorithms, and outputs the initial classification results and feature region identifiers of the materials. The initial classification results include normal materials to be processed and materials in abnormal state.

[0057] Branch processing module: used to perform branch processing based on the initial classification result: if it is an abnormal material, directly output the status information containing the abnormal type; if it is a normal material to be processed, extract the local image region corresponding to the feature region identifier;

[0058] Scene adaptation optimization module: used to perform scene adaptation optimization processing on the local image region, the processing including adaptive noise reduction and contrast enhancement based on material surface characteristics;

[0059] Feature extraction module: used to extract the contour features of the material and at least two types of key geometric parameters from the optimized local image region, wherein the key geometric parameters include the relative positional relationship between features and the feature size;

[0060] Attitude determination module: used to compare the key geometric parameters with a preset standard parameter library in multiple dimensions, and determine the spatial attitude of the material based on the comparison deviation value;

[0061] The result integration and output module is used to integrate the attitude determination results with the coordinate information of the feature area, and output a comprehensive recognition result that includes material status and precise positioning.

[0062] By adopting the above technical solution, multiple states of materials within the loading frame in machine tool loading and unloading scenarios can be accurately identified, effectively solving the problems of low recognition accuracy and poor robustness of traditional vision technologies without visual perception or based on manually designed features. Even in the face of complex industrial interferences such as changes in lighting, metal reflection, residual cutting fluid, and dynamic material displacement caused by frame shaking, it can still output stable and highly accurate recognition results. At the same time, through the synergistic effect of hybrid recognition models and scene adaptation optimization processing, high-precision spatial positioning of normal materials to be processed is achieved, accurately obtaining the key geometric parameters and coordinate information required for material grasping, completely avoiding problems such as invalid grasping, processing logic errors, and jig collisions, significantly optimizing the production line operation process, and improving the efficiency and stability of automated production line operation. In addition, this method does not require frequent manual intervention to adjust parameters or processing conditions, which can effectively reduce labor input costs, reduce batch quality defects caused by inaccurate detection, reduce raw material waste and rework losses, and fully meet the stringent requirements of precision machining for "zero-defect and fully continuous production", providing strong support for the intelligent upgrade of machine tool loading and unloading processes.

[0063] The third objective of this application is to provide a visual perception and recognition device for material status during machine tool loading and unloading, the device comprising:

[0064] The memory and processor, wherein the memory stores a computer program that can be loaded and executed by the processor to perform the above-described visual perception and recognition method for material status during machine tool loading and unloading.

[0065] The fourth objective of this application is to provide a storage medium.

[0066] The fourth objective of this application is achieved through the following technical solution:

[0067] A storage medium storing a computer program capable of being loaded by a processor and executed by the aforementioned visual perception and recognition method for material status during machine tool loading and unloading.

[0068] In summary, this application includes at least one of the following beneficial technical effects:

[0069] 1. This method can accurately identify various states of materials within the loading frame during machine tool loading and unloading, effectively solving the problems of low accuracy and poor robustness inherent in traditional vision technologies lacking visual perception or based on manually designed features. Even when faced with complex industrial interferences such as changes in lighting, metal reflections, residual cutting fluid, and dynamic material displacement caused by frame shaking, it can still stably output highly accurate recognition results. Simultaneously, through the synergistic effect of a hybrid recognition model and scene-adaptive optimization processing, it achieves high-precision spatial positioning of normal materials, accurately acquiring the key geometric parameters and coordinate information required for material grasping, completely avoiding problems such as invalid grasping, processing logic errors, and jig collisions. This significantly optimizes the production line operation process and improves the efficiency and stability of automated production line operation. Furthermore, this method eliminates the need for frequent manual intervention to adjust parameters or processing conditions, effectively reducing labor costs and minimizing batch quality defects caused by inaccurate detection. It also reduces raw material waste and rework losses, fully meeting the stringent requirements of precision machining for "zero-defect, fully continuous production," providing strong support for the intelligent upgrading of machine tool loading and unloading processes.

[0070] 2. The hybrid recognition model can fully learn the material characteristics under different working conditions in machine tool loading and unloading scenarios, effectively adapting to the recognition challenges brought about by changes in lighting, differences in material surface conditions, and different stacking methods of material frames, significantly improving the model's adaptability to complex industrial environments. By using transfer learning to initialize the deep learning object detection network, the amount of samples and training time required for model training can be reduced, improving training efficiency and reducing training costs, while avoiding overfitting or slow convergence problems that may occur when training the model from scratch. By dynamically balancing the classification loss of the deep learning network and the feature localization loss of traditional algorithms through the loss function, the model can achieve excellent performance in both material state classification and recognition and accurate feature region localization. This ensures the accuracy of the initial classification results and lays a reliable foundation for subsequent local image region processing and key parameter extraction, thereby improving the stability and efficiency of the entire material state visual perception and recognition process, and providing strong support for the intelligent operation of the machine tool loading and unloading process. Attached Figure Description

[0071] Figure 1 This is a flowchart illustrating the material status visual perception and recognition method for machine tool loading and unloading provided in this application embodiment;

[0072] Figure 2 This is a virtual structural diagram of a material status visual perception and recognition system for machine tool loading and unloading provided in an embodiment of this application. Detailed Implementation

[0073] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0074] Furthermore, the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this article, unless otherwise specified, generally indicates that the preceding and following related objects have an "or" relationship.

[0075] The embodiments of this application will now be described in further detail with reference to the accompanying drawings.

[0076] This application provides a material state visual perception and recognition method for machine tool loading and unloading, referring to... Figure 1 The main process of the method is described as follows:

[0077] S1: Collect real-time images of materials in the material box during the machine tool loading and unloading scenario. The real-time images include the shape of the materials and scene environment information.

[0078] The system uses industrial cameras installed at preset positions on the machine tool loading and unloading stations to capture real-time images. The camera's installation height and angle are adjusted according to the size of the material frame and the material placement range to ensure complete coverage of all material areas within the frame. The collected material morphology information includes the material's outline, surface features (such as positioning holes, protruding structures, etc.), and placement posture. Scene environment information includes the station's lighting conditions, whether the material frame shows signs of shaking, and whether the material surface is covered with cutting fluid or oil. The complete collection of the above information provides comprehensive data support for subsequent material status recognition and positioning, avoiding recognition errors due to missing information.

[0079] S2: Call the pre-trained hybrid recognition model to process the real-time image. The hybrid recognition model integrates deep learning object detection and traditional visual feature extraction algorithms, and outputs the initial classification result of the material and feature region identification. The initial classification result includes normal materials to be processed and abnormal materials.

[0080] In the hybrid recognition model, a deep learning object detection algorithm is first used to perform a global scan of the real-time image to quickly identify whether materials exist in the image and their approximate types. Then, a traditional visual feature extraction algorithm is used to initially locate the distinctive features in the image (such as positioning holes and unique contour lines on the material surface). The two work together to output the initial classification results and feature region identifiers. Normal materials to be processed specifically refer to raw materials in the front state that meet the processing requirements. Abnormal materials include empty materials, raw materials on the reverse side, raw materials upside down, and cooked materials, which do not meet the requirements for direct processing. Feature region identifiers are the coordinate ranges of specific areas in the image that contain key features of the material. They are used to lock the core areas for subsequent processing and reduce interference from irrelevant backgrounds.

[0081] S3: Perform branch processing based on the initial classification result: If it is an abnormal material, directly output the status information containing the abnormal type; if it is a normal material to be processed, extract the local image region corresponding to the feature region identifier.

[0082] When the initial classification result is an abnormal material, the output status information must clearly indicate the abnormality type (such as "empty material", "raw material inverted", "cooked material", etc.) so that the machine tool control system can respond quickly and execute the corresponding operation (such as skipping the grab, issuing a prompt, etc.). When the initial classification result is a normal material to be processed, the corresponding local image area is cropped from the real-time image according to the coordinate range of the feature area identifier. This area only retains the key features of the material and the necessary image information around it, and removes irrelevant content such as the edge of the material frame and background debris, thereby improving the efficiency and accuracy of subsequent processing.

[0083] S4: Perform scene adaptation optimization processing on the local image region, the processing including adaptive noise reduction and contrast enhancement based on material surface characteristics;

[0084] Among them, adaptive noise reduction processing needs to be carried out in combination with the surface characteristics of the material. For example, when the material is metal, an appropriate noise reduction method should be selected to address the noise caused by surface reflection and oil stains. This will suppress noise interference while avoiding damage to the key features of the material surface. Contrast enhancement processing adjusts the grayscale distribution of local image areas to increase the grayscale difference between key material features (such as the edges of positioning holes and contour lines) and the background, making the originally blurry features clear and distinguishable, thus laying a good foundation for subsequent feature extraction.

[0085] S5: Extract the contour features of the material and at least two types of key geometric parameters from the optimized local image region, wherein the key geometric parameters include the relative positional relationship between features and the feature size;

[0086] The contour feature extraction is based on the optimized local image. By identifying areas with abrupt changes in grayscale values ​​in the image, the complete edge contour of the material is delineated to ensure that the contour accurately reflects the shape and structure of the material. The extraction of key geometric parameters needs to focus on parameters that play a decisive role in the material's posture and positioning. The relative positional relationship between features may include the distance between two positioning holes on the material surface, the relative position of the positioning holes and the material edge, etc. The feature dimensions may include the diameter of the positioning holes, the length or width of the material contour, etc. These parameters directly reflect the placement posture and actual size of the material and are the core basis for subsequent posture determination.

[0087] S6: Compare the key geometric parameters with the preset standard parameter library in multiple dimensions, and determine the spatial orientation of the material based on the comparison deviation value;

[0088] The preset standard parameter library stores the standard values ​​of key geometric parameters for materials in normal state that meet processing requirements. During multi-dimensional comparison, each extracted key geometric parameter is compared with the corresponding standard value in the standard parameter library one by one, and the deviation value of each parameter is calculated. Based on the sensitivity of material processing to each parameter, all deviation values ​​are comprehensively analyzed. If all deviation values ​​are within the preset allowable range, the material's spatial posture is determined to meet the processing requirements. If any parameter deviation value exceeds the allowable range, the material's spatial posture is determined to be abnormal, ensuring the accuracy of posture determination.

[0089] S7: Integrates the attitude determination result with the coordinate information of the feature area, and outputs a comprehensive recognition result that includes material status and precise positioning;

[0090] The posture determination result includes a clear conclusion of "posture meets requirements" or "posture is abnormal," and the coordinate information of the feature area is the specific location data of the key features in the image coordinate system. The integrated recognition result needs to clearly present the material status (normal waiting to be grasped / abnormal status), posture determination conclusion, and precise positioning information of the material (such as the coordinates of the grasping point, the placement angle of the material, etc.). The integrated recognition result is output in a format that the machine tool control system can recognize, providing precise instructions for the robot to perform loading and unloading actions, ensuring that the robot can accurately grasp the required materials and transfer them to the designated processing position.

[0091] Specifically, in some possible embodiments, the pre-training process of the hybrid recognition model includes:

[0092] Construct a sample library containing multiple working conditions of machine tool loading and unloading scenarios. The samples cover labeled images with different light intensities, material surface conditions, and material frame stacking states.

[0093] A deep learning object detection network was initialized using transfer learning, and trained using the categories and feature regions of materials in the sample library as labeled targets.

[0094] The traditional visual feature extraction module is trained synchronously, and the classification loss of the deep learning network and the feature localization loss of the traditional algorithm are dynamically balanced by the loss function.

[0095] In the pre-training process of the hybrid recognition model, when constructing the sample library, images of various working conditions in machine tool loading and unloading scenarios were systematically collected, covering scenarios such as light intensity (e.g., low light in the early morning, strong light at noon, nighttime workstation lighting and backlighting), material surface condition (clean and free of impurities, with cutting fluid, covered with oil, with processing scratches or rust), and material frame stacking status (single-layer neat placement, multi-layer staggered stacking, material partially obscured or tilted). Each sample was labeled with the material category (e.g., different types of raw material, cooked material, empty material) and the pixel coordinates of feature regions (e.g., positioning holes, edge contours, protruding structures). When initializing the deep learning object detection network using transfer learning, Faster datasets pre-trained on general datasets such as COCO were selected. Using R-CNN or YOLOv5 as the base model, the first 10 convolutional layers are frozen to retain general image feature extraction capabilities. Only the subsequent classification and regression layers are fine-tuned. Material category labels and feature region coordinates in the sample library are used as supervision signals, and the model is iteratively trained using the stochastic gradient descent algorithm. When training the traditional visual feature extraction module simultaneously, SIFT feature point detection and HOG feature description algorithms are integrated. A joint loss function is designed: the classification loss uses cross-entropy loss to calculate the class determination error of the deep learning network, and the localization loss uses IOU loss to calculate the feature region localization deviation of the traditional algorithm. The two types of losses are synergistically optimized through dynamic weight coefficients (the classification loss weight is set to 0.7 in the early stage of training to prioritize convergence of class judgment, and then adjusted to 0.5 in the later stage to balance the accuracy of both) to ensure that the model has both accurate classification and precise localization capabilities.

[0096] Building upon this foundation, during the sample library construction phase, image samples from extreme interference scenarios such as strong metallic reflections and oil stains were incorporated. Additional interference type labels were added during annotation, allowing the model to learn methods for distinguishing between various interference features and effective material features during pre-training. In the transfer learning initialization process, the prior distributions of key geometric parameters such as the diameter of the positioning hole and the feature spacing were embedded into the regression layer initialization parameters of the deep learning network, enabling the network to focus on core features crucial for material posture determination from the initial training phase. When designing the loss function, the deviation between the key geometric parameters and the standard parameter library was introduced as a penalty. When the feature localization results extracted by the traditional algorithm deviate too much from the standard parameters, the localization loss weight is automatically increased, forcing the model to enhance its learning accuracy of key geometric features.

[0097] Specifically, in some possible embodiments, scene adaptation optimization processing is performed on the local image region. This processing includes adaptive noise reduction and contrast enhancement steps based on material surface characteristics, comprising:

[0098] Pixel brightness analysis is performed on the local image area to identify the highlight area and non-highlight area on the material surface, wherein the highlight area is a cluster of high-brightness pixels formed by metal reflection or oil reflection.

[0099] A bilateral filtering algorithm is used to denoise the highlight areas, which suppresses noise while preserving feature edge information by maintaining the edge weight factor.

[0100] For non-highlight areas, a Gaussian filtering algorithm is used for noise reduction. The pixel values ​​are weighted and averaged using a preset Gaussian kernel function to suppress random noise.

[0101] The local image regions after noise reduction are subjected to contrast enhancement processing using local histogram equalization or multiplicative image fusion algorithms to improve the grayscale difference between the feature regions and the background regions. The feature regions are image sub-regions containing preset geometric features.

[0102] In the process of performing scene adaptation optimization for local image regions, pixel brightness analysis is first performed on the local image regions. A sliding window is used to calculate the grayscale value of each pixel and the grayscale distribution of its neighborhood. A dynamic threshold based on the grayscale mean and standard deviation is set (e.g., regions with grayscale values ​​exceeding the mean + 1.8 times the standard deviation and with ≥30 consecutive pixels are classified as highlight regions). This accurately identifies clusters of high-brightness pixels formed by metallic reflections (such as mirror reflections from stainless steel surfaces) or oil stain reflections (such as light spots formed by residual cutting fluid). The remaining regions are classified as non-highlight regions. A bilateral filtering algorithm is used for highlight regions, with a 5×5 or 7×7 spatial Gaussian kernel controlling the filtering range. Simultaneously, a grayscale similarity kernel is used to strengthen the difference weight between high-brightness pixels and surrounding pixels (the similarity factor decreases as the grayscale difference increases), smoothing reflections. While removing noise, the image retains feature edges such as the positioning hole edges and contour lines. For non-highlight areas, a Gaussian filtering algorithm is used, dynamically selecting a Gaussian kernel from 3×3 to 9×9 based on local noise evaluation results (e.g., through variance calculation). Random noise caused by ambient light fluctuations or sensor noise is suppressed through weighted averaging of pixel values. Contrast enhancement is performed on the denoised image. If local histogram equalization is used, the image is divided into 16×16 or 32×32 sub-blocks (sub-blocks in feature-dense areas are reduced), and histogram equalization is performed on each sub-block to enhance local details. If a multiplicative image fusion algorithm is used, the original image and the edge image extracted by the Laplacian operator are fused with weights ranging from 0.6:0.4 to 0.8:0.2, focusing on enhancing the grayscale difference between feature areas containing preset geometric features such as positioning holes and stepped surfaces and the background.

[0103] Building upon this foundation, during the pixel brightness analysis stage, material information (such as the differences in reflective properties between aluminum alloy, cast iron, and plastic) is introduced to establish a material-threshold mapping table. For example, the highlight judgment threshold is lowered for highly reflective materials like aluminum alloy, while the threshold is raised for low-reflective materials like cast iron, making the region segmentation more accurate. During filtering, the feature region types output by the hybrid recognition model (such as positioning holes, edges, and protrusions) are combined, and morphological opening operations (such as 3×3 rectangular structural elements) are additionally superimposed on the highlight regions where positioning holes are located to eliminate the interference of tiny reflective spots on the edge of the hole. For edge regions, an edge sharpening operator (such as the Sobel operator) is added after bilateral filtering to enhance the continuity of the contour. In the contrast enhancement stage, multi-scale processing results are integrated—small-scale (8×8 sub-blocks) enhancement is used for feature regions to highlight details, while large-scale (64×64 sub-blocks) enhancement is used for the overall region to ensure global consistency. At the same time, the sharpness feedback from subsequent contour extraction (such as the mean edge gradient intensity) is introduced to adjust the enhancement parameters (such as sub-block size and fusion weight), forming a closed loop of "analysis-processing-verification".

[0104] Specifically, in some possible embodiments, the step of extracting the contour features of the material and at least two types of key geometric parameters from the optimized local image region, wherein the key geometric parameters include the relative positional relationship between features and the feature size, includes:

[0105] The Sobel operator is used to perform convolution operations on the optimized local image region to obtain the gradient components in the x and y directions;

[0106] Based on the gradient components, the gradient intensity and gradient direction of the pixel are calculated, and the edge is thinned to a single pixel width through non-maximum suppression processing.

[0107] A dual-threshold filtering mechanism is used to retain effective edges. Pixels with gradient strength greater than the high threshold are judged as strong edges, while weak edges with gradient strength between the high and low thresholds and connected to the strong edges are retained.

[0108] Subpixel contours are extracted from materials based on effective edges, and key geometric parameters are extracted from the contours.

[0109] Specifically, a 3×3 Sobel operator is used to perform convolution operations in the x and y directions on the optimized local image region, respectively. This is achieved using the formulas Gx=[f(x+1,y-1)+2f(x+1,y)+f(x+1,y+1)]-[f(x-1,y-1)+2f(x-1,y)+f(x-1,y+1)] and Gy=[f(x-1,y+1)+2f(x,y+1)+f(x... The gradient component of each pixel is calculated using the formula G=√(Gx²+Gy²), and the gradient direction is calculated using the formula θ=arctan(Gy / Gx). Then, each pixel is processed along the gradient direction. For domain comparison, only pixels with local maximum gradient intensity are retained, and edges are refined to single-pixel width. A dual-threshold screening mechanism is adopted, with the high threshold and low threshold set in a ratio of 3:1 to 4:1 (e.g., the high threshold is set to 60 and the low threshold is set to 15, which can be dynamically adjusted according to the image noise level). Strong edges with gradient intensity greater than the high threshold are directly retained, and weak edges with gradient intensity between the high and low thresholds and connected to strong edges are retained through connectivity analysis, while isolated weak edges are removed. Based on the effective edges after screening, the edge curve is fitted using the least squares method, and the contour positioning accuracy is improved to the sub-pixel level through sub-pixel interpolation algorithms (such as bilinear interpolation) to obtain the sub-pixel contour of the material. Then, two key geometric parameters are extracted from the contour: the relative positional relationship between features (such as the center distance between two positioning holes and the vertical distance between the positioning hole and the edge of the material) and the feature size (such as the diameter of the positioning hole and the length and width of the material contour).

[0110] For special industrial scenarios such as material edges with processing gaps, surface scratches, or partial occlusion by chips, the highlight / non-highlight region segmentation results from the scene adaptation optimization stage are introduced before the Sobel operator convolution. Gradient intensity correction coefficients are added to the edge parts corresponding to highlight regions to offset gradient distortion caused by reflection, while the original operator weights are maintained for non-highlight regions. During non-maximum suppression processing, the feature region confidence scores output by the hybrid recognition model are combined, and a stricter local maximum judgment standard (neighborhood range expanded to 5×5) is adopted for core features with high confidence scores (such as the edge of positioning holes) to strengthen the continuity of key edges. In the dual threshold screening stage, the prior range of key geometric parameters in the standard parameter library is embedded. If the parameters corresponding to the initially extracted edges exceed the prior range, the lower threshold is automatically reduced and re-screened to avoid effective edges being mistakenly deleted due to local noise. After sub-pixel contour extraction, a contour completion module is added. For contour breaks caused by gaps or occlusions, the broken parts are completed by segmented matching interpolation using contour templates of similar materials in the standard parameter library, and then the key geometric parameters are extracted.

[0111] Specifically, in some possible embodiments, the step of comparing the key geometric parameters with a preset standard parameter library in multiple dimensions includes:

[0112] Based on the sub-pixel contour, at least two types of key geometric parameters of the material are extracted, wherein:

[0113] To determine the relative positional relationship between features, the relative distance and angle between material surface features are calculated using the coordinates of contour feature points, thereby generating positional relationship parameters.

[0114] For feature dimensions, the length, width, diameter and other dimensional parameters of the feature are calculated by fitting contour edge segments or curves.

[0115] The extracted key geometric parameters are compared item by item with the preset standard parameter library, and the deviation value of each parameter is calculated.

[0116] The deviation value is calculated comprehensively based on the weight of the parameters in determining the material posture.

[0117] Based on the comparison between the comprehensive deviation value and the preset threshold, it is determined whether the material is within the preset spatial orientation range.

[0118] In the step of comparing key geometric parameters with a preset standard parameter library in multiple dimensions, when extracting key geometric parameters based on sub-pixel contours, for the relative positional relationship between features, key feature points (such as the center of the positioning hole, edge endpoints, etc.) are first screened from the contour. Euclidean distance (such as the center distance between two positioning holes) is calculated using coordinate differences, and the angle between the feature lines (such as the angle between the positioning hole line and the long side of the material) is calculated using the vector dot product formula to generate positional relationship parameters. For feature dimensions, the least squares method is used to fit straight lines at the edges of straight segments (such as the side of the material), and the length or width is calculated using the two-point distance formula. For curved edges (such as circular positioning holes), a circular curve is fitted. The diameter is calculated from the average distance from the center of the circle to the contour point, generating dimensional parameters. The extracted parameters are compared item by item with the standard values ​​of the corresponding material model in the standard parameter library to calculate the deviation value (e.g., absolute deviation = |extracted value - standard value|, relative deviation = absolute deviation / standard value). Weights are assigned according to the degree of influence of the parameters on the posture (e.g., the center distance of the positioning hole affects the grasping accuracy, so the weight is set to 0.6, and the edge width weight is set to 0.4). The comprehensive deviation value is calculated by weighted summation. The comprehensive deviation value is compared with the preset threshold (set according to the processing accuracy requirements, such as ±0.5mm). If it is less than or equal to the threshold, the material is determined to be within the preset spatial posture range; otherwise, it is determined to be an abnormal posture.

[0119] For special scenarios such as slight material deformation, local feature occlusion, or mixed production of multiple models, during the parameter extraction stage, the integrity analysis of sub-pixel contours is combined to supplement missing features due to occlusion (such as edges partially obscured by the material frame) by referencing correlation data of similar features in the standard parameter library (such as the fixed ratio of the distance between two positioning holes to the material length), ensuring parameter integrity. During deviation calculation, a dynamic threshold mechanism is introduced, adjusting the threshold based on the material surface condition output by the hybrid recognition model (such as whether there is rust or deformation) (e.g., the threshold for deformed materials is relaxed to ±0.8mm) to avoid overly stringent judgments. Weight allocation adopts an adaptive update strategy, dynamically adjusting the weight coefficients (e.g., increasing the angle parameter weight from 0.3 to 0.5) through correlation analysis of parameter deviations and actual posture errors in historical production data (e.g., statistics show that positioning hole angle deviations have a greater impact on posture). For multi-model scenarios, the corresponding sub-library in the standard parameter library is located using the material classification results of the hybrid recognition model before comparison, reducing cross-model comparison errors.

[0120] Specifically, in some possible embodiments, the steps for constructing the preset standard parameter library include:

[0121] Collect reference images of the same type of material under standard working conditions. The reference images cover the feature data of the material's normal posture and typical abnormal posture.

[0122] Feature extraction is performed on the reference image to obtain the contour features and key geometric parameters under standard conditions, which are then used as reference parameters.

[0123] By statistically analyzing multiple sets of benchmark parameters, the allowable fluctuation range of each parameter is determined, and standard parameter thresholds are formed.

[0124] Classify and store the benchmark parameters and thresholds corresponding to different material types and different working conditions to establish an expandable standard parameter library;

[0125] It supports updating the standard parameter library by adding new benchmark image samples and dynamically adjusting parameter thresholds to adapt to changes in operating conditions.

[0126] The specific steps for constructing the pre-set standard parameter library are as follows: When collecting reference images of materials of the same type, the standard working conditions are set as an environment with no light interference, a clean material surface, no shaking of the material frame, and the material neatly arranged in a single layer. The reference images must completely cover the normal processing posture of the material (such as facing up, horizontal placement) and typical abnormal postures (such as inverted, tilted, partially occluded). No less than 50 samples are collected for each posture to ensure data representativeness. When extracting features from the reference images, a method consistent with the actual recognition process is adopted. First, the image is optimized through scene adaptation, then sub-pixel contours are extracted, and then contour features and key geometric parameters (such as the center distance of positioning holes, hole diameter, material length and width, etc.) are obtained. These parameters are used as reference parameters. Through statistical analysis of multiple sets of reference parameters, the mean and mean of each parameter are calculated using the normal distribution analysis method. The standard deviation is set at ±3 times the standard deviation of the mean as the allowable fluctuation range of the parameter (or adjusted to ±2.5 times the standard deviation of the mean according to the processing accuracy requirements), forming the standard parameter threshold. When storing by category, a primary directory is established by material model, and a secondary directory is established under the same model by working condition type (such as normal lighting, low lighting, and slight oil contamination of the material). The corresponding benchmark parameters, thresholds, and benchmark image thumbnails are stored together. The parameter data is stored in Extensible Markup Language (XML) format for easy reading and modification. When a new benchmark image sample is added, an automatic update process is designed. After the user uploads a new sample, the system automatically completes feature extraction and parameter calculation. By comparing and analyzing with the existing benchmark parameters, the allowable fluctuation range and threshold are dynamically adjusted, which can adapt to changes in working conditions without manual intervention (such as adding a working condition sample with slight scratches on the material surface).

[0127] For special scenarios such as multi-model mixed-line production, frequent fluctuations in operating conditions, and large batch differences in materials, a hybrid recognition model's classification capability is introduced during the baseline image acquisition stage. This automatically filters valid samples (removing blurry or severely occluded invalid images). Simultaneously, environmental parameters such as light intensity and reflectivity are associated with the scene adaptation and optimization process. These environmental parameters are bound and stored with baseline parameters and thresholds, enabling the parameter library to automatically match corresponding thresholds based on the actual environment. In the feature extraction stage, based on the accuracy feedback from key geometric parameter extraction, baseline parameters are hierarchically labeled (e.g., the center distance of positioning holes is a primary core parameter, and the chamfer size of material edges is a secondary secondary parameter). Different allowable fluctuation ranges are set for each level, with stricter thresholds for core parameters to ensure accuracy. For processing accuracy, during statistical analysis, benchmark data from multiple batches of materials are integrated to establish a batch-parameter mapping table. For dimensional deviations in different batches of materials (such as slight differences in the diameter of positioning holes in different batches of the same model), batch-specific thresholds are automatically generated to avoid attitude misjudgment caused by cross-batch comparisons. In terms of update mechanism, closed-loop calibration logic is introduced—data such as the accuracy of attitude determination and the success rate of robot grasping in actual production are fed back to the parameter library. If the misjudgment rate exceeds the preset value (such as 5%) under a certain working condition, the system automatically triggers supplementary acquisition of benchmark images and recalculation of thresholds. At the same time, it is linked with the parameter adjustment for scene adaptation optimization (such as when the misjudgment rate is high under reflective conditions, the filtering algorithm parameters and the threshold in the parameter library are optimized simultaneously).

[0128] Specifically, in some possible embodiments, the step of integrating the attitude determination result with the coordinate information of the feature region to output a comprehensive identification result including material status and precise positioning includes:

[0129] The material status is divided into two categories: normal and awaiting capture, and abnormal. A unique identifier code is assigned to each status.

[0130] Calculate the three-dimensional coordinates of the material grabbing point based on the coordinate information of the feature area identifier;

[0131] Output digital signals containing status identification codes, three-dimensional coordinates of the gripping point, and attitude angles to the machine tool control system;

[0132] Visual information, including bounding boxes of feature regions, values ​​of key geometric parameters, and textual annotations of state determination results, is overlaid on real-time images.

[0133] If the condition is determined to be abnormal, an explanation of the abnormality type and corresponding handling suggestions will be output synchronously.

[0134] The material status is clearly divided into two categories: normal and abnormal. A unique identifier code (e.g., 001) is assigned to the normal and abnormal status, and a basic identifier code (e.g., 002) is assigned to the abnormal status, with additional suffixes added according to the abnormality type (e.g., 002-1 represents empty material, 002-2 represents raw material inverted, and 002-3 represents cooked material). Based on the two-dimensional pixel coordinates of the feature area identifier, combined with the pre-calibrated intrinsic parameters (focal length, principal point coordinates) and extrinsic parameters (relative position and orientation of the camera and the material frame) of the industrial camera, the pixel coordinates are converted into three-dimensional coordinates in the world coordinate system through a perspective transformation algorithm. The material's center of gravity or a preset gripping reference point (e.g., the midpoint of the line connecting the centers of the positioning holes) is selected as the gripping point, and its three-dimensional coordinates are calculated. Modbus is used. The TCP or Profinet industrial communication protocol outputs standardized digital signals to the machine tool control system. These signals include status identification codes, three-dimensional coordinates of the gripping point (X, Y, and Z axis values), and material posture angles (rotation angles around the X, Y, and Z axes), ensuring the control system can quickly analyze and respond. Visual information is overlaid on the real-time image, with feature area boundaries marked by red rectangles. Key geometric parameters (such as 8mm diameter of positioning holes and 50mm center distance) are marked inside the rectangles or at the corners of the image. Status judgment results such as "Normal, waiting to be gripped" in green or "Abnormal - raw material inverted" in red are marked, facilitating intuitive monitoring by staff. If an abnormal state is determined, a textual description of the abnormality type (such as "Large area of ​​oil stains on the material surface, feature extraction failed") and targeted handling suggestions (such as "Clean the oil stains on the material surface and re-inspect" or "Manually adjust the material posture to face up") are output simultaneously.

[0135] For special scenarios such as simultaneous multi-material identification, high-speed production line operation, and emergency handling of extreme anomalies, in multi-material identification scenarios, the system combines the multi-target detection capabilities of a hybrid identification model, assigns an independent ID to each material, calculates the state, grasping point coordinates, and posture angle of each material, and outputs a comprehensive correlation result including multiple material IDs, states, and locations, supporting robot grasping according to priority. For high-speed production line operation scenarios, the system optimizes the digital signal output strategy, adopting a cache pre-storage + trigger-based transmission mechanism. As soon as the machine tool control system completes the processing of the previous material, it immediately pushes the identification result of the next material, shortening production line waiting time. Simultaneously, the system synchronizes the visualized information to the central monitoring platform of the production line, supporting remote real-time viewing from multiple workstations. Regarding extreme anomaly handling, the system integrates the anomaly detection capabilities of a standard parameter library. Within a constant threshold range, when the overall deviation value far exceeds the preset threshold (e.g., the deviation of deformable material parameters exceeds 30%), a graded alarm is automatically triggered (minor anomalies only provide text prompts, while severe anomalies trigger a production line halt), and emergency handling solutions for similar materials in the parameter library are invoked (e.g., prompting unrepairable deformable materials to be sorted to the waste area). A new closed-loop feedback mechanism is added, which transmits the robot's actual grasping success rate and positioning deviation data back to the system, dynamically adjusting the calculation weight of the grasping point coordinates (e.g., increasing the coordinate weight of surrounding feature points when the grasping deviation of a certain positioning hole is large), and simultaneously linking with the scene adaptation optimization process, automatically enhancing image noise reduction and contrast enhancement for scenes with frequent anomalies (e.g., continuous reflection at a certain workstation), forming a full-link collaboration of "recognition output - execution feedback - parameter optimization".

[0136] Another embodiment of this application provides a visual perception and recognition system for material status during machine tool loading and unloading, wherein, see reference Figure 2 A visual perception and recognition system for material status during machine tool loading and unloading includes:

[0137] Image acquisition module 100: used to acquire real-time images of materials in the material box during machine tool loading and unloading, the real-time images including the shape of the materials and scene environment information;

[0138] Hybrid recognition model processing module 200: used to call a pre-trained hybrid recognition model to process the real-time image. The hybrid recognition model integrates deep learning object detection and traditional visual feature extraction algorithms, and outputs the initial classification result of the material and feature region identification. The initial classification result includes normal materials to be processed and abnormal state materials.

[0139] Branch processing module 300: used to perform branch processing based on the initial classification result: if it is an abnormal material, directly output the status information containing the abnormal type; if it is a normal material to be processed, extract the local image region corresponding to the feature region identifier;

[0140] Scene adaptation optimization module 400: used to perform scene adaptation optimization processing on the local image region, the processing including adaptive noise reduction and contrast enhancement based on material surface characteristics;

[0141] Feature extraction module 500: used to extract the contour features of the material and at least two types of key geometric parameters from the optimized local image region, wherein the key geometric parameters include the relative positional relationship between features and the feature size;

[0142] Attitude determination module 600: used to compare the key geometric parameters with a preset standard parameter library in multiple dimensions, and determine the spatial attitude of the material based on the comparison deviation value;

[0143] Result Integration Output Module 700: This module integrates the attitude determination results with the coordinate information of the feature region, and outputs a comprehensive recognition result that includes material status and precise positioning.

[0144] The material status visual perception and recognition system for machine tool loading and unloading provided in this embodiment can achieve the steps of the aforementioned embodiments due to the functions of each module and the logical connections between them. Therefore, it can achieve the same technical effect as the aforementioned embodiments. For the principle analysis, please refer to the relevant description of the steps of the aforementioned material status visual perception and recognition method for machine tool loading and unloading, which will not be repeated here.

[0145] This application embodiment also provides a material state visual perception and recognition device for machine tool loading and unloading, including a memory and a processor. The memory stores a computer program that can be loaded by the processor and executed by the above-described material state visual perception and recognition method for machine tool loading and unloading.

[0146] This application embodiment also provides a storage medium storing a computer program that can be loaded by a processor and executed by the above-described material state visual perception and recognition method for machine tool loading and unloading.

[0147] The storage medium provided in this embodiment can achieve the same technical effect as the aforementioned embodiments because the computer program therein, after being loaded and run on the processor, will implement the various steps of the aforementioned embodiments. For the principle analysis, please refer to the relevant description of the aforementioned method steps, which will not be repeated here.

[0148] The storage medium includes, for example, various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0149] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0150] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0151] Furthermore, features defined by the terms "first" and "second" may explicitly or implicitly include at least one of those features. In the description of this invention, "a plurality of" means at least two, such as two, three, etc., and unless otherwise explicitly specified, is used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features.

[0152] Therefore, any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing custom logic functions or processes, and the scope of the preferred embodiments of the invention includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as will be understood by those skilled in the art to which embodiments of the invention pertain.

[0153] The embodiments described in this specific implementation are preferred embodiments of this application and are not intended to limit the scope of protection of this application. Therefore, all equivalent changes made in accordance with the structure, shape and principle of this application should be covered within the scope of protection of this application.

Claims

1. A material state visual sense recognition method for feeding and discharging of a machine tool, characterized in that, The method comprises the following steps: Collecting a real-time image of the material in the material frame in the machine tool feeding and discharging scene, wherein the real-time image contains the shape of the material and the scene environment information; Calling a pre-trained hybrid recognition model to process the real-time image, wherein the hybrid recognition model combines deep learning target detection and traditional visual feature extraction algorithm, and outputs the initial classification result of the material and the feature region label, and the initial classification result includes normal material to be processed and abnormal state material; According to the initial classification result, branch processing is performed: if it is an abnormal state material, the state information containing the abnormal type is directly output; if it is a normal material to be processed, the local image region corresponding to the feature region label is extracted; Scene adaptive optimization processing is performed on the local image region, which includes adaptive noise reduction and contrast enhancement based on the surface characteristics of the material; Pixel brightness analysis is performed on the local image region to identify the highlight area and non-highlight area on the surface of the material, wherein the highlight area is a high brightness pixel cluster formed by metal reflection or oil stain reflection; The highlight area is subjected to noise reduction processing by using a bilateral filtering algorithm, which suppresses noise while retaining feature edge information by maintaining edge weight factor; The non-highlight area is subjected to noise reduction processing by using a Gaussian filtering algorithm, which suppresses random noise by weighting and averaging pixel values through a preset Gaussian kernel function; Contrast enhancement processing is performed on the noise-reduced local image region by using local histogram equalization or multiplicative image fusion algorithm to improve the gray difference between the feature region and the background region, wherein the feature region is an image sub-region containing preset geometric features; The contour features and at least two types of key geometric parameters of the material are extracted from the optimized local image region, wherein the key geometric parameters include the relative position relationship between features and the feature size; Sobel operator is used to perform convolution operation on the optimized local image region to obtain gradient components in x direction and y direction; Based on the gradient components, the gradient strength and gradient direction of the pixel points are calculated, and the edges are refined to single-pixel width through non-maximum suppression processing; A double-threshold screening mechanism is used to retain effective edges, and pixels with gradient strength greater than the high threshold are determined as strong edges, and weak edges with gradient strength between the high and low thresholds and connected with the strong edges are retained; Based on the effective edges, the sub-pixel contour of the material is extracted, and the key geometric parameters are extracted from the contour; The key geometric parameters are compared with the preset standard parameter library in multiple dimensions, and the spatial pose of the material is determined combined with the comparison deviation value; The pose determination result and the coordinate information of the feature region are integrated to output the comprehensive recognition result containing the material state and accurate positioning.

2. The method of claim 1, wherein, The pre-training process of the hybrid recognition model comprises: Constructing a sample library containing multiple working conditions of the machine tool feeding and discharging scene, wherein the samples cover labeled images of different light intensities, material surface states and material frame stacking states; Using transfer learning to initialize the deep learning target detection network, and training the network with the categories and feature regions of the materials in the sample library as labeled targets; Synchronously training the traditional visual feature extraction module, and dynamically balancing the classification loss of the deep learning network and the feature positioning loss of the traditional algorithm through the loss function.

3. The method of claim 1, wherein, The step of multi-dimensionally comparing the key geometric parameters with the preset standard parameter library comprises: Based on the sub-pixel contour, at least two types of key geometric parameters of the material are extracted, wherein: For the relative position relationship between features, the relative distance and angle of the material surface features are calculated through the contour feature point coordinates to generate the position relationship parameters; For feature size, the length, width, diameter and other size parameters of the feature are calculated by fitting the contour edge line segment or curve; The extracted key geometric parameters are compared with the preset standard parameter library item by item, and the deviation value of each parameter is calculated; According to the influence weight of the parameters on the material posture determination, the deviation values are comprehensively calculated; According to the comparison result of the comprehensive deviation value and the preset threshold value, it is determined whether the material is within the preset space posture range.

4. The method of claim 1, wherein, The construction steps of the preset standard parameter library comprise: Collect the reference image of the same model material under standard working conditions, and the reference image covers the feature data of normal posture and typical abnormal posture of the material; Feature extraction is performed on the reference image to obtain the contour features and key geometric parameters under the standard state as reference parameters; Through statistical analysis of multiple groups of reference parameters, the allowable fluctuation range of each parameter is determined to form the standard parameter threshold; The reference parameters and threshold values corresponding to different material models and different working conditions are stored in categories to establish an extensible standard parameter library; Supporting updating the standard parameter library by adding reference image samples, dynamically adjusting the parameter threshold to adapt to the working condition change.

5. A material state visual sense knowledge recognition system for feeding and discharging of a machine tool, characterized by, Comprise: An image acquisition module is configured to acquire real-time images of materials in a material frame in a machine tool loading and unloading scene, wherein the real-time images contain the shape of the materials and the scene environment information; A mixed recognition model processing module is configured to call a pre-trained mixed recognition model to process the real-time images, wherein the mixed recognition model integrates deep learning target detection and traditional visual feature extraction algorithm, and outputs initial classification results of the materials and feature region labels, wherein the initial classification results include normal materials to be processed and abnormal state materials; A branch processing module is configured to perform branch processing according to the initial classification results: if the material is an abnormal state material, directly output state information containing the abnormal type; if the material is a normal material to be processed, extract the local image region corresponding to the feature region label. The scene adaptation optimization module is configured to perform scene adaptation optimization processing on the local image region, the processing including adaptive noise reduction and contrast enhancement based on material surface characteristics; perform pixel brightness analysis on the local image region to identify high-light regions and non-high-light regions of the material surface, wherein the high-light region is a high-brightness pixel cluster formed by metal reflection or oil stain reflection; perform noise reduction processing on the high-light region using a bilateral filtering algorithm to suppress noise while retaining feature edge information by maintaining an edge weight factor; perform noise reduction processing on the non-high-light region using a Gaussian filtering algorithm to suppress random noise by weighting and averaging pixel values using a preset Gaussian kernel function; and perform contrast enhancement processing on the noise-reduced local image region using local histogram equalization or multiplication image fusion algorithm to improve the gray difference between the feature region and the background region, wherein the feature region is an image sub-region containing a preset geometric feature. The feature extraction module is configured to extract contour features and at least two types of key geometric parameters of the material from the optimized local image region, the key geometric parameters including relative position relationships between features and feature sizes; perform convolution operation on the optimized local image region using a Sobel operator to obtain gradient components in x and y directions; calculate gradient strength and gradient direction of a pixel based on the gradient components, and thin the edges to a single-pixel width through non-maximum suppression processing; retain effective edges using a double-threshold screening mechanism, and determine pixels with gradient strength greater than a high threshold as strong edges, and retain weak edges with gradient strength between the high and low thresholds and connected to the strong edges; extract sub-pixel contours of the material based on the effective edges, and extract key geometric parameters from the contours; The posture determination module is configured to compare the key geometric parameters with a preset standard parameter library in multiple dimensions, and determine the spatial posture of the material in combination with comparison deviation values. The result integration and output module is configured to integrate the posture determination result and coordinate information of the feature region, and output a comprehensive recognition result including the material state and accurate positioning.

6. A material state visual sensing recognition device for material loading and unloading of a machine tool, characterized by, The method comprises the following steps: A memory and a processor, the memory storing a computer program capable of being loaded and executed by the processor to implement the material state visual sensing recognition method for machine tool feeding and discharging according to any one of claims 1-4.

7. A storage medium, characterized by The memory stores a computer program capable of being loaded and executed by the processor to implement the material state visual sensing recognition method for machine tool feeding and discharging according to any one of claims 1-4.

Images