Image processing method for industrial robots based on image fusion

Abstract

This invention provides an image processing method for industrial robots based on image fusion. [Solution] The method involves acquiring an image of the object to be detected awaiting processing by an industrial robot, removing noise components, retaining edge and detail information, calculating and evaluating the image noise reduction coefficient, selecting and fusing the images, converting them into an industrial robot fused image, generating grayscale features based on this, calculating and evaluating the visibility coefficient of the occluded area, labeling the current fused image with an "occluded warning" label according to the results, analyzing the color and shape features of the fused image, constructing a color diversity feature coefficient and a shape complexity coefficient, comparing them with pre-set thresholds for color features and shape complexity, evaluating the fit in the diversity dimension of the current object, comprehensively analyzing each coefficient of the object in the image, calculating and evaluating the feature matching degree in the warehouse environment, generating a recognition report and executing a selection operation if the results meet the set conditions, and labeling it as unrecognized and triggering a re-recognition command if the conditions are not met.

Description

Technical Field

[0001] This application relates to the technical field of image fusion, and specifically, it is an image processing method for industrial robots based on image fusion.

Background Art

[0002] The background art related to the image processing method of industrial robots dates back to the rapid development of image processing and computer vision in the mid-20th century. Initially, it was applied in fields such as industrial inspection and quality control. The initial image processing technology mainly relied on conventional algorithms such as edge detection and morphology. However, with the improvement of computer processing capabilities, its application scope has expanded to more complex scenes. Entering the 21st century, due to the leapfrog development of deep learning technology, the application of image recognition and processing to industrial robots has accelerated, and robots have become able to perform tasks such as precise recognition, sorting, and assembly on the production line. In recent years, with the development of technologies such as 3D vision, real-time processing, and cloud computing, the performance of industrial robot image processing has been further improved, and its application in emerging fields such as automated warehouses and flexible manufacturing has been further expanded. The integration and optimization of these technologies have promoted the smartness and efficiency improvement of industrial robots, and have become an important support for the development of the intelligence and automation of modern industry.

[0003] However, although the image processing method of industrial robots applied in automated warehouses has made remarkable progress in terms of intelligence and efficiency, there are still some technical problems in image processing. Specifically, the following three points can be mentioned.

[0004] 1. Object occlusion problem. In automated warehouses, stacking and occlusion phenomena of goods generally exist. It is difficult for robots to accurately recognize occluded objects. Especially when the discrimination ability of image processing technology for multi-layer occluders is not fully optimized, it is easy to cause errors in sorting and recognition.

[0005] 2. Interference due to image noise. In warehouse environments, factors such as dust and vibration can easily generate image noise. Conventional image noise reduction algorithms remove noise, but at the same time, they may lose important image detail information, potentially reducing the accuracy of subsequent image recognition. For example, barcode recognition is prone to scanning failures due to the effects of noise.

[0006] 3. Challenges due to the diversity of object shapes and colors. Items in automated warehouses are diverse in type, shape, and color. Conventional image processing algorithms have limited effectiveness in recognizing objects with complex shapes and colors, requiring training and optimization using large amounts of annotated data, which increases costs and processing difficulty. [Overview of the project] [Problems that the invention aims to solve]

[0007] In light of the shortcomings of prior art, this invention provides an image processing method for industrial robots based on image fusion that solves technical shortcomings in the background art, such as object occlusion problems, interference due to image noise, and challenges due to the diversity of object shapes and colors. [Means for solving the problem]

[0008] To achieve the above objective, the image processing method for industrial robots based on image fusion according to the proposed technology adopted in this application is: Step 1 involves placing the object to be detected within the detection area of ​​an automated warehouse, collecting real-time images of the object from different viewpoints and heights, and obtaining images of the object awaiting processing by an industrial robot. Step 2 involves preprocessing the collected images awaiting processing by industrial robots, processing the images using adaptive multilayer noise reduction technology, removing noise components using a noise reduction decomposition algorithm while retaining edge and detail information of the images awaiting processing by industrial robots, calculating and evaluating the image noise reduction coefficient Ncj, and selecting the images awaiting processing by industrial robots. Step 3 involves performing fusion processing on the images awaiting industrial robot processing after noise reduction and sorting, converting images from multiple viewpoints into an industrial robot fused image using a hierarchical fusion algorithm based on deep learning, generating grayscale features based on the industrial robot fused image, calculating the visibility coefficient Zjy of the occluded area, evaluating the visibility coefficient Zjy using a pre-set threshold, labeling the current fused image as "occluded warning" according to the evaluation result, and issuing a notification signal. Step 4 involves performing color and shape feature analysis on industrial robot fusion images, constructing a color diversity feature coefficient Ssc and a shape complexity coefficient Xtx using a color histogram and shape geometric feature extraction method, presetting a color feature threshold E and a shape complexity threshold R, and comparing them with the resulting color diversity feature coefficient Ssc and shape complexity coefficient Xtx to evaluate the goodness of fit in the current diversity dimension of the object. Step 5 involves comprehensively analyzing the visibility coefficient Zjy, image noise reduction coefficient Ncj, color diversity feature coefficient Ssc, and shape complexity coefficient Xtx of objects in the industrial robot fusion image, calculating and evaluating the feature matching degree Tzpd of the detected object in the warehouse environment, generating a recognition report and executing a sorting operation if the evaluation result meets pre-set conditions, and otherwise labeling it as unrecognized in the system and triggering a re-recognition command. Step 6 includes the following: if object recognition does not meet the criteria, the system automatically corrects the object feature matching algorithm in real time based on past data and information on previously recognized objects, thereby re-recognizing the target object, and in this process, continuously updates and adjusts the object feature model using incremental learning and comparative learning techniques.

[0009] Preferably, step 1 specifically means: First, the object to be detected is placed within the detection area of ​​the automated warehouse, and the lighting conditions, background interference elements, and sensor state within the detection area of ​​the automated warehouse are adjusted to meet pre-set criteria. Next, the initial spatial contour of the detection area is acquired using a three-dimensional laser radar and a depth camera, and the position, size, and orientation of the object to be detected within the detection area are determined using a spatial point cloud analysis algorithm. A step of generating collection paths for multiple viewpoints and multiple heights based on information corresponding to posture, Finally, the process includes the step of acquiring images of the object to be detected in real time and obtaining images awaiting processing by an industrial robot.

[0010] Preferably, step 2 specifically means: First, a noise reduction decomposition algorithm is used to remove noise components from the image awaiting processing by the industrial robot, while simultaneously preserving edge and detail information of the image awaiting processing by the industrial robot through adaptive adjustment. Next, the image noise reduction coefficient Ncj is calculated, and the image gradient Grd, image noise standard deviation Noi, and image signal-to-noise ratio Sig are obtained in real time using an image processing algorithm that includes edge detection, noise estimation, and statistical analysis, and the formula is used. The steps involve calculating the image noise reduction coefficient Ncj by combining JPEG2026109591000002.jpg15161, and Next, the process includes the step of comparing and evaluating the image noise reduction coefficient Ncj using a pre-set noise reduction threshold Q, and selecting images awaiting processing by an industrial robot after noise reduction. As for specific evaluation criteria, If the image noise reduction coefficient Ncj ≥ noise reduction threshold Q, then the noise reduction effect of the current industrial robot processing image meets the pre-set criteria, and the current image is further analyzed as a valid image. If the image noise reduction coefficient Ncj < noise reduction threshold Q, the noise reduction effect of the current industrial robot awaiting processing does not meet the pre-set criteria. Therefore, the current image is removed from processing, processed further, or the image is re-collected.

[0011] Preferably, step 3 is, specifically, Before converting images from multiple viewpoints into industrial robot fusion images, the image data processing method is used to convert the bitmap images to a non-bitmap format with richer features, and frequency domain processing techniques including Fourier transform and discrete wavelet transform are used to extract frequency component information and texture features of the images. The process involves using a hierarchical fusion algorithm to extract and synthesize features from images of industrial robots awaiting processing from different viewpoints, and then combining these images into a single fused industrial robot image. Based on industrial robot fusion images, grayscale features and occluded area-related data, including image grayscale value Gra, occluded area Occ, and image viewpoint difference value Vie, are extracted, the visibility of the occluded area is analyzed, and the image grayscale value Gra, occluded area Occ, and image viewpoint difference value Vie are extracted, and the formula The process includes the step of calculating the visibility coefficient Zjy of the occluded area by combining JPEG2026109591000003.jpg15161.

[0012] Preferably, step 3 is, specifically, The process further includes the step of labeling an industrial robot fusion image and emitting a presentation signal by comparing and evaluating a pre-set occlusion threshold W and a visibility coefficient Zjy. As for specific evaluation criteria, If the visibility coefficient Zjy ≥ the occlusion threshold W, the effect of occlusion regions on visibility in the industrial robot fusion image is normal, and it is labeled as "passing". If the visibility coefficient Zjy < occlusion threshold W, the impact on visibility due to the occlusion area in the industrial robot fusion image is abnormal, so it is labeled as an "occlusion alarm" and a notification signal is sent to the administrator.

[0013] Preferably, step 4 is, specifically, First, the industrial robot fusion image is subjected to a color space conversion, and the original industrial robot fusion image is converted back to the RGB color space. Next, through the color histogram, data on the distribution frequency of each color in the fused image, including the red channel average value MeaA, the green channel average value MeaB, the blue channel average value MeaC, the red channel variance value VarA, the green channel variance value VarB, the blue channel variance value VarC, and the color saturation Sat, is extracted. Next, the formula The step of calculating the color diversity feature coefficient Ssc using JPEG2026109591000004.jpg16165, Next, the step of extracting the contour shape of the object using edge detection and image segmentation techniques, The geometric morphology analysis method is used to represent the complexity of the shape. Specifically, object shape-related data is collected and extracted in real time, and the formula The step of calculating the shape complexity coefficient Xtx using JPEG2026109591000005.jpg16163 is included. In the formula, Per represents the contour length in the object shape-related data, Are represents the contour area in the object shape-related data, Com represents the contour compactness in the object shape-related data, and Irr represents the shape irregularity in the object shape-related data.

[0014] Preferably, in step 4 above, As the specific evaluation content of the color diversity feature coefficient Ssc and the shape complexity coefficient Xtx, The color diversity feature coefficient Ssc is compared with a preset color feature threshold E. If the color diversity feature coefficient Ssc ≥ the color feature threshold E, the color diversity of the object in the current industrial robot fused image is suitable, the color features of the object are rich and diverse, and the requirements of the diversity dimension are met. If the color diversity feature coefficient Ssc < the color feature threshold E, the color diversity of the object in the current industrial robot fused image does not match, and the color features are single, so further adjustment is required. The shape complexity coefficient Xtx is compared with a preset shape complexity threshold R. If the shape complexity coefficient Xtx ≥ the shape complexity threshold R, the shape complexity of the object in the current industrial robot fused image is suitable, the shape of the object is complex, and the requirements of the diversity dimension are met. If the shape complexity coefficient Xtx < the shape complexity threshold R, the shape complexity of the object in the current industrial robot fusion image does not conform, and since the shape of the object is simple, further adjustment is required.

[0015] Preferably, step 4 specifically is Finally, based on the evaluation results of color and shape, it further includes the step of comprehensively judging the degree of conformity of the object in the industrial robot fusion image in the diversity dimension. Specifically, If the coefficients in both color diversity evaluation and shape complexity evaluation are each greater than or equal to the corresponding threshold, it is determined that the object in the industrial robot fusion image meets the diversity requirements. If either the color diversity feature coefficient Ssc or the shape complexity coefficient is less than the corresponding threshold, it is determined that the object in the industrial robot fusion image does not meet the diversity requirements and further adjustment is necessary.

[0016] Preferably, in step 5, specifically Extract the visibility coefficient Zjy, the image noise removal coefficient Ncj, the color diversity feature coefficient Ssc, and the shape complexity coefficient Xtx, and use the formula Include the step of calculating the feature matching degree Tzpd using JPEG2026109591000006.jpg10162. In the formula, f() is a pre-set matching degree calculation function, which comprehensively considers the influence of the four sub-parameters of the visibility coefficient Zjy, the image noise removal coefficient Ncj, the color diversity feature coefficient Ssc, and the shape complexity coefficient Xtx. Preferably, step 5 specifically is Further include the step of pre-setting the feature matching threshold T, comparing and evaluating the feature matching degree Tzpd with the feature matching threshold T, and determining whether the recognition requirements are met. As specific evaluation content If the feature matching degree Tzpd ≥ the feature matching threshold T, since the recognition of the object meets the standard, a recognition report is generated and a sorting operation is executed. If the feature matching score Tzpd < feature matching threshold T, the object recognition does not meet the criteria, so the system labels it as "unrecognized" and triggers a re-acquisition command to perform image re-acquisition.

[0017] Preferably, step 6 is, specifically, After triggering a re-recognition command, the process first extracts image data of currently unrecognized objects, compares and analyzes it with past recognition data, uses comparative learning techniques to recognize differences in key features of similar objects, and generates a feature correction parameter set. Next, the object feature model is updated using incremental learning techniques, the feature weight assignments of the image noise reduction coefficient Ncj, the color diversity feature coefficient Ssc, and the shape complexity coefficient Xtx are adjusted, and the feature matching algorithm is optimized. In addition, during the image data processing stage, the image to be detected is converted from a bitmap format to a non-bitmap format, and texture features and frequency component information of the image are extracted using a frequency domain processing method to improve the recognition accuracy of the object to be detected under complex occlusion or noise conditions. Next, the process involves combining non-bitmap data with a deep learning model, regenerating grayscale features and visibility coefficients Zjy for the fused image, and evaluating the feature matching degree Tzpd in the object's storage environment using a feature matching algorithm that dynamically adjusts these features. If the re-evaluation results meet the pre-set criteria, the system generates a recognition report and performs a sorting operation. Otherwise, the process includes labeling the current image as unrecognized and triggering further system-level optimization mechanisms.

[0018] This invention provides an image processing method for industrial robots based on image fusion. This invention has the following beneficial effects.

[0019] (1) This image processing method for industrial robots based on image fusion successfully solves the object occlusion problem in automated warehouses by using image fusion technology, noise reduction algorithms, and multidimensional feature analysis. In warehouse environments, stacking of goods and occlusion phenomena are common. Conventional image processing technology has difficulty accurately recognizing occluded objects, and in particular, in the case of multi-layer occlusion, it is prone to errors in sorting and recognition. Through data fusion technology and deep learning hierarchical fusion algorithms, this invention effectively fuses images from different viewpoints to generate clear, occluded object images. Furthermore, by calculating and evaluating the visibility coefficient Zjy of the occluded area, the system can issue an "occluded area alarm" in a timely manner, avoiding misrecognition due to occlusion and ensuring the accuracy and completeness of image processing.

[0020] (2) This image fusion-based image processing method for industrial robots addresses the issue of interference due to image noise in warehouse environments by introducing adaptive multilayer noise reduction technology. It removes noise components from the image using a noise reduction decomposition algorithm while preserving edge and detail information of the image, thus avoiding the loss of important image detail information during noise reduction in conventional noise reduction algorithms. According to this invention, the image noise reduction coefficient Ncj is calculated in real time and compared with a preset noise reduction threshold Q to effectively evaluate the noise reduction effect and select high-quality, effective images with less noise interference. This improves the accuracy of subsequent recognition and processing and avoids problems such as barcode recognition failures due to image noise.

[0021] (3) In this image fusion-based image processing method for industrial robots, to address the challenges posed by the diversity of object shapes and colors, the present invention constructs a color diversity feature coefficient Ssc and a shape complexity coefficient Xtx, and comprehensively improves the recognition capability for complex objects by using a color histogram-based analysis method and a geometric morphological analysis method. In this invention, the degree of diversity in the color and shape of an object can be determined by comparing it with a pre-set color feature threshold E and a shape complexity threshold R, and if the criteria are met, the object is considered to satisfy the requirements of the diversity dimension. This method makes it possible to recognize objects with various shapes and colors, overcoming the problem of conventional image processing algorithms' poor adaptability to the diversity of object shapes and colors, reducing the demand for data annotation, lowering the difficulty of processing, and significantly improving the recognition efficiency and accuracy of automated warehouse systems. [Brief explanation of the drawing]

[0022] [Figure 1] This flowchart shows the steps of an image processing method for industrial robots based on image fusion. [Modes for carrying out the invention]

[0023] The following describes the technical proposal in the embodiments of this application clearly and completely, in combination with the accompanying drawings. It is clear that the embodiments described are not all embodiments of this application, but only a selection. All other embodiments that a person skilled in the art could obtain without creative work based on the embodiments of this application are within the scope of protection of this application. [Examples]

[0024] Example 1 Referring to Figure 1, the image processing method for industrial robots based on image fusion is: Step 1 involves placing the object to be detected within the detection area of ​​an automated warehouse, collecting real-time images of the object from different viewpoints and heights, and obtaining images of the object awaiting processing by an industrial robot. Step 2 involves preprocessing the collected images awaiting processing by industrial robots, processing the images using adaptive multilayer noise reduction technology, removing noise components using a noise reduction decomposition algorithm while retaining edge and detail information of the images awaiting processing by industrial robots, calculating and evaluating the image noise reduction coefficient Ncj, and selecting the images awaiting processing by industrial robots. Step 3 involves performing fusion processing on the images awaiting industrial robot processing after noise reduction and sorting, converting images from multiple viewpoints into an industrial robot fused image using a hierarchical fusion algorithm based on deep learning, generating grayscale features based on the industrial robot fused image, calculating the visibility coefficient Zjy of the occluded area, evaluating the visibility coefficient Zjy using a pre-set threshold, labeling the current fused image as "occluded warning" according to the evaluation result, and issuing a notification signal. Step 4 involves performing color and shape feature analysis on industrial robot fusion images, constructing a color diversity feature coefficient Ssc and a shape complexity coefficient Xtx using a color histogram and shape geometric feature extraction method, presetting a color feature threshold E and a shape complexity threshold R, and comparing them with the resulting color diversity feature coefficient Ssc and shape complexity coefficient Xtx to evaluate the goodness of fit in the current diversity dimension of the object. Step 5 involves comprehensively analyzing the visibility coefficient Zjy, image noise reduction coefficient Ncj, color diversity feature coefficient Ssc, and shape complexity coefficient Xtx of objects in the industrial robot fusion image, calculating and evaluating the feature matching degree Tzpd of the detected object in the warehouse environment, generating a recognition report and executing a sorting operation if the evaluation result meets pre-set conditions, and otherwise labeling it as unrecognized in the system and triggering a re-collection command. Step 6 includes the following: if object recognition does not meet the criteria, the system automatically corrects the object feature matching algorithm in real time based on past data and information on previously recognized objects, thereby re-recognizing the target object, and in this process, continuously updates and adjusts the object feature model using incremental learning and comparative learning techniques.

[0025] In this embodiment, according to step 1, the problem of difficulty in recognition due to object occlusion in complex warehouse environments is solved by collecting images from multiple viewpoints and at multiple heights.

[0026] According to Step 2, adaptive multilayer denoising technology is used in combination with a denoising decomposition algorithm to effectively remove noise while preserving image edge and detail information. The image denoising coefficient Ncj is also calculated and evaluated to ensure that the image quality meets the processing requirements and improves the accuracy of subsequent image recognition.

[0027] According to Step 3, images of industrial robots awaiting processing from different viewpoints are fused based on a hierarchical fusion algorithm based on deep learning to generate high-quality fused industrial robot images. At the same time, the visibility coefficient Zjy of the occluded area is calculated to accurately recognize the occluded area, improve recognition accuracy, and issue an "occluded area warning" in a timely manner.

[0028] According to Step 4, the color and shape features of the industrial robot fusion image are analyzed, and the color diversity feature coefficient Ssc and shape complexity coefficient Xtx are calculated and evaluated by combining the color histogram and geometric morphological methods. This evaluates the color and shape diversity of objects and improves the recognition ability for complex objects.

[0029] According to Step 5, each feature coefficient is comprehensively analyzed to calculate the feature matching score Tzpd, and based on pre-set conditions, it is determined whether or not the recognition requirements are met. If they are met, a recognition report is generated and the sorting operation is performed; otherwise, a re-collection command is triggered. This ensures high accuracy of the recognition results and high efficiency of the system.

[0030] According to Step 6, through comparison and analysis of historical data, the system can improve recognition accuracy and robustness by adjusting and optimizing the object feature matching algorithm in real time. Incremental learning techniques enable continuous updating of the object feature model, ensuring that the system maintains relatively high recognition accuracy in different environments and conditions. Comparative learning techniques enhance the adaptability of object features, allowing the system to automatically correct and adapt to new recognition demands when faced with new objects or changing environments, reducing the occurrence of misrecognition and recognition omissions. Furthermore, automatic optimization allows the system to recover quickly after object recognition failures, reducing the need for human intervention and improving overall selection efficiency and intelligence. [Examples]

[0031] Example 2 Step 1 specifically involves: The process includes the steps of: first, placing the object to be detected within the detection area of ​​the automated warehouse and adjusting the lighting conditions, background interference elements, and sensor state within the detection area of ​​the automated warehouse to meet pre-set criteria; second, acquiring the initial spatial contour of the detection area using a three-dimensional laser radar and a depth camera, and determining the position, size, and orientation of the object to be detected within the detection area using a spatial point cloud analysis algorithm; generating collection paths for multiple viewpoints and multiple heights based on the information corresponding to the orientation; and finally, acquiring images of the object to be detected in real time and obtaining images awaiting processing by an industrial robot.

[0032] Step 2 specifically involves: First, a noise reduction decomposition algorithm is used to remove noise components from the image awaiting processing by the industrial robot, while simultaneously preserving edge and detail information of the image awaiting processing by the industrial robot through adaptive adjustment. Next, an image noise reduction coefficient Ncj is calculated, and the image gradient Grd, image noise standard deviation Noi, and image signal-to-noise ratio Sig are acquired in real time using an image processing algorithm that includes edge detection, noise estimation, and statistical analysis. The steps involve calculating the image noise reduction coefficient Ncj by combining JPEG2026109591000007.jpg15161, and Next, the process includes a step of comparing and evaluating the image noise reduction coefficient Ncj using a pre-set noise reduction threshold Q, and selecting images awaiting processing by an industrial robot after noise reduction, with specific evaluation details as follows: If the image noise reduction coefficient Ncj ≥ noise reduction threshold Q, then the noise reduction effect of the current industrial robot processing image meets the pre-set criteria, and the current image is further analyzed as a valid image. If the image noise reduction coefficient Ncj < noise reduction threshold Q, the noise reduction effect of the current industrial robot awaiting processing does not meet the pre-set criteria. Therefore, the current image is removed from processing, processed further, or the image is re-collected.

[0033] In this embodiment, the integrity and accuracy of image data of the object to be detected are achieved by using multiple viewpoint and multiple height viewpoint techniques, solving the problems of object occlusion and uncertainty of position. In the image noise reduction stage, noise is removed while retaining edge and detail information of the image using a noise reduction decomposition algorithm and adaptive adjustment method, and the image noise reduction coefficient Ncj is calculated. By combining the image gradient Grd, the image noise standard deviation Noi, and the image signal-to-noise ratio Sig, image quality is further guaranteed, and misrecognition due to the influence of noise is avoided. The image noise reduction coefficient Ncj is evaluated using a pre-set noise reduction threshold Q, and unacceptable images are removed in a timely manner, thereby avoiding the influence of low-quality data on subsequent recognition and decision processes and improving the robustness and reliability of the system. [Examples]

[0034] Example 3 Step 3 specifically involves: Before converting images from multiple viewpoints into an industrial robot fusion image, the bitmap image is converted to a non-bitmap format with richer features using an image data processing method that converts from bitmap to non-bitmap, and frequency domain processing techniques including Fourier transform and discrete wavelet transform are used to extract frequency component information and texture features of the image. A hierarchical fusion algorithm is used to extract and synthesize features from images awaiting industrial robot processing from different viewpoints, and the images awaiting industrial robot processing from multiple viewpoints are synthesized into a single industrial robot fusion image. Based on the industrial robot fusion image, grayscale features and occlusion region-related data, including the image grayscale value Gra, occlusion area Occ, and image viewpoint difference value Vie, are extracted, the visibility of the occlusion region is analyzed, and the image grayscale value Gra, occlusion area Occ, and image viewpoint difference value Vie are extracted, and the formula The method includes the step of calculating the visibility coefficient Zjy of the occluded area by combining JPEG2026109591000008.jpg15161.

[0035] Step 3 specifically involves: The process further includes a step of labeling the industrial robot fusion image and emitting a presentation signal by comparing and evaluating a pre-set occlusion threshold W and a visibility coefficient Zjy, with specific evaluation details including: If the visibility coefficient Zjy ≥ the occlusion threshold W, the effect of occlusion regions on visibility in the industrial robot fusion image is normal, and it is labeled as "passing". If the visibility coefficient Zjy < occlusion threshold W, the impact on visibility due to the occlusion area in the industrial robot fusion image is abnormal, so it is labeled as an "occlusion alarm" and a notification signal is sent to the administrator.

[0036] Step 4 specifically involves: First, a color space conversion is performed on the industrial robot fusion image to convert the original industrial robot fusion image to the RGB color space. Next, data on the distribution frequency of each color in the fusion image is extracted through a color histogram, including the mean red channel MeaA, mean green channel MeaB, mean blue channel MeaC, red channel variance VarA, green channel variance VarB, blue channel variance VarC, and color saturation Sat. Then, the formula is applied. The steps involve calculating the color diversity feature coefficient Ssc using JPEG2026109591000009.jpg16165, Next, the process involves extracting the object's contour shape using edge detection and image segmentation techniques, and representing the complexity of the shape using geometric morphological analysis methods. Specifically, object shape-related data is collected and extracted in real time, and the formula is used. The process includes the step of calculating the shape complexity coefficient Xtx using JPEG2026109591000010.jpg16163, In the formula, Per represents the contour length in the object shape-related data, Are represents the contour area in the object shape-related data, Com represents the contour compactness in the object shape-related data, and Irr represents the shape irregularity in the object shape-related data.

[0037] In step 4, The specific evaluation methods for the color diversity feature coefficient Ssc and the shape complexity coefficient Xtx are as follows: The color diversity feature coefficient Ssc is compared with a pre-set color feature threshold E. If the color diversity feature coefficient Ssc ≥ color feature threshold E, then the color diversity of objects in the current industrial robot fusion image is suitable, the color features of the objects are rich and diverse, and the requirements of the diversity dimension are met. If the color diversity feature coefficient Ssc < color feature threshold E, then the color diversity of objects in the current industrial robot fusion image does not match, and the color feature is single, so further adjustment is needed. The shape complexity coefficient Xtx is compared with a pre-set shape complexity threshold R. If the shape complexity coefficient Xtx ≥ shape complexity threshold R, then the shape complexity of the object in the current industrial robot fusion image is suitable, the object's shape is complex, and the diversity dimension requirement is met. If the shape complexity coefficient Xtx < shape complexity threshold R, the shape complexity of the object in the current industrial robot fusion image does not match, and the object's shape is simple, so further adjustments are needed.

[0038] In this embodiment, a hierarchical fusion algorithm is used to synthesize multi-view image data into a single industrial robot fused image, thereby effectively improving the overall information content and accuracy of the image. The image grayscale value Gra, occlusion area Occ, and image viewpoint difference value Vie are extracted and combined with the visibility coefficient Zjy of the occlusion area to analyze and evaluate the visibility of the occlusion area in the image, thereby ensuring the visibility effect of the fused image in occlusion situations. By setting an occlusion threshold W and comparing it with the visibility coefficient Zjy, the system's intelligent recognition capability and real-time response are enhanced by recognizing occlusion alarms in a timely manner and issuing a notification signal. In the color and shape feature analysis stage, the average values ​​of the three color channels (red, green, and blue) and the shape complexity coefficient Xtx are extracted using a color histogram to comprehensively evaluate the color diversity and shape complexity of the object, and by comparing it with a pre-set color feature threshold E and shape complexity threshold R, it is ensured that the object meets the diversity requirements. Ultimately, the system's intelligence level and flexibility were improved by comprehensively determining the degree of fit in the diversity dimension of objects based on the evaluation results of color and shape, thereby ensuring the accuracy and adaptability of object recognition and classification. [Examples]

[0039] Example 4 Step 5 specifically involves: The visibility coefficient Zjy, image noise reduction coefficient Ncj, color diversity feature coefficient Ssc, and shape complexity coefficient Xtx are extracted, and the formula The process includes a step of calculating the feature matching score Tzpd using JPEG2026109591000011.jpg10162, where f() is a pre-configured matching score calculation function that comprehensively considers the influence of four sub-parameters: the visibility coefficient Zjy, the image noise reduction coefficient Ncj, the color diversity feature coefficient Ssc, and the shape complexity coefficient Xtx. Step 5 is, The process further includes pre-setting a feature matching threshold T, comparing and evaluating the feature matching degree Tzpd with the feature matching threshold T, and determining whether or not the recognition requirements are met. The specific evaluation content includes: If the feature matching degree Tzpd ≥ feature matching threshold T, then the object recognition meets the criteria, so a recognition report is generated and the sorting operation is performed. If the feature matching score Tzpd < feature matching threshold T, the object recognition does not meet the criteria, so the system labels it as "unrecognized" and triggers a re-acquisition command to perform image re-acquisition.

[0040] Step 6 specifically involves: After triggering a re-recognition command, the process first extracts image data of currently unrecognized objects, compares and analyzes it with past recognition data, uses comparative learning techniques to recognize differences in key features of similar objects, and generates a feature correction parameter set. Next, it updates the object feature model using incremental learning techniques, adjusts the feature weight assignments of the image noise reduction coefficient Ncj, the color diversity feature coefficient Ssc, and the shape complexity coefficient Xtx, and optimizes the feature matching algorithm. In addition, the image data processing stage includes the steps of: converting the target image from bitmap format to non-bitmap format, extracting texture features and frequency component information of the image using a frequency domain processing method to improve the recognition accuracy of the target object under complex occlusion or noise conditions; then combining the non-bitmap data with a deep learning model, regenerating grayscale features and visibility coefficients Zjy for the fused image, and evaluating the degree of feature matching Tzpd of the object in its storage environment using a dynamically adjusted feature matching algorithm; if the re-evaluation results meet pre-set criteria, generating a recognition report and executing a selection operation in the system; otherwise, labeling the current image as unrecognized and further triggering a system-level optimization mechanism.

[0041] In this embodiment, the visibility coefficient Zjy, image noise reduction coefficient Ncj, color diversity feature coefficient Ssc, and shape complexity coefficient Xtx are extracted, and the feature matching degree Tzpd is calculated to comprehensively evaluate the feature matching status of an object from multiple dimensions. The collection and calculation of each subparameter ensures accurate recognition of the object under different environments and processing conditions. The visibility coefficient Zjy reflects the visibility effect of occluded areas in the image, the image noise reduction coefficient Ncj ensures the preservation of image quality and detail information, the color diversity feature coefficient Ssc quantitatively represents the color diversity of the object, and the shape complexity coefficient Xtx represents the complexity of the object's shape. By comparing and evaluating the feature matching degree Tzpd with a pre-set feature matching threshold T, the system can accurately determine whether an object meets the recognition criteria. If it does, it generates a recognition report and executes a sorting operation, improving the accuracy and degree of automation of recognition. If it does not meet the criteria, it is labeled "unrecognized" and a re-collection command is triggered. This optimized the overall operating process and the efficiency of intelligent recognition to ensure the reliability of object recognition and the flexibility of system response.

[0042] In object recognition, factors such as changes in lighting, occlusion, image noise, or changes in morphology can lead to recognition failures or misrecognitions. According to Step 6, by optimizing the object feature matching algorithm in real time and adjusting it based on historical data and information on previously recognized objects, the system can quickly adapt to changing environments and recognition conditions, increasing the system's robustness and avoiding system failures or efficiency reductions in the selection process due to initial misrecognitions. By introducing incremental and comparative learning techniques, the system can dynamically update the object feature model and optimize the recognition algorithm through continuous learning. This adaptive learning capability means that the system can gradually improve recognition accuracy over time and with accumulated experience, reducing human intervention and corrections.

[0043] While embodiments of the present application have been illustrated and described, those skilled in the art will understand that various modifications, alterations, substitutions, and variations can be made to these embodiments without departing from the principles and spirit of the present application, and that the scope of the present application is limited by the appended claims and equivalents.

Claims

1. Image processing method for industrial robots based on image fusion, Step 1 involves placing the object to be detected within the detection area of ​​an automated warehouse, collecting real-time images of the object from different viewpoints and heights, and obtaining images of the object awaiting processing by an industrial robot. Step 2 involves preprocessing the collected images awaiting processing by industrial robots, processing the images awaiting processing by industrial robots using adaptive multilayer noise reduction technology, removing noise components using a noise reduction decomposition algorithm while retaining edge and detail information of the images awaiting processing by industrial robots, calculating and evaluating the image noise reduction coefficient Ncj, and selecting the images awaiting processing by industrial robots. Step 3 involves performing fusion processing on the images awaiting industrial robot processing after noise reduction and sorting, converting images from multiple viewpoints into an industrial robot fused image using a hierarchical fusion algorithm based on deep learning, generating grayscale features based on the industrial robot fused image, calculating the visibility coefficient Zjy of the occluded area, evaluating the visibility coefficient Zjy using a pre-set threshold, labeling the current fused image as "occluded warning" according to the evaluation result, and issuing a notification signal. Step 4 involves performing color and shape feature analysis on industrial robot fusion images, constructing a color diversity feature coefficient Ssc and a shape complexity coefficient Xtx using a color histogram and shape geometric feature extraction method, presetting a color feature threshold E and a shape complexity threshold R, and comparing them with the resulting color diversity feature coefficient Ssc and shape complexity coefficient Xtx to evaluate the fit in the current diversity dimension of the object. Step 5 involves comprehensively analyzing the visibility coefficient Zjy, image noise reduction coefficient Ncj, color diversity feature coefficient Ssc, and shape complexity coefficient Xtx of objects in the industrial robot fusion image, calculating and evaluating the feature matching degree Tzpd of the detected object in the warehouse environment, generating a recognition report and executing a sorting operation if the evaluation result meets pre-set conditions, and otherwise labeling it as unrecognized in the system and triggering a re-recognition command. Step 6 involves the system automatically correcting the object feature matching algorithm in real time by optimizing it based on past data and information on previously recognized objects, thereby re-recognizing the target object, and in this process, continuously updating and adjusting the object feature model using incremental learning techniques. An image processing method for industrial robots based on image fusion, characterized by including [a specific element].

2. Step 1 is, First, the object to be detected is placed within the detection area of ​​the automated warehouse, and the lighting conditions, background interference elements, and sensor state within the detection area of ​​the automated warehouse are adjusted to meet pre-set criteria. Next, the initial spatial contour of the detection area is acquired using a three-dimensional laser radar and a depth camera, and the position, size, and orientation of the object to be detected within the detection area are determined using a spatial point cloud analysis algorithm. A step of generating collection paths for multiple viewpoints and multiple heights based on information corresponding to posture, Finally, the process includes the step of acquiring images of the object to be detected in real time and obtaining images awaiting processing by an industrial robot. Image processing method for industrial robots based on image fusion as described in feature 1.

3. Step 2 is, First, a noise reduction decomposition algorithm is used to remove noise components from the image awaiting processing by the industrial robot, while simultaneously preserving edge and detail information of the image awaiting processing by the industrial robot through adaptive adjustment. Next, the image noise reduction coefficient Ncj is calculated, and the image gradient Grd, image noise standard deviation Noi, and image signal-to-noise ratio Sig are acquired in real time using an image processing algorithm that includes edge detection, noise estimation, and statistical analysis. The steps include: calculating the image noise reduction coefficient Ncj by combining these steps; Next, the process includes the step of comparing and evaluating the image noise reduction coefficient Ncj using a pre-set noise reduction threshold Q, and selecting images awaiting processing by an industrial robot after noise reduction. As for specific evaluation criteria, If the image noise reduction coefficient Ncj ≥ noise reduction threshold Q, then the noise reduction effect of the current industrial robot processing image meets the pre-set criteria, and the current image is further analyzed as a valid image. If the image noise reduction coefficient Ncj < noise reduction threshold Q, the noise reduction effect of the current industrial robot awaiting processing does not meet the pre-set criteria. Therefore, the current image is removed from processing, processed further, or the image is re-collected. Image processing method for industrial robots based on image fusion as described in feature 1.

4. Step 3 above is, Before converting images from multiple viewpoints into industrial robot fusion images, the image data processing method is used to convert the bitmap images to a non-bitmap format with richer features, and frequency domain processing techniques including Fourier transform and discrete wavelet transform are used to extract frequency component information and texture features of the images. The process involves using a hierarchical fusion algorithm to extract and synthesize features from images of industrial robots awaiting processing from different viewpoints, and then combining these images into a single fused industrial robot image. Based on industrial robot fusion images, grayscale features and occluded area-related data are extracted, including the image grayscale value Gra, occluded area Occ, and image viewpoint difference value Vie. The visibility of the occluded area is analyzed, and the image grayscale value Gra, occluded area Occ, and image viewpoint difference value Vie are extracted, and the formula The steps include: calculating the visibility coefficient Zjy of the occluded area by combining the following; Image processing method for industrial robots based on image fusion as described in feature 1.

5. Step 3 above is, The process further includes the step of labeling an industrial robot fusion image and emitting a presentation signal by comparing and evaluating a pre-set occlusion threshold W and a visibility coefficient Zjy. As for specific evaluation criteria, If the visibility coefficient Zjy ≥ the occlusion threshold W, the effect of occlusion regions on visibility within the industrial robot fusion image is normal, and it is labeled as "passing." If the visibility coefficient Zjy < occlusion threshold W, the impact of occlusion on visibility in the industrial robot fusion image is abnormal, so it is labeled as an "occlusion alarm" and a notification signal is sent to the administrator. Image processing method for industrial robots based on image fusion as described in feature 1.

6. Step 4 is, First, a color space conversion is performed on the industrial robot fusion image, and the original industrial robot fusion image is converted to the RGB color space. Next, data on the distribution frequency of each color in the fused image is extracted through a color histogram, including the mean red channel MeaA, mean green channel MeaB, mean blue channel MeaC, red channel variance VarA, green channel variance VarB, blue channel variance VarC, and color saturation Sat. Then, the formula is used. The steps include: calculating the color diversity feature coefficient Ssc using; Next, the process involves extracting the contour shape of an object using edge detection and image segmentation techniques. The complexity of the shape is represented using geometric morphological analysis methods, specifically by collecting and extracting object shape-related data in real time, and formulating an equation. The steps include: calculating the shape complexity coefficient Xtx using In the formula, Per represents the contour length in the object shape-related data, Are represents the contour area in the object shape-related data, Com represents the contour compactness in the object shape-related data, and Irr represents the shape irregularity in the object shape-related data. Image processing method for industrial robots based on image fusion as described in feature 1.

7. In step 4, The specific evaluation methods for the color diversity feature coefficient Ssc and the shape complexity coefficient Xtx are as follows: The color diversity feature coefficient Ssc is compared with a pre-set color feature threshold E. If the color diversity feature coefficient Ssc ≥ color feature threshold E, then the color diversity of objects in the current industrial robot fusion image is suitable, the color features of the objects are rich and diverse, and the requirements of the diversity dimension are met. If the color diversity feature coefficient Ssc < color feature threshold E, then the color diversity of objects in the current industrial robot fusion image does not match, and the color feature is single, so further adjustment is necessary. The shape complexity coefficient Xtx is compared with a pre-set shape complexity threshold R. If the shape complexity coefficient Xtx ≥ shape complexity threshold R, then the shape complexity of the object in the current industrial robot fusion image is suitable, the object's shape is complex, and the diversity dimension requirement is met. If the shape complexity coefficient Xtx < shape complexity threshold R, then the shape complexity of the object in the current industrial robot fusion image is not suitable, and the object's shape is simple, so further adjustment is necessary. Image processing method for industrial robots based on image fusion as described in feature 1.

8. Step 4 is, Finally, the process further includes a step of comprehensively determining the degree of fit in the diversity dimension of objects within the industrial robot fusion image based on the evaluation results of color and shape, specifically, If the coefficients for both color diversity evaluation and shape complexity evaluation are above the corresponding thresholds, it is determined that the objects in the industrial robot fusion image meet the diversity requirements. If either the color diversity feature coefficient Ssc or the shape complexity coefficient is smaller than the corresponding threshold, it is determined that the objects in the industrial robot fusion image do not meet the diversity requirements and require further adjustment. Image processing method for industrial robots based on image fusion as described in feature 7.

9. Step 5 above is, The visibility coefficient Zjy, image noise reduction coefficient Ncj, color diversity feature coefficient Ssc, and shape complexity coefficient Xtx are extracted, and the formula The step includes calculating the feature matching degree Tzpd using the following method: In the formula, f() is a pre-defined matching degree calculation function that comprehensively considers the influence of four sub-parameters: the visibility coefficient Zjy, the image noise reduction coefficient Ncj, the color diversity feature coefficient Ssc, and the shape complexity coefficient Xtx. Step 5 above is, The process further includes pre-setting a feature matching threshold T, comparing and evaluating the feature matching degree Tzpd with the feature matching threshold T, and determining whether or not the recognition requirements are met. As for specific evaluation criteria, If the feature matching degree Tzpd ≥ feature matching threshold T, then the object recognition meets the criteria, so a recognition report is generated and the sorting operation is performed. If the feature matching degree Tzpd < feature matching threshold T, then the object recognition does not meet the criteria, so the system labels it as "unrecognized" and triggers a re-recognition command. Image processing method for industrial robots based on image fusion as described in feature 1.

10. Step 6 is, After triggering a re-recognition command, the process first extracts image data of currently unrecognized objects, compares and analyzes it with past recognition data, uses comparative learning techniques to recognize differences in key features of similar objects, and generates a feature correction parameter set. Next, the object feature model is updated using incremental learning techniques, the feature weight assignments of the image noise reduction coefficient Ncj, the color diversity feature coefficient Ssc, and the shape complexity coefficient Xtx are adjusted, and the feature matching algorithm is optimized. In addition, during the image data processing stage, the image to be detected is converted from a bitmap format to a non-bitmap format, and texture features and frequency component information of the image are extracted using a frequency domain processing method to improve the recognition accuracy of the object to be detected under complex occlusion or noise conditions. Next, the process involves combining non-bitmap data with a deep learning model, regenerating grayscale features and visibility coefficients Zjy for the fused image, and evaluating the feature matching degree Tzpd in the object's storage environment using a feature matching algorithm that dynamically adjusts these features. If the re-evaluation results meet the pre-set criteria, the system generates a recognition report and performs a sorting operation. Otherwise, the current image is labeled as unrecognized, and the following steps are taken to further trigger a system-level optimization mechanism: Image processing method for industrial robots based on image fusion as described in feature 1.

Images