An industrial robot image processing method based on image fusion

Abstract

The present invention discloses an industrial robot image processing method based on image fusion, which relates to the technical field of image fusion. Through real-time multi-angle image acquisition and deep learning hierarchical fusion algorithm, the present invention effectively solves the problem of object occlusion in warehousing. By evaluating the visibility coefficient Zjy of the occlusion area, the recognition error caused by occlusion is avoided; by calculating the image denoising coefficient Ncj in real time, the image quality is ensured and the subsequent recognition accuracy is improved; for the diversification of shapes and colors, the present invention calculates the color diversification feature coefficient Ssc and the shape complexity coefficient Xtx, and combines the color histogram and geometric morphology analysis to comprehensively improve the recognition ability of complex objects; by comparing with the preset threshold, the diversification compliance of the object is judged, and objects with various shape and color combinations are effectively recognized, reducing the need for labeled data and improving the recognition efficiency and accuracy of the automated warehousing system.

Description

Technical Field

[0001] The present invention relates to the technical field of image fusion, and specifically provides an image processing method for industrial robots based on image fusion. Background Art

[0002] The background art of the image processing method for industrial robots originated from the rapid development of image processing and computer vision in the mid-20th century, and was initially applied to fields such as industrial inspection and quality control. Early image processing technologies mainly relied on traditional algorithms such as edge detection and morphology. With the improvement of computer processing power, they were gradually applied to more complex scenarios. Entering the 21st century, the breakthrough of deep learning technology has promoted the application of image recognition and processing in industrial robots, enabling robots to perform precise recognition, sorting, assembly and other tasks in the production line. In recent years, the development of technologies such as 3D vision, real-time processing, and cloud computing has further improved the performance of industrial robot image processing, making its application in complex environments more extensive, such as emerging fields like automated warehousing and flexible manufacturing. The integration and optimization of these technologies have promoted the intelligence and efficiency improvement of industrial robots, providing important support for the intelligent and automated development of modern industry.

[0003] However, although the image processing method for industrial robots applied to automated warehousing has made significant progress in terms of intelligence and efficiency, there are still some technical drawbacks in image processing, specifically including the following three aspects:

[0004] 1. Object occlusion problem: In automated warehousing, the stacking and occlusion of goods are common. It is difficult for robots to accurately identify occluded objects, especially when the image processing technology has not fully optimized the ability to distinguish multi-level occluders, which easily leads to sorting and recognition errors.

[0005] 2. Image noise interference: In the warehousing environment, factors such as dust and vibration are likely to introduce image noise. While traditional image denoising algorithms remove noise, they may lose important image details, resulting in a decrease in the accuracy of subsequent image recognition. For example, barcode recognition is easily affected by noise, leading to scanning failures.

[0006] 3. Challenges of diverse object shapes and colors: There is a wide variety of items in automated warehousing, with diverse shapes and colors. Traditional image processing algorithms have limited effects in identifying objects with complex shape and color combinations, and require a large amount of labeled data for learning and optimization, increasing the cost and processing difficulty. Summary of the Invention

[0007] Aiming at the deficiencies of the prior art, the present invention provides an image processing method for industrial robots based on image fusion, which solves the technical drawbacks of object occlusion problems, image noise interference, and challenges of diverse object shapes and colors in the background art.

[0008] To achieve the above object, the present invention is implemented through the following technical solutions: An industrial robot image processing method based on image fusion, comprising the following steps:

[0009] Step 1: Place the object to be detected in the automated warehousing detection area, and perform real-time image acquisition of the object to be detected from different angles and heights to obtain the image to be processed by the industrial robot;

[0010] Step 2: Preprocess the image to be processed by the industrial robot. Use the adaptive multi-layer denoising technology to process the image to be processed by the industrial robot. Remove the noise components through the denoising decomposition algorithm, and at the same time retain the edge and detail information of the image to be processed by the industrial robot. Calculate the image denoising coefficient Ncj and evaluate it, and screen the image to be processed by the industrial robot;

[0011] Step 3: Perform fusion processing on the denoised and screened image to be processed by the industrial robot. Use the hierarchical fusion algorithm based on deep learning to convert the images from multiple perspectives into the industrial robot fusion image; then, generate gray-scale features based on the industrial robot fusion image and calculate the visibility coefficient Zjy of the occluded area; Evaluate the visibility coefficient Zjy through a preset threshold, and mark the current fusion image as "occlusion alarm" according to the evaluation content and send a prompt signal;

[0012] Step 4: Analyze the color and shape features of the industrial robot fusion image. Construct the color diversification feature coefficient Ssc and the shape complexity coefficient Xtx through the color histogram and shape geometric feature extraction methods, and respectively preset the color feature threshold E and the shape complexity threshold R, and compare them with the color diversification feature coefficient Ssc and the shape complexity coefficient Xtx to evaluate the compliance degree of the current object in the diversification dimension;

[0013] Step 5: Comprehensively analyze the visibility coefficient Zjy, the image denoising coefficient Ncj, the color diversification feature coefficient Ssc, and the shape complexity coefficient Xtx of the object in the industrial robot fusion image, calculate the feature matching degree Tzpd of the object to be detected in the warehousing environment and evaluate it. If the evaluation result meets the preset conditions, generate an identification report and perform a sorting operation; otherwise, the system is marked as unrecognized and a re-identification instruction is triggered;

[0014] Step 6: When the object recognition does not meet the standard, the system will automatically correct and re-identify the object to be detected by real-time optimizing the object feature matching algorithm according to the historical data and the previously recognized object information; During the process, incremental learning and contrast learning technologies are used to continuously update and adjust the object feature model.

[0015] Optionally, Step 1 specifically includes:

[0016] First, place the object to be detected within the automated warehousing detection area, and adjust the lighting conditions, background interference factors, and sensor status within the automated warehousing detection area to the preset standards; then use a 3D lidar and a depth camera to obtain the initial spatial contour of the detection area, and determine the position, size, and pose of the object to be detected in the detection area through a spatial point cloud analysis algorithm; and generate multi-angle and multi-height acquisition paths according to the information corresponding to the pose; finally, perform real-time image acquisition on the object to be detected to obtain the image to be processed by the industrial robot.

[0017] Optionally, step two specifically includes:

[0018] First, use a denoising decomposition algorithm to remove the noise components in the image to be processed by the industrial robot, and at the same time retain the edge and detail information of the image to be processed by the industrial robot through adaptive adjustment; then calculate the image denoising coefficient Ncj, and use image processing algorithms, including edge detection, noise estimation, and statistical analysis, to obtain the image gradient Grd, the image noise standard deviation Noi, and the image signal-to-noise ratio Sig in real time, and calculate and obtain the image denoising coefficient Ncj by combining the following formula:

[0019]

[0020] Then, compare and evaluate the image denoising coefficient Ncj through a preset denoising threshold Q, and screen the denoised image to be processed by the industrial robot. The specific evaluation content is as follows:

[0021] If the image denoising coefficient Ncj ≥ the denoising threshold Q, it means that the denoising effect of the current image to be processed by the industrial robot meets the preset standards. At this time, the current image is used as a valid image and further analyzed;

[0022] If the image denoising coefficient Ncj < the denoising threshold Q, it means that the denoising effect of the current image to be processed by the industrial robot does not meet the preset standards. At this time, the current image is excluded, and further processed or the image is re-acquired.

[0023] Optionally, step three specifically includes:

[0024] Before converting images from multiple perspectives into an industrial robot fusion image, an image data processing method from bit image to non-bit image is adopted to convert the bit image into a non-bit image form with richer features. Frequency domain processing techniques including Fourier transform and discrete wavelet transform are used to extract the frequency component information and texture features of the image. The hierarchical fusion algorithm extracts and synthesizes features from the industrial robot images to be processed from different perspectives, and synthesizes the industrial robot images to be processed from multiple perspectives into an industrial robot fusion image. Based on the industrial robot fusion image, gray features and data related to the occluded area are extracted, including the image gray value Gra, the occlusion area Occ, and the image view angle difference value Vie. The visibility of the occluded area is analyzed, and the image gray value Gra, the occlusion area Occ, and the image view angle difference value Vie are extracted. The visibility coefficient Zjy of the occluded area is calculated by combining the following formula:

[0025] Optionally, step three specifically further includes:

[0026] The industrial robot fusion image is marked and a prompt signal is sent by comparing and evaluating the preset occlusion degree threshold W with the visibility coefficient Zjy. The specific evaluation content is as follows:

[0027] If the visibility coefficient Zjy ≥ the occlusion degree threshold W, it means that the occluded area in the industrial robot fusion image has a normal impact on visibility, and it is marked as the "qualified" state at this time;

[0028] If the visibility coefficient Zjy < the occlusion degree threshold W, it means that the occluded area in the industrial robot fusion image has an abnormal impact on visibility, and it is marked as "occlusion alarm" at this time, and a prompt signal is sent to the management personnel.

[0029] Optionally, step four specifically includes:

[0030] First, the color space of the industrial robot fusion image is converted, and the original industrial robot fusion image is converted into the RGB color space; then, the frequency data of each color distribution in the fusion image is extracted through the color histogram, including the red channel mean MeaA, the green channel mean MeaB, the blue channel mean MeaC, the red channel variance VarA, the green channel variance VarB, the blue channel variance VarC, and the color saturation Sat. Then, the color diversification feature coefficient Ssc is calculated by using the following formula:

[0031] Then, the contour shape of the object is extracted through edge detection and image segmentation techniques; a geometric morphology analysis method is used to describe the complexity of the shape, and the data related to the object shape is collected and extracted in real time, and the shape complexity coefficient Xtx is calculated. The specific calculation formula is as follows: Wherein, Per represents the contour length in the object shape-related data, Are represents the contour area in the object shape-related data, Com represents the contour compactness in the object shape-related data, and Irr represents the shape irregularity in the object shape-related data.

[0032] Optionally, step four specifically further includes:

[0033] The specific evaluation contents of the color diversification feature coefficient Ssc and the shape complexity coefficient Xtx are as follows:

[0034] Compare the color diversification feature coefficient Ssc with a preset color feature threshold E:

[0035] If the color diversification feature coefficient Ssc ≥ the color feature threshold E, it indicates that the object color diversity in the current industrial robot fusion image meets the requirement, the object color features are rich and diverse, and meet the diversification dimension requirements;

[0036] If the color diversification feature coefficient Ssc < the color feature threshold E, it indicates that the object color diversity in the current industrial robot fusion image does not meet the requirement, the color features are single, and further adjustment is required at this time;

[0037] Compare the shape complexity coefficient Xtx with a preset shape complexity threshold R:

[0038] If the shape complexity coefficient Xtx ≥ the shape complexity threshold R, it indicates that the shape complexity of the object in the current industrial robot fusion image meets the requirement, the object shape is complex, and meets the diversification dimension requirements;

[0039] If the shape complexity coefficient Xtx < the shape complexity threshold R, it indicates that the shape complexity of the object in the current industrial robot fusion image does not meet the requirement, the object shape is simple, and further adjustment is required at this time.

[0040] Optionally, step four specifically further includes:

[0041] Finally, comprehensively consider the evaluation results of color and shape to judge the compliance degree of the object in the industrial robot fusion image in terms of diversification dimension:

[0042] If both the color diversification evaluation and the shape complexity evaluation meet their respective thresholds, the object in the industrial robot fusion image meets the diversification requirements;

[0043] If any one of the evaluations of the color diversification feature coefficient Ssc and the shape complexity coefficient Xtx does not meet the requirement, it means that the object in the industrial robot fusion image does not meet the diversification requirements and further adjustment is required.

[0044] Optionally, step five specifically includes:

[0045] Extract the visibility coefficient Zjy, the image denoising coefficient Ncj, the color diversification feature coefficient Ssc, and the shape complexity coefficient Xtx, and calculate the feature matching degree Tzpd through the following formula: In the formula, is a preset matching degree calculation function, which comprehensively considers the influences of the four subordinate parameters: the visibility coefficient Zjy, the image denoising coefficient Ncj, the color diversification feature coefficient Ssc, and the shape complexity coefficient Xtx.

[0046] Step five specifically further includes:

[0047] Preset a feature matching threshold T, and compare and evaluate the feature matching degree Tzpd with the feature matching threshold T to determine whether the recognition requirements are met. The specific evaluation content is as follows:

[0048] If the feature matching degree Tzpd ≥ the feature matching threshold T, it means that the object recognition meets the standard, and an identification report is generated and the sorting operation is triggered;

[0049] If the feature matching degree Tzpd < the feature matching threshold T, it means that the object recognition does not meet the standard. At this time, the system is marked as "unrecognized", and a re-acquisition instruction is triggered and re-image acquisition is performed.

[0050] Optionally, step six specifically includes:

[0051] After triggering the re-identification instruction, first extract the image data of the currently unrecognized object, and perform a comparative analysis in combination with the historical recognition data. Identify the key feature differences of similar objects through comparative learning technology to generate a feature correction parameter set; secondly, use incremental learning technology to update the object feature model, adjust the feature weight distribution of the image denoising coefficient Ncj, the color diversification feature coefficient Ssc, and the shape complexity coefficient Xtx, and optimize the feature matching algorithm;

[0052] At the same time, in the image data processing link, convert the image to be detected from the bitmap form to the non-bitmap form, extract the texture features and frequency component information of the image through the frequency domain processing method, and enhance the recognition accuracy of the object to be detected under complex occlusion or noise conditions; then combine the non-bitmap data with the deep learning model to regenerate the gray-scale features and the visibility coefficient Zjy of the fused image, and evaluate the feature matching degree Tzpd of the object in the warehouse environment through a dynamically adjusted feature matching algorithm; if the re-evaluation result meets the preset standard, the system generates an identification report and completes the sorting operation; otherwise, mark the current image as unrecognized and further trigger the system-level optimization mechanism.

[0053] The present invention provides an industrial robot image processing method based on image fusion, which has the following beneficial effects:

[0054] (1) The image processing method for industrial robots based on image fusion successfully solves the problem of object occlusion in automated warehousing by adopting image fusion technology, denoising algorithms, and multi-dimensional feature analysis. In the warehousing environment, the stacking and occlusion of goods are common. Traditional image processing technologies are difficult to accurately identify occluded objects. Especially in the case of multi-level occlusion, it is easy to cause sorting and recognition errors. Through data fusion technology and deep learning hierarchical fusion algorithms, the present invention effectively fuses images from different perspectives to generate unoccluded and clear object images. And through the calculation and evaluation of the visibility coefficient Zjy of the occlusion area, the system can timely issue an "occlusion alarm" to avoid recognition errors caused by occlusion, ensuring the accuracy and integrity of image processing;

[0055] (2) For the problem of image noise interference in the warehousing environment in the image processing method for industrial robots based on image fusion, the present invention introduces an adaptive multi-layer denoising technology and removes the noise components in the image through a denoising decomposition algorithm while retaining the edge and detail information of the image, avoiding the loss of important image details by traditional denoising algorithms when removing noise. By calculating the image denoising coefficient Ncj in real time and comparing it with the preset denoising threshold Q, the present invention effectively evaluates the denoising effect, screens out effective images with less noise interference and high quality, thereby improving the accuracy of subsequent recognition and processing and avoiding problems such as barcode recognition failure caused by image noise;

[0056] (3) For the challenge of diverse object shapes and colors in the image processing method for industrial robots based on image fusion, the present invention constructs a color diversity feature coefficient Ssc and a shape complexity coefficient Xtx, and adopts an analysis method based on color histograms and a geometric morphology analysis method to comprehensively improve the recognition ability of complex objects. By comparing with the preset color feature threshold E and shape complexity threshold R, the present invention can judge the degree of diversity of objects in terms of color and shape. If it meets the standard, the object is considered to meet the requirements of the diversity dimension. This method can identify objects with various shape and color combinations, overcomes the problem of poor adaptability of traditional image processing algorithms to diverse object shapes and colors, reduces the need for labeled data and the processing difficulty, and significantly improves the recognition efficiency and accuracy of the automated warehousing system. Description of the Drawings

[0057] Figure 1 It is a schematic flow chart of the steps of the image processing method for industrial robots based on image fusion of the present invention. Detailed Embodiments

[0058] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention. Embodiment

[0059] Please refer to Figure 1 , an industrial robot image processing method based on image fusion, comprising the following steps:

[0060] Step 1: Place the object to be detected in the automated warehousing detection area, and perform real-time image acquisition of the object to be detected from different angles and heights to obtain the image to be processed by the industrial robot.

[0061] Step 2: Preprocess the image to be processed by the industrial robot. Use the adaptive multi-layer denoising technology to process the image to be processed by the industrial robot. Remove the noise components through the denoising decomposition algorithm, and at the same time retain the edge and detail information of the image to be processed by the industrial robot. Calculate the image denoising coefficient Ncj and evaluate it, and screen the image to be processed by the industrial robot.

[0062] Step 3: Perform fusion processing on the denoised and screened image to be processed by the industrial robot. Use the hierarchical fusion algorithm based on deep learning to convert the images from multiple perspectives into the industrial robot fusion image; then, generate gray-scale features based on the industrial robot fusion image and calculate the visibility coefficient Zjy of the occlusion area; evaluate the visibility coefficient Zjy through a preset threshold, and mark the current fusion image as "occlusion alarm" according to the evaluation content and send a prompt signal.

[0063] Step 4: Analyze the color and shape features of the industrial robot fusion image. Construct the color diversification feature coefficient Ssc and the shape complexity coefficient Xtx through the color histogram and shape geometric feature extraction methods, and respectively preset the color feature threshold E and the shape complexity threshold R, and compare them with the color diversification feature coefficient Ssc and the shape complexity coefficient Xtx to evaluate the compliance degree of the current object in the diversification dimension.

[0064] Step 5: Comprehensively analyze the visibility coefficient Zjy, the image denoising coefficient Ncj, the color diversification feature coefficient Ssc, and the shape complexity coefficient Xtx of the object in the industrial robot fusion image, calculate the feature matching degree Tzpd of the object to be detected in the warehousing environment and evaluate it. If the evaluation result meets the preset conditions, generate an identification report and perform a sorting operation; otherwise, the system is marked as unrecognized and a re-acquisition instruction is triggered.

[0065] Step 6: When the object recognition does not meet the standard, the system will automatically correct and re-recognize the object to be detected by real-time optimizing the object feature matching algorithm based on historical data and previously recognized object information; during the process, incremental learning and contrastive learning techniques are used to continuously update and adjust the object feature model.

[0066] In this embodiment, Step 1 solves the recognition difficulty caused by object occlusion in complex warehousing environments through multi-angle and multi-height image acquisition;

[0067] Step 2 adopts an adaptive multi-layer denoising technique, combines a denoising decomposition algorithm to effectively remove noise, retains the edge and detail information of the image, and at the same time evaluates by calculating the image denoising coefficient Ncj to ensure that the image quality meets the processing requirements, improving the accuracy of subsequent image recognition;

[0068] Step 3 fuses the industrial robot images to be processed from different perspectives through a hierarchical fusion algorithm based on deep learning, generates a high-quality industrial robot fused image, and calculates the visibility coefficient Zjy of the occlusion area, which can accurately identify the occlusion area, improve the recognition accuracy and timely issue an "occlusion alarm" prompt;

[0069] Step 4 analyzes the color and shape features of the industrial robot fused image, combines the color histogram and geometric morphology methods, calculates and evaluates the color diversification feature coefficient Ssc and the shape complexity coefficient Xtx, evaluates the color and shape diversity of the object, and improves the recognition ability of complex objects;

[0070] Step 5 comprehensively analyzes each feature coefficient, calculates the feature matching degree Tzpd, and judges whether it meets the recognition requirements according to preset conditions. If it meets, it generates a recognition report and triggers the sorting operation. If it does not meet, it triggers a re-acquisition instruction to ensure the high accuracy of the recognition result and the high efficiency of the system;

[0071] Through the comparative analysis of historical data in Step 6, the system can adjust and optimize the object feature matching algorithm in real time, thereby improving the recognition accuracy and robustness; the incremental learning technique enables the continuous update of the object feature model, ensuring that the system can maintain a high recognition accuracy under different environments and conditions; the contrastive learning technique enhances the adaptability of object features, enabling the system to automatically correct and adapt to new recognition requirements when facing new objects or changing environments, reducing the cases of misrecognition and missed recognition; in addition, through automatic optimization, the system can quickly recover after object recognition fails, reduce the need for manual intervention, and improve the overall sorting efficiency and intelligent level. Embodiment

[0072] Step 1 specifically includes:

[0073] First, place the object to be detected in the automated warehousing detection area, and adjust the lighting conditions, background interference factors, and sensor status in the automated warehousing detection area to the preset standards. Then, use a 3D lidar and a depth camera to obtain the initial spatial contour of the detection area, and determine the position, size, and attitude of the object to be detected in the detection area through a spatial point cloud analysis algorithm. And generate multi-angle and multi-height acquisition paths according to the information corresponding to the attitude. Finally, perform real-time image acquisition on the object to be detected to obtain the image to be processed by the industrial robot.

[0074] Step two specifically includes:

[0075] First, use a denoising decomposition algorithm to remove the noise components in the image to be processed by the industrial robot, and at the same time retain the edge and detail information of the image to be processed by the industrial robot through adaptive adjustment. Then, calculate the image denoising coefficient Ncj, and use image processing algorithms, including edge detection, noise estimation, and statistical analysis, to obtain the image gradient Grd, the image noise standard deviation Noi, and the image signal-to-noise ratio Sig in real time, and calculate and obtain the image denoising coefficient Ncj by combining the following formula:

[0076] Then, compare and evaluate the image denoising coefficient Ncj with a preset denoising threshold Q, and screen the image to be processed by the industrial robot after denoising. The specific evaluation content is as follows:

[0077] If the image denoising coefficient Ncj ≥ the denoising threshold Q, it means that the denoising effect of the current image to be processed by the industrial robot meets the preset standards. At this time, take the current image as a valid image and perform further analysis;

[0078] If the image denoising coefficient Ncj < the denoising threshold Q, it means that the denoising effect of the current image to be processed by the industrial robot does not meet the preset standards. At this time, reject the current image, perform further processing or re-acquire the image.

[0079] In this embodiment, through the multi-angle and multi-height perspective technology, the integrity and accuracy of the image data of the object to be detected are realized, and the problems of object occlusion and position uncertainty are solved; in the image denoising stage, while using the denoising decomposition algorithm and the adaptive adjustment method to remove noise, the edge and detail information of the image are retained. By calculating the image denoising coefficient Ncj and combining the image gradient Grd, the image noise standard deviation Noi, and the image signal-to-noise ratio Sig, the image quality is further ensured, and the misrecognition caused by noise influence is avoided; by evaluating the image denoising coefficient Ncj with a preset denoising threshold Q, unqualified images are promptly rejected, and the influence of low-quality data on the subsequent recognition and decision-making processes is avoided, improving the robustness and reliability of the system. Embodiment

[0080] Step three specifically includes:

[0081] Before converting the images from multiple perspectives into the fused image of the industrial robot, an image data processing method from bit image to non-bit image is adopted to convert the bit image into a non-bit image with richer features. Frequency component information and texture features of the image are extracted by using frequency domain processing techniques including Fourier transform and discrete wavelet transform. The hierarchical fusion algorithm extracts and synthesizes features from the industrial robot images to be processed from different perspectives, and synthesizes the industrial robot images to be processed from multiple perspectives into a fused image of the industrial robot. Based on the fused image of the industrial robot, gray-scale features and data related to the occluded area are extracted, including the image gray value Gra, the occlusion area Occ, and the image perspective difference value Vie. The visibility of the occluded area is analyzed, and the image gray value Gra, the occlusion area Occ, and the image perspective difference value Vie are extracted. The visibility coefficient Zjy of the occluded area is calculated by combining the following formula:

[0082] Step three specifically further includes:

[0083] The fused image of the industrial robot is marked and a prompt signal is sent by comparing and evaluating the preset occlusion degree threshold W with the visibility coefficient Zjy. The specific evaluation content is as follows:

[0084] If the visibility coefficient Zjy ≥ the occlusion degree threshold W, it means that the occluded area in the fused image of the industrial robot has a normal impact on visibility, and it is marked as the "qualified" state at this time;

[0085] If the visibility coefficient Zjy < the occlusion degree threshold W, it means that the occluded area in the fused image of the industrial robot has an abnormal impact on visibility, and it is marked as "occlusion alarm" at this time, and a prompt signal is sent to the management personnel.

[0086] Step four specifically includes:

[0087] First, the color space of the fused image of the industrial robot is converted, and the original fused image of the industrial robot is converted into the RGB color space; then, data related to the color distribution frequency in the fused image is extracted through the color histogram, including the red channel mean MeaA, the green channel mean MeaB, the blue channel mean MeaC, the red channel variance VarA, the green channel variance VarB, the blue channel variance VarC, and the color saturation Sat. Then, the color diversification feature coefficient Ssc is calculated by using the following formula: Then, the contour shape of the object is extracted through edge detection and image segmentation techniques; a geometric morphology analysis method is used to describe the complexity of the shape, and data related to the object shape is collected and extracted in real time, and the shape complexity coefficient Xtx is calculated. The specific calculation formula is as follows: Wherein, Per represents the contour length in the object shape-related data, Are represents the contour area in the object shape-related data, Com represents the contour compactness in the object shape-related data, and Irr represents the shape irregularity in the object shape-related data.

[0088] Step four specifically further includes:

[0089] The specific evaluation contents of the color diversification feature coefficient Ssc and the shape complexity coefficient Xtx are as follows:

[0090] Compare the color diversification feature coefficient Ssc with the preset color feature threshold E:

[0091] If the color diversification feature coefficient Ssc ≥ the color feature threshold E, it means that the object color diversity in the current industrial robot fusion image meets the requirement, the object color features are rich and diverse, and meet the diversification dimension requirements;

[0092] If the color diversification feature coefficient Ssc < the color feature threshold E, it means that the object color diversity in the current industrial robot fusion image does not meet the requirement, the color features are single, and further adjustment is required at this time;

[0093] Compare the shape complexity coefficient Xtx with the preset shape complexity threshold R:

[0094] If the shape complexity coefficient Xtx ≥ the shape complexity threshold R, it means that the shape complexity of the object in the current industrial robot fusion image meets the requirement, the object shape is complex, and meets the diversification dimension requirements;

[0095] If the shape complexity coefficient Xtx < the shape complexity threshold R, it means that the shape complexity of the object in the current industrial robot fusion image does not meet the requirement, the object shape is simple, and further adjustment is required at this time.

[0096] In this embodiment, multi-view image data is synthesized into an industrial robot fusion image through a hierarchical fusion algorithm, effectively improving the comprehensive information volume and accuracy of the image; by extracting the image gray value Gra, the occlusion area Occ, and the image view difference value Vie, and combining the visibility coefficient Zjy of the occlusion area, the visibility of the occlusion area in the image is analyzed and evaluated to ensure the visual effect of the fusion image under occlusion; by setting the occlusion degree threshold W and comparing it with the visibility coefficient Zjy, occlusion alarms are identified in a timely manner and prompt signals are sent, enhancing the intelligent recognition ability and real-time response of the system; in the color and shape feature analysis stage, the mean values of the color channels of red, green, and blue are extracted using a color histogram, as well as the shape complexity coefficient Xtx, to comprehensively evaluate the color diversity and shape complexity of the object, and by comparing with the preset color feature threshold E and shape complexity threshold R, it is ensured that the object meets the diversification requirements; finally, based on the comprehensive evaluation results of color and shape, the compliance degree of the object in the diversification dimension is judged to ensure the accuracy and adaptability of object recognition and classification, thereby improving the intelligent level and flexibility of the system. Embodiment

[0097] Step five specifically includes:

[0098] Extract the visibility coefficient Zjy, the image denoising coefficient Ncj, the color diversification feature coefficient Ssc, and the shape complexity coefficient Xtx, and calculate the feature matching degree Tzpd through the following formula:

[0099] In the formula, is a preset matching degree calculation function, which comprehensively considers the influences of the four subordinate parameters of the visibility coefficient Zjy, the image denoising coefficient Ncj, the color diversification feature coefficient Ssc, and the shape complexity coefficient Xtx.

[0100] Step five also specifically includes:

[0101] Preset a feature matching threshold T, and compare and evaluate the feature matching degree Tzpd with the feature matching threshold T to determine whether the recognition requirements are met. The specific evaluation content is as follows:

[0102] If the feature matching degree Tzpd ≥ the feature matching threshold T, it means that the object recognition meets the standard, and an identification report is generated and the sorting operation is triggered;

[0103] If the feature matching degree Tzpd < the feature matching threshold T, it means that the object recognition does not meet the standard. At this time, the system is marked as "unrecognized", and a re-acquisition instruction is triggered and a new image acquisition is performed.

[0104] Step six specifically includes:

[0105] After triggering the re-identification instruction, first extract the image data of the currently unrecognized object, and conduct a comparative analysis in combination with historical identification data. Identify the key feature differences of similar objects through comparative learning technology, and generate a feature correction parameter set; secondly, use incremental learning technology to update the object feature model, adjust the feature weight distribution of the image denoising coefficient Ncj, the color diversification feature coefficient Ssc, and the shape complexity coefficient Xtx, and optimize the feature matching algorithm;

[0106] Meanwhile, in the image data processing link, convert the image to be detected from the bitmap form to the non-bitmap form, extract the texture features and frequency component information of the image through the frequency domain processing method, and enhance the recognition accuracy of the object to be detected under complex occlusion or noise conditions; then combine the non-bitmap data with the deep learning model, regenerate the gray-scale features and visibility coefficient Zjy for the fused image, and evaluate the feature matching degree Tzpd of the object in the warehouse environment through a dynamically adjusted feature matching algorithm; if the re-evaluation result meets the preset standard, the system generates an identification report and completes the sorting operation; otherwise, mark the current image as unrecognized and further trigger the system-level optimization mechanism.

[0107] In this embodiment, by extracting the visibility coefficient Zjy, the image denoising coefficient Ncj, the color diversification feature coefficient Ssc, and the shape complexity coefficient Xtx, calculate the feature matching degree Tzpd, and comprehensively evaluate the feature matching of the object from multiple dimensions; the acquisition and calculation of each subordinate parameter ensure the accurate identification of the object under different environments and processing conditions. The visibility coefficient Zjy reflects the visual effect of the occluded area in the image, the image denoising coefficient Ncj ensures the image quality and detail retention, the color diversification feature coefficient Ssc measures the color diversity of the object, and the shape complexity coefficient Xtx describes the complexity of the object shape; by comparing and evaluating the feature matching degree Tzpd with the preset feature matching threshold T, the system can accurately judge whether the object meets the recognition standard. If it meets the standard, generate an identification report and trigger the sorting operation, improving the recognition accuracy and automation degree; if it does not meet the standard, mark it as "unrecognized" and trigger the re-acquisition instruction, ensuring the reliability of object recognition and the flexibility of system response, thus optimizing the overall operation process and intelligent recognition efficiency;

[0108] During the object recognition process, due to factors such as illumination changes, occlusion, image noise, or morphological changes, recognition failures or errors may occur. In step six, the object feature matching algorithm is optimized in real time and adjusted according to historical data and information of previously recognized objects, enabling the system to quickly adapt to changing environments and recognition conditions, enhancing the system's robustness, and avoiding system failures or inefficiencies during the sorting process caused by preliminary recognition errors. Incremental learning and contrastive learning techniques are introduced, enabling the system to dynamically update the object feature model and optimize the recognition algorithm through continuous learning. This adaptive learning ability means that the system can gradually improve the accuracy of recognition over time and with the accumulation of experience, reducing manual intervention and correction.

[0109] Although the embodiments of the present invention have been shown and described, it will be understood by those of ordinary skill in the art that various changes, modifications, substitutions, and variations can be made to these embodiments without departing from the principles and spirit of the present invention, and the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. An industrial robot image processing method based on image fusion, characterized in that: The following steps are involved: Step 1: Place the object to be inspected in the automated warehouse inspection area, collect real-time images of the object to be inspected from different angles and heights, and obtain the image to be processed by the industrial robot; Step 2: pre-process the collected images of industrial robots to be processed, use adaptive multi-layer denoising technology to process the images of industrial robots to be processed, remove the noise components through the denoising decomposition algorithm, and retain the edge and detail information of the images of industrial robots to be processed, calculate and evaluate the image denoising coefficient Ncj, and screen the images of industrial robots to be processed; Step 3: The denoised and screened industrial robot images to be processed are fused, and the images from multiple perspectives are converted into industrial robot fused images using a layered fusion algorithm based on deep learning. Then, grayscale features are generated based on the industrial robot fused images and the visibility coefficient Zjy of the occluded area is calculated. The visibility coefficient Zjy is evaluated by a preset threshold, and the current fused image is marked as "occlusion alarm" based on the evaluation content, and a prompt signal is issued. Step 4: Perform color and shape feature analysis on the industrial robot fusion image, construct the color diversity feature coefficient Ssc and shape complexity coefficient Xtx through the color histogram and shape geometric feature extraction method, and preset the color feature threshold E and shape complexity threshold R respectively, and compare them with the color diversity feature coefficient Ssc and shape complexity coefficient Xtx to evaluate the compliance of the current object in the diversity dimension; Step 5: Comprehensively analyze the visibility coefficient Zjy, image denoising coefficient Ncj, color diversity feature coefficient Ssc and shape complexity coefficient Xtx of the objects in the fusion image of the industrial robot, calculate and evaluate the feature matching degree Tzpd of the object to be detected in the storage environment. If the evaluation result meets the preset conditions, generate an identification report and perform sorting operations; otherwise, the system marks it as unrecognized and triggers a re-recognition instruction; Step 6. When object recognition does not meet the standards, the system will automatically correct and re-identify the object to be detected by optimizing the object feature matching algorithm in real time based on historical data and previously identified object information; incremental learning technology is used in the process to continuously update and adjust the object feature model.

2. The industrial robot image processing method based on image fusion according to claim 1, characterized in that: Step 1 specifically includes: First, the object to be detected is placed in the automated warehouse detection area, and the lighting conditions, background interference factors and sensor status in the automated warehouse detection area are adjusted to the preset standards; then, the three-dimensional lidar and depth camera are used to obtain the initial spatial contour of the detection area, and the position, size and posture of the object to be detected in the detection area are determined through the spatial point cloud analysis algorithm; and multi-angle and multi-height acquisition paths are generated according to the information corresponding to the posture; finally, real-time image acquisition is performed on the object to be detected to obtain the image to be processed by the industrial robot.

3. The industrial robot image processing method based on image fusion according to claim 1, characterized in that: Step 2 specifically includes: Firstly, the noise components in the image to be processed by the industrial robot are removed by using the denoising decomposition algorithm, and the edge and detail information of the image to be processed by the industrial robot is retained by adaptive adjustment; then the image denoising coefficient Ncj is calculated, and the image processing algorithm, including edge detection, noise estimation and statistical analysis, is used to obtain the image gradient Grd, image noise standard deviation Noi and image signal-to-noise ratio Sig in real time, and the image denoising coefficient Ncj is calculated by combining the following formula: Then, the image denoising coefficient Ncj is compared and evaluated by the preset denoising threshold Q, and the denoised industrial robot images to be processed are screened. The specific evaluation contents are as follows: If the image denoising coefficient Ncj ≥ the denoising threshold Q, it means that the denoising effect of the current industrial robot image to be processed meets the preset standard. At this time, the current image is taken as a valid image and further analyzed; If the image denoising coefficient Ncj is less than the denoising threshold Q, it means that the denoising effect of the current image to be processed by the industrial robot does not meet the preset standard. At this time, the current image is discarded and further processed or the image is re-collected.

4. The industrial robot image processing method based on image fusion according to claim 1, characterized in that: Step three specifically includes: Before converting images from multiple perspectives into industrial robot fusion images, the image data processing method from bitmap to non-bitmap is used to convert the bitmap image into a non-bitmap form with richer features, and the frequency component information and texture features of the image are extracted by Fourier transform and discrete wavelet transform in the internal frequency domain processing technology; the hierarchical fusion algorithm extracts and synthesizes the industrial robot images to be processed from different perspectives, and synthesizes the industrial robot images to be processed from multiple perspectives into an industrial robot fusion image; based on the industrial robot fusion image, the grayscale features and the occlusion area related data are extracted, including the image grayscale value Gra, the occlusion area Occ and the image perspective difference value Vie, the visibility of the occlusion area is analyzed, and the image grayscale value Gra, the occlusion area Occ and the image perspective difference value Vie are extracted, and the visibility coefficient Zjy of the occlusion area is calculated by combining the following formula:

5. The industrial robot image processing method based on image fusion according to claim 1, characterized in that: Step three specifically includes: By comparing and evaluating the preset occlusion threshold W with the visibility coefficient Zjy, the industrial robot fusion image is marked and a prompt signal is issued. The specific evaluation content is as follows: If the visibility coefficient Zjy ≥ the occlusion degree threshold W, it means that the occlusion area in the industrial robot fusion image has a normal impact on visibility, and it is marked as "qualified"; If the visibility coefficient Zjy is less than the occlusion degree threshold W, it means that the occlusion area in the industrial robot fusion image has an abnormal impact on visibility. At this time, it is marked as "occlusion alarm" and a prompt signal is sent to the management personnel.

6. The industrial robot image processing method based on image fusion according to claim 1, characterized in that: Step 4 specifically includes: First, the color space conversion of the industrial robot fusion image is performed to convert the original industrial robot fusion image into RGB color space; then the color histogram is used to extract the relevant data of the color distribution frequency in the fusion image, including the red channel mean MeaA, the green channel mean MeaB, the blue channel mean MeaC, the red channel variance VarA, the green channel variance VarB, the blue channel variance VarC and the color saturation Sat, and then the color diversity characteristic coefficient Ssc is calculated using the following formula: Then, the contour shape of the object is extracted through edge detection and image segmentation technology; the geometric morphological analysis method is used to describe the complexity of the shape, and the shape-related data of the object is collected and extracted in real time to calculate the shape complexity coefficient Xtx. The specific calculation formula is as follows: Wherein, Per represents the contour length in the object shape related data, Are represents the contour area in the object shape related data, Com represents the contour compactness in the object shape related data, and Irr represents the shape irregularity in the object shape related data.

7. The industrial robot image processing method based on image fusion according to claim 1, characterized in that: Step 4 specifically includes: The specific evaluation contents of the color diversity characteristic coefficient Ssc and the shape complexity coefficient Xtx are as follows: Compare the color diversity characteristic coefficient Ssc with the preset color characteristic threshold E: If the color diversity feature coefficient Ssc ≥ color feature threshold E, it means that the color diversity of objects in the current industrial robot fusion image is in compliance, and the color features of the objects are rich and diverse, which meets the requirements of the diversity dimension; If the color diversity characteristic coefficient Ssc is less than the color characteristic threshold E, it means that the color diversity of the objects in the current industrial robot fusion image does not meet the requirements and the color characteristics are single, and further adjustments are required at this time; Compare the shape complexity coefficient Xtx with the preset shape complexity threshold R: If the shape complexity coefficient Xtx ≥ shape complexity threshold R, it means that the shape complexity of the object in the current industrial robot fusion image meets the requirements of diversified dimensions; If the shape complexity coefficient Xtx is less than the shape complexity threshold R, it means that the shape complexity of the object in the current industrial robot fusion image does not meet the requirements and the shape of the object is simple, and further adjustments are made at this time.

8. The industrial robot image processing method based on image fusion according to claim 7, characterized in that: Step 4 specifically includes: Finally, the evaluation results of color and shape are combined to determine the degree of conformity of objects in the industrial robot fusion image in various dimensions: If both the color diversity assessment and the shape complexity assessment meet their respective thresholds, the objects in the industrial robot fusion image meet the diversity requirements; If any evaluation of the color diversity feature coefficient Ssc and the shape complexity coefficient Xtx does not meet the requirements, it means that the objects in the industrial robot fusion image do not meet the diversity requirements and need further adjustment.

9. The industrial robot image processing method based on image fusion according to claim 1, characterized in that: Step 5 specifically includes: Extract the visibility coefficient Zjy, image denoising coefficient Ncj, color diversity feature coefficient Ssc and shape complexity coefficient Xtx, and calculate the feature matching degree Tzpd by the following formula: In the formula, It is a preset matching degree calculation function, which comprehensively considers the influence of four lower parameters: visibility coefficient Zjy, image denoising coefficient Ncj, color diversity characteristic coefficient Ssc and shape complexity coefficient Xtx; Step 5 specifically includes: The feature matching threshold T is preset, and the feature matching degree Tzpd is compared and evaluated with the feature matching threshold T to determine whether the recognition requirements are met. The specific evaluation contents are as follows: If the feature matching degree Tzpd ≥ the feature matching threshold T, it means that the object recognition meets the standard, and an identification report is generated and the sorting operation is triggered; If the feature matching degree Tzpd is less than the feature matching threshold T, it means that the object recognition does not meet the standard. At this time, the system marks it as "unrecognized" and triggers the re-recognition instruction.

10. The industrial robot image processing method based on image fusion according to claim 1, characterized in that: Step 6 specifically includes: After the re-identification command is triggered, the image data of the currently unrecognized object is first extracted and compared with the historical recognition data for analysis. The key feature differences of similar objects are identified through comparative learning technology, and a feature correction parameter set is generated. Secondly, the object feature model is updated using incremental learning technology, the feature weight distribution of the image denoising coefficient Ncj, the color diversity feature coefficient Ssc and the shape complexity coefficient Xtx is adjusted, and the feature matching algorithm is optimized. At the same time, in the image data processing link, the image to be detected is converted from a bitmap form to a non-bitmap form, and the texture features and frequency component information of the image are extracted through the frequency domain processing method to enhance the recognition accuracy of the object to be detected under complex occlusion or noise conditions; then the non-bitmap data is combined with the deep learning model to regenerate the grayscale features and visibility coefficient Zjy of the fused image, and the feature matching degree Tzpd of the object in the storage environment is evaluated through a dynamically adjusted feature matching algorithm; if the re-evaluation result meets the preset standard, the system generates a recognition report and completes the sorting operation; otherwise, the current image is marked as an unrecognized state and the system-level optimization mechanism is further triggered.

Images