Industrial robot image processing method based on image fusion

Abstract

The invention discloses an industrial robot image processing method based on image fusion, relates to the technical field of image fusion, effectively solves the problem of object occlusion in storage through multi-angle real-time image acquisition and a deep learning hierarchical fusion algorithm, and improves the image fusion accuracy by evaluating the visibility coefficient Zjy of an occlusion area. Identification errors caused by shielding are avoided; the image denoising coefficient Ncj is calculated in real time, so that the image quality is ensured, and the subsequent recognition accuracy is improved; for shape and color diversification, a color diversification characteristic coefficient Ssc and a shape complexity coefficient Xtx are calculated, and a color histogram and geometric morphology analysis are combined, so that the recognition capability of a complex object is comprehensively improved; through comparison with the preset threshold value, the diversification conformity of the object is judged, objects of various shapes and color combinations are effectively recognized, the data labeling requirement is reduced, and the recognition efficiency and accuracy of an automatic warehousing system are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of image fusion, and in particular to an industrial robot image processing method based on image fusion. Background Art

[0002] The background technology of industrial robot image processing methods originated from the rapid development of image processing and computer vision in the mid-20th century, and was initially applied to industrial inspection, quality control and other fields. Early image processing technology mainly relied on traditional algorithms such as edge detection and morphology. With the improvement of computer processing power, it was gradually applied to more complex scenes. Entering the 21st century, breakthroughs in deep learning technology have promoted the application of image recognition and processing in industrial robots, allowing robots to perform precise recognition, sorting, assembly and other tasks in production lines. In recent years, the development of technologies such as 3D vision, real-time processing, and cloud computing has further improved the performance of industrial robot image processing, making it more widely used in complex environments, such as automated warehousing, flexible manufacturing and other emerging fields. The integration and optimization of these technologies have promoted the intelligence and efficiency of industrial robots, and provided important support for the intelligent and automated development of modern industry.

[0003] However, although the image processing methods of industrial robots used in automated warehousing have made significant progress in intelligence and efficiency, there are still some technical shortcomings in image processing, including the following three aspects: 1. Object occlusion problem: In automated warehousing, goods stacking and occlusion are common. It is difficult for robots to accurately identify occluded objects, especially when image processing technology has not been fully optimized to resolve multi-level occlusions, which can easily lead to sorting and recognition errors.

[0004] 2. Image noise interference: In a warehouse environment, factors such as dust and vibration can easily introduce image noise. Traditional image denoising algorithms may lose important image details while removing noise, resulting in a decrease in the accuracy of subsequent image recognition. For example, barcode recognition is easily affected by noise, resulting in scanning failure.

[0005] 3. Challenges of diverse object shapes and colors: There are many types of items in automated warehousing, with diverse shapes and colors. Traditional image processing algorithms are limited in their effectiveness in identifying objects with complex shapes and color combinations, and require a large amount of labeled data for learning and optimization, which increases costs and processing difficulty. Summary of the invention

[0006] In view of the shortcomings of the prior art, the present invention provides an industrial robot image processing method based on image fusion, which solves the technical shortcomings of the background technology such as object occlusion problems, image noise interference, and the challenges of diverse object shapes and colors.

[0007] To achieve the above objectives, the present invention is implemented through the following technical solutions: an industrial robot image processing method based on image fusion, comprising the following steps: Step 1: Place the object to be inspected in the automated warehouse inspection area, collect real-time images of the object to be inspected from different angles and heights, and obtain the image to be processed by the industrial robot; Step 2: pre-process the collected images of industrial robots to be processed, use adaptive multi-layer denoising technology to process the images of industrial robots to be processed, remove the noise components through the denoising decomposition algorithm, and retain the edge and detail information of the images of industrial robots to be processed, calculate and evaluate the image denoising coefficient Ncj, and screen the images of industrial robots to be processed; Step 3: The denoised and screened industrial robot images to be processed are fused, and the images from multiple perspectives are converted into industrial robot fused images using a layered fusion algorithm based on deep learning. Then, grayscale features are generated based on the industrial robot fused images and the visibility coefficient Zjy of the occluded area is calculated. The visibility coefficient Zjy is evaluated by a preset threshold, and the current fused image is marked as "occlusion alarm" based on the evaluation content, and a prompt signal is issued. Step 4: Perform color and shape feature analysis on the industrial robot fusion image, construct the color diversity feature coefficient Ssc and shape complexity coefficient Xtx through the color histogram and shape geometric feature extraction method, and preset the color feature threshold E and shape complexity threshold R respectively, and compare them with the color diversity feature coefficient Ssc and shape complexity coefficient Xtx to evaluate the compliance of the current object in the diversity dimension; Step 5: Comprehensively analyze the visibility coefficient Zjy, image denoising coefficient Ncj, color diversity feature coefficient Ssc and shape complexity coefficient Xtx of the objects in the fusion image of the industrial robot, calculate and evaluate the feature matching degree Tzpd of the object to be detected in the storage environment. If the evaluation result meets the preset conditions, generate an identification report and perform sorting operations; otherwise, the system marks it as unrecognized and triggers a re-recognition instruction; Step 6. When object recognition does not meet the standards, the system will automatically correct and re-identify the object to be detected based on historical data and previously recognized object information by optimizing the object feature matching algorithm in real time; the incremental learning and comparative learning techniques are used in the process to continuously update and adjust the object feature model.

[0008] Optionally, step one specifically includes: First, the object to be detected is placed in the automated warehouse detection area, and the lighting conditions, background interference factors and sensor status in the automated warehouse detection area are adjusted to the preset standards; then, the three-dimensional lidar and depth camera are used to obtain the initial spatial contour of the detection area, and the position, size and posture of the object to be detected in the detection area are determined through the spatial point cloud analysis algorithm; and multi-angle and multi-height acquisition paths are generated according to the information corresponding to the posture; finally, real-time image acquisition is performed on the object to be detected to obtain the image to be processed by the industrial robot.

[0009] Optionally, step 2 specifically includes: Firstly, the noise components in the image to be processed by the industrial robot are removed by using the denoising decomposition algorithm, and the edge and detail information of the image to be processed by the industrial robot is retained by adaptive adjustment; then the image denoising coefficient Ncj is calculated, and the image processing algorithm, including edge detection, noise estimation and statistical analysis, is used to obtain the image gradient Grd, image noise standard deviation Noi and image signal-to-noise ratio Sig in real time, and the image denoising coefficient Ncj is calculated by combining the following formula:

[0010] Then, the image denoising coefficient Ncj is compared and evaluated by the preset denoising threshold Q, and the denoised industrial robot images to be processed are screened. The specific evaluation contents are as follows: If the image denoising coefficient Ncj ≥ the denoising threshold Q, it means that the denoising effect of the current industrial robot image to be processed meets the preset standard. At this time, the current image is taken as a valid image and further analyzed; If the image denoising coefficient Ncj is less than the denoising threshold Q, it means that the denoising effect of the current image to be processed by the industrial robot does not meet the preset standard. At this time, the current image is discarded and further processed or the image is re-collected.

[0011] Optionally, step three specifically includes: Before converting images from multiple perspectives into industrial robot fusion images, the image data processing method from bitmap to non-bitmap is used to convert the bitmap image into a non-bitmap form with richer features, and the frequency component information and texture features of the image are extracted by Fourier transform and discrete wavelet transform in the internal frequency domain processing technology; the hierarchical fusion algorithm extracts and synthesizes the industrial robot images to be processed from different perspectives, and synthesizes the industrial robot images to be processed from multiple perspectives into an industrial robot fusion image; based on the industrial robot fusion image, the grayscale features and the occlusion area related data are extracted, including the image grayscale value Gra, the occlusion area Occ and the image perspective difference value Vie, the visibility of the occlusion area is analyzed, and the image grayscale value Gra, the occlusion area Occ and the image perspective difference value Vie are extracted, and the visibility coefficient Zjy of the occlusion area is calculated by combining the following formula:

[0012] Optionally, step three further includes: By comparing and evaluating the preset occlusion threshold W with the visibility coefficient Zjy, the industrial robot fusion image is marked and a prompt signal is issued. The specific evaluation content is as follows: If the visibility coefficient Zjy ≥ the occlusion degree threshold W, it means that the occlusion area in the industrial robot fusion image has a normal impact on visibility, and it is marked as "qualified"; If the visibility coefficient Zjy is less than the occlusion degree threshold W, it means that the occlusion area in the industrial robot fusion image has an abnormal impact on visibility. In this case, it is marked as "occlusion alarm" and a prompt signal is sent to the management personnel.

[0013] Optionally, step 4 specifically includes: First, the color space conversion of the industrial robot fusion image is performed to convert the original industrial robot fusion image into RGB color space; then the color histogram is used to extract the relevant data of the color distribution frequency in the fusion image, including the red channel mean MeaA, the green channel mean MeaB, the blue channel mean MeaC, the red channel variance VarA, the green channel variance VarB, the blue channel variance VarC and the color saturation Sat, and then the color diversity characteristic coefficient Ssc is calculated using the following formula:

[0014] Then, the contour shape of the object is extracted through edge detection and image segmentation technology; the geometric morphological analysis method is used to describe the complexity of the shape, and the shape-related data of the object is collected and extracted in real time to calculate the shape complexity coefficient Xtx. The specific calculation formula is as follows: Wherein, Per represents the contour length in the object shape related data, Are represents the contour area in the object shape related data, Com represents the contour compactness in the object shape related data, and Irr represents the shape irregularity in the object shape related data.

[0015] Optionally, step 4 further includes: The specific evaluation contents of the color diversity characteristic coefficient Ssc and the shape complexity coefficient Xtx are as follows: Compare the color diversity characteristic coefficient Ssc with the preset color characteristic threshold E: If the color diversity feature coefficient Ssc ≥ color feature threshold E, it means that the color diversity of objects in the current industrial robot fusion image is in compliance, and the color features of the objects are rich and diverse, which meets the requirements of the diversity dimension; If the color diversity characteristic coefficient Ssc is less than the color characteristic threshold E, it means that the color diversity of the objects in the current industrial robot fusion image does not meet the requirements and the color characteristics are single, and further adjustments are required at this time; Compare the shape complexity coefficient Xtx with the preset shape complexity threshold R: If the shape complexity coefficient Xtx ≥ shape complexity threshold R, it means that the shape complexity of the object in the current industrial robot fusion image meets the requirements of diversified dimensions; If the shape complexity coefficient Xtx is less than the shape complexity threshold R, it means that the shape complexity of the object in the current industrial robot fusion image does not meet the requirements and the shape of the object is simple, and further adjustments are made at this time.

[0016] Optionally, step 4 further includes: Finally, the evaluation results of color and shape are combined to determine the degree of conformity of objects in the industrial robot fusion image in various dimensions: If both the color diversity assessment and the shape complexity assessment meet their respective thresholds, the objects in the industrial robot fusion image meet the diversity requirements; If any evaluation of the color diversity feature coefficient Ssc and the shape complexity coefficient Xtx does not meet the requirements, it means that the objects in the industrial robot fusion image do not meet the diversity requirements and need further adjustment.

[0017] Optionally, step five specifically includes: Extract the visibility coefficient Zjy, image denoising coefficient Ncj, color diversity feature coefficient Ssc and shape complexity coefficient Xtx, and calculate the feature matching degree Tzpd by the following formula: In the formula, It is a preset matching degree calculation function, which comprehensively considers the influence of four lower parameters: visibility coefficient Zjy, image denoising coefficient Ncj, color diversity feature coefficient Ssc and shape complexity coefficient Xtx.

[0018] Step 5 specifically includes: The feature matching threshold T is preset, and the feature matching degree Tzpd is compared and evaluated with the feature matching threshold T to determine whether the recognition requirements are met. The specific evaluation contents are as follows: If the feature matching degree Tzpd ≥ the feature matching threshold T, it means that the object recognition meets the standard, and an identification report is generated and the sorting operation is triggered; If the feature matching degree Tzpd is less than the feature matching threshold T, it means that the object recognition does not meet the standard. At this time, the system marks it as "unrecognized", triggering the re-capture instruction and re-image capture.

[0019] Optionally, step six specifically includes: After the re-identification command is triggered, the image data of the currently unrecognized object is first extracted and compared with the historical recognition data for analysis. The key feature differences of similar objects are identified through comparative learning technology, and a feature correction parameter set is generated. Secondly, the object feature model is updated using incremental learning technology, the feature weight distribution of the image denoising coefficient Ncj, the color diversity feature coefficient Ssc and the shape complexity coefficient Xtx is adjusted, and the feature matching algorithm is optimized. At the same time, in the image data processing link, the image to be detected is converted from a bitmap form to a non-bitmap form, and the texture features and frequency component information of the image are extracted through the frequency domain processing method to enhance the recognition accuracy of the object to be detected under complex occlusion or noise conditions; then the non-bitmap data is combined with the deep learning model to regenerate the grayscale features and visibility coefficient Zjy of the fused image, and the feature matching degree Tzpd of the object in the storage environment is evaluated through a dynamically adjusted feature matching algorithm; if the re-evaluation result meets the preset standard, the system generates a recognition report and completes the sorting operation; otherwise, the current image is marked as an unrecognized state and the system-level optimization mechanism is further triggered.

[0020] The present invention provides an industrial robot image processing method based on image fusion. It has the following beneficial effects: (1) This industrial robot image processing method based on image fusion successfully solves the problem of object occlusion in automated warehousing by adopting image fusion technology, denoising algorithm and multi-dimensional feature analysis. In the warehousing environment, cargo stacking and occlusion are common. Traditional image processing technology is difficult to accurately identify occluded objects, especially in the case of multi-level occlusion, which can easily lead to sorting and recognition errors. Through data fusion technology and deep learning hierarchical fusion algorithm, the present invention effectively fuses images from different perspectives to generate unobstructed and clear object images, and through the calculation and evaluation of the visibility coefficient Zjy of the occluded area, the system can issue an "occlusion alarm" in time to avoid recognition errors caused by occlusion, thereby ensuring the accuracy and integrity of image processing; (2) This is an industrial robot image processing method based on image fusion. To address the problem of image noise interference in a warehouse environment, the present invention introduces an adaptive multi-layer denoising technology and removes the noise components in the image through a denoising decomposition algorithm while retaining the edge and detail information of the image, thereby preventing the traditional denoising algorithm from losing important image details when removing noise. By calculating the image denoising coefficient Ncj in real time and comparing it with the preset denoising threshold Q, the present invention effectively evaluates the denoising effect and selects effective images with less noise interference and high quality, thereby improving the accuracy of subsequent recognition and processing and avoiding problems such as barcode recognition failure due to image noise; (3) This industrial robot image processing method based on image fusion, in response to the challenge of object shape and color diversity, the present invention constructs a color diversity feature coefficient Ssc and a shape complexity coefficient Xtx, and adopts an analysis method based on a color histogram and a geometric morphology analysis method to comprehensively improve the recognition ability of complex objects. By comparing with the preset color feature threshold E and shape complexity threshold R, the present invention can judge the degree of diversity of the object in color and shape. If it meets the standard, it is considered that the object meets the diversity dimension requirements. This method can recognize objects of various shapes and color combinations, overcomes the problem of poor adaptability of traditional image processing algorithms to object shape and color diversity, reduces the demand for annotation data and reduces the difficulty of processing, and significantly improves the recognition efficiency and accuracy of the automated warehousing system. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 The present invention is a schematic diagram of the steps of an industrial robot image processing method based on image fusion. DETAILED DESCRIPTION

[0022] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention. Example

[0023] See also Figure 1 , an industrial robot image processing method based on image fusion, comprising the following steps: Step 1: Place the object to be inspected in the automated warehouse inspection area, collect real-time images of the object to be inspected from different angles and heights, and obtain the image to be processed by the industrial robot; Step 2: pre-process the collected images of industrial robots to be processed, use adaptive multi-layer denoising technology to process the images of industrial robots to be processed, remove the noise components through the denoising decomposition algorithm, and retain the edge and detail information of the images of industrial robots to be processed, calculate and evaluate the image denoising coefficient Ncj, and screen the images of industrial robots to be processed; Step 3: The denoised and screened industrial robot images to be processed are fused, and the images from multiple perspectives are converted into industrial robot fused images using a layered fusion algorithm based on deep learning. Then, grayscale features are generated based on the industrial robot fused images and the visibility coefficient Zjy of the occluded area is calculated. The visibility coefficient Zjy is evaluated by a preset threshold, and the current fused image is marked as "occlusion alarm" based on the evaluation content, and a prompt signal is issued. Step 4: Perform color and shape feature analysis on the industrial robot fusion image, construct the color diversity feature coefficient Ssc and shape complexity coefficient Xtx through the color histogram and shape geometric feature extraction method, and preset the color feature threshold E and shape complexity threshold R respectively, and compare them with the color diversity feature coefficient Ssc and shape complexity coefficient Xtx to evaluate the compliance of the current object in the diversity dimension; Step 5: Comprehensively analyze the visibility coefficient Zjy, image denoising coefficient Ncj, color diversity feature coefficient Ssc and shape complexity coefficient Xtx of the objects in the fusion image of the industrial robot, calculate and evaluate the feature matching degree Tzpd of the object to be detected in the storage environment. If the evaluation result meets the preset conditions, generate an identification report and perform sorting operations; otherwise, the system marks it as unrecognized and triggers a re-collection instruction; Step 6. When object recognition does not meet the standards, the system will automatically correct and re-identify the object to be detected based on historical data and previously recognized object information by optimizing the object feature matching algorithm in real time; the incremental learning and comparative learning techniques are used in the process to continuously update and adjust the object feature model.

[0024] In this embodiment, step 1 solves the recognition difficulties caused by object occlusion in complex storage environments by collecting images at multiple angles and heights; Step 2 uses adaptive multi-layer denoising technology and denoising decomposition algorithm to effectively remove noise, retaining the edge and detail information of the image. At the same time, the image denoising coefficient Ncj is calculated for evaluation to ensure that the image quality meets the processing requirements and improve the accuracy of subsequent image recognition. Step 3: The industrial robot images to be processed from different perspectives are fused through a layered fusion algorithm based on deep learning to generate a high-quality industrial robot fusion image, and the visibility coefficient Zjy of the occluded area is calculated, which can accurately identify the occluded area, improve the recognition accuracy, and issue an "occlusion alarm" prompt in time; Step 4: Analyze the color and shape features of the industrial robot fusion image, combine the color histogram and geometric morphology method, calculate and evaluate the color diversity characteristic coefficient Ssc and shape complexity coefficient Xtx, evaluate the color and shape diversity of the object, and improve the recognition ability of complex objects; Step 5: Comprehensively analyze each feature coefficient, calculate the feature matching degree Tzpd, and determine whether the recognition requirements are met according to the preset conditions. If they are met, an identification report is generated and a sorting operation is triggered. If they are not met, a re-collection instruction is triggered to ensure the high accuracy of the recognition results and the high efficiency of the system. Step 6: Through comparative analysis of historical data, the system can adjust and optimize the object feature matching algorithm in real time, thereby improving recognition accuracy and robustness; incremental learning technology enables the object feature model to be continuously updated, ensuring that the system can maintain high recognition accuracy under different environments and conditions; contrastive learning technology enhances the adaptability of object features, allowing the system to automatically correct and adapt to new recognition requirements when faced with new objects or changing environments, reducing misidentification and missed identification; in addition, through automated optimization, the system can quickly recover after object recognition failure, reducing the need for manual intervention and improving overall sorting efficiency and intelligence. Example

[0025] Step 1 specifically includes: First, the object to be detected is placed in the automated warehouse detection area, and the lighting conditions, background interference factors and sensor status in the automated warehouse detection area are adjusted to the preset standards; then, the three-dimensional lidar and depth camera are used to obtain the initial spatial contour of the detection area, and the position, size and posture of the object to be detected in the detection area are determined through the spatial point cloud analysis algorithm; and multi-angle and multi-height acquisition paths are generated according to the information corresponding to the posture; finally, real-time image acquisition is performed on the object to be detected to obtain the image to be processed by the industrial robot.

[0026] Step 2 specifically includes: Firstly, the noise components in the image to be processed by the industrial robot are removed by using the denoising decomposition algorithm, and the edge and detail information of the image to be processed by the industrial robot is retained by adaptive adjustment; then the image denoising coefficient Ncj is calculated, and the image processing algorithm, including edge detection, noise estimation and statistical analysis, is used to obtain the image gradient Grd, image noise standard deviation Noi and image signal-to-noise ratio Sig in real time, and the image denoising coefficient Ncj is calculated by combining the following formula:

[0027] Then, the image denoising coefficient Ncj is compared and evaluated by the preset denoising threshold Q, and the denoised industrial robot images to be processed are screened. The specific evaluation contents are as follows: If the image denoising coefficient Ncj ≥ the denoising threshold Q, it means that the denoising effect of the current industrial robot image to be processed meets the preset standard. At this time, the current image is taken as a valid image and further analyzed; If the image denoising coefficient Ncj is less than the denoising threshold Q, it means that the denoising effect of the current image to be processed by the industrial robot does not meet the preset standard. At this time, the current image is discarded and further processed or the image is re-collected.

[0028] In this embodiment, the integrity and accuracy of the image data of the object to be detected are achieved through multi-angle and multi-height viewing angle technology, and the problems of object occlusion and position uncertainty are solved; in the image denoising stage, the denoising decomposition algorithm and the adaptive adjustment method are used to remove noise while retaining the edge and detail information of the image. By calculating the image denoising coefficient Ncj and combining the image gradient Grd, the image noise standard deviation Noi and the image signal-to-noise ratio Sig, the image quality is further ensured and the incorrect recognition caused by the influence of noise is avoided; the image denoising coefficient Ncj is evaluated by presetting the denoising threshold Q, and unqualified images are eliminated in time to avoid low-quality data affecting the subsequent recognition and decision-making process, thereby improving the robustness and reliability of the system. Example

[0029] Step three specifically includes: Before converting images from multiple perspectives into industrial robot fusion images, the image data processing method from bitmap to non-bitmap is used to convert the bitmap image into a non-bitmap form with richer features, and the frequency component information and texture features of the image are extracted by Fourier transform and discrete wavelet transform in the internal frequency domain processing technology; the hierarchical fusion algorithm extracts and synthesizes the industrial robot images to be processed from different perspectives, and synthesizes the industrial robot images to be processed from multiple perspectives into an industrial robot fusion image; based on the industrial robot fusion image, the grayscale features and the occlusion area related data are extracted, including the image grayscale value Gra, the occlusion area Occ and the image perspective difference value Vie, the visibility of the occlusion area is analyzed, and the image grayscale value Gra, the occlusion area Occ and the image perspective difference value Vie are extracted, and the visibility coefficient Zjy of the occlusion area is calculated by combining the following formula:

[0030] Step three specifically includes: By comparing and evaluating the preset occlusion threshold W with the visibility coefficient Zjy, the industrial robot fusion image is marked and a prompt signal is issued. The specific evaluation content is as follows: If the visibility coefficient Zjy ≥ the occlusion degree threshold W, it means that the occlusion area in the industrial robot fusion image has a normal impact on visibility, and it is marked as "qualified"; If the visibility coefficient Zjy is less than the occlusion degree threshold W, it means that the occlusion area in the industrial robot fusion image has an abnormal impact on visibility. In this case, it is marked as "occlusion alarm" and a prompt signal is sent to the management personnel.

[0031] Step 4 specifically includes: First, the color space conversion of the industrial robot fusion image is performed to convert the original industrial robot fusion image into RGB color space; then the color histogram is used to extract the relevant data of the color distribution frequency in the fusion image, including the red channel mean MeaA, the green channel mean MeaB, the blue channel mean MeaC, the red channel variance VarA, the green channel variance VarB, the blue channel variance VarC and the color saturation Sat, and then the color diversity characteristic coefficient Ssc is calculated using the following formula: Then, the contour shape of the object is extracted through edge detection and image segmentation technology; the geometric morphological analysis method is used to describe the complexity of the shape, and the shape-related data of the object is collected and extracted in real time to calculate the shape complexity coefficient Xtx. The specific calculation formula is as follows: Wherein, Per represents the contour length in the object shape related data, Are represents the contour area in the object shape related data, Com represents the contour compactness in the object shape related data, and Irr represents the shape irregularity in the object shape related data.

[0032] Step 4 specifically includes: The specific evaluation contents of the color diversity characteristic coefficient Ssc and the shape complexity coefficient Xtx are as follows: Compare the color diversity characteristic coefficient Ssc with the preset color characteristic threshold E: If the color diversity feature coefficient Ssc ≥ color feature threshold E, it means that the color diversity of objects in the current industrial robot fusion image is in compliance, and the color features of the objects are rich and diverse, which meets the requirements of the diversity dimension; If the color diversity characteristic coefficient Ssc is less than the color characteristic threshold E, it means that the color diversity of the objects in the current industrial robot fusion image does not meet the requirements and the color characteristics are single, and further adjustments are required at this time; Compare the shape complexity coefficient Xtx with the preset shape complexity threshold R: If the shape complexity coefficient Xtx ≥ shape complexity threshold R, it means that the shape complexity of the object in the current industrial robot fusion image meets the requirements of diversified dimensions; If the shape complexity coefficient Xtx is less than the shape complexity threshold R, it means that the shape complexity of the object in the current industrial robot fusion image does not meet the requirements and the shape of the object is simple, and further adjustments are made at this time.

[0033] In this embodiment, the multi-view image data is synthesized into an industrial robot fusion image through a hierarchical fusion algorithm, which effectively improves the comprehensive information content and accuracy of the image; by extracting the image gray value Gra, the occlusion area Occ and the image perspective difference value Vie, combined with the visibility coefficient Zjy of the occlusion area, the visibility of the occlusion area in the image is analyzed and evaluated, ensuring the visual effect of the fused image under occlusion; by setting the occlusion degree threshold W and comparing it with the visibility coefficient Zjy, the occlusion alarm is timely identified and a prompt signal is issued, thereby enhancing the intelligent recognition ability and real-time response of the system; in the color and shape feature analysis stage, the color channel means of the three colors of red, green and blue, as well as the shape complexity coefficient Xtx are extracted using the color histogram, and the color diversity and shape complexity of the object are comprehensively evaluated, and the preset color feature threshold E and shape complexity threshold R are compared to ensure that the object meets the diversity requirements; finally, the evaluation results of color and shape are combined to judge the degree of compliance of the object in the diversity dimension, to ensure the accuracy and adaptability of object recognition and classification, thereby improving the intelligence level and flexibility of the system. Example

[0034] Step 5 specifically includes: Extract the visibility coefficient Zjy, image denoising coefficient Ncj, color diversity feature coefficient Ssc and shape complexity coefficient Xtx, and calculate the feature matching degree Tzpd by the following formula: In the formula, It is a preset matching degree calculation function, which comprehensively considers the influence of four lower parameters: visibility coefficient Zjy, image denoising coefficient Ncj, color diversity feature coefficient Ssc and shape complexity coefficient Xtx.

[0035] Step 5 specifically includes: The feature matching threshold T is preset, and the feature matching degree Tzpd is compared and evaluated with the feature matching threshold T to determine whether the recognition requirements are met. The specific evaluation contents are as follows: If the feature matching degree Tzpd ≥ the feature matching threshold T, it means that the object recognition meets the standard, and an identification report is generated and the sorting operation is triggered; If the feature matching degree Tzpd is less than the feature matching threshold T, it means that the object recognition does not meet the standard. At this time, the system marks it as "unrecognized", triggering the re-capture instruction and re-image capture.

[0036] Step 6 specifically includes: After the re-identification command is triggered, the image data of the currently unrecognized object is first extracted and compared with the historical recognition data for analysis. The key feature differences of similar objects are identified through comparative learning technology, and a feature correction parameter set is generated. Secondly, the object feature model is updated using incremental learning technology, the feature weight distribution of the image denoising coefficient Ncj, the color diversity feature coefficient Ssc and the shape complexity coefficient Xtx is adjusted, and the feature matching algorithm is optimized. At the same time, in the image data processing link, the image to be detected is converted from a bitmap form to a non-bitmap form, and the texture features and frequency component information of the image are extracted through the frequency domain processing method to enhance the recognition accuracy of the object to be detected under complex occlusion or noise conditions; then the non-bitmap data is combined with the deep learning model to regenerate the grayscale features and visibility coefficient Zjy of the fused image, and the feature matching degree Tzpd of the object in the storage environment is evaluated through a dynamically adjusted feature matching algorithm; if the re-evaluation result meets the preset standard, the system generates a recognition report and completes the sorting operation; otherwise, the current image is marked as an unrecognized state and the system-level optimization mechanism is further triggered.

[0037] In this embodiment, by extracting the visibility coefficient Zjy, the image denoising coefficient Ncj, the color diversity characteristic coefficient Ssc and the shape complexity coefficient Xtx, the feature matching degree Tzpd is calculated, and the feature matching of the object is comprehensively evaluated from multiple dimensions; the collection and calculation of each lower parameter ensures the accurate recognition of the object under different environments and processing conditions, the visibility coefficient Zjy reflects the visual effect of the occluded area in the image, the image denoising coefficient Ncj ensures the image quality and detail retention, the color diversity characteristic coefficient Ssc measures the color diversity of the object, and the shape complexity coefficient Xtx describes the complexity of the object shape; by comparing and evaluating the feature matching degree Tzpd with the preset feature matching threshold T, the system can accurately determine whether the object meets the recognition standard. If it meets the standard, an identification report is generated and a sorting operation is triggered, thereby improving the recognition accuracy and automation; if it does not meet the standard, it is marked as "unrecognized" and a re-collection instruction is triggered, thereby ensuring the reliability of object recognition and the flexibility of system response, thereby optimizing the overall operation process and intelligent recognition efficiency; During the object recognition process, recognition failures or errors may occur due to factors such as lighting changes, occlusion, image noise or morphological changes. Step 6 optimizes the object feature matching algorithm in real time and makes adjustments based on historical data and information of previously recognized objects, so that the system can quickly adapt to changing environments and recognition conditions, enhance the robustness of the system, and avoid system failures or inefficiencies in the sorting process due to initial recognition errors; incremental learning and contrastive learning techniques are introduced, so that the system can dynamically update the object feature model and optimize the recognition algorithm through continuous learning. This adaptive learning capability means that the system can gradually improve the accuracy of recognition over time and with the accumulation of experience, reducing manual intervention and corrections.

[0038] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. An industrial robot image processing method based on image fusion, characterized in that: The following steps are involved: Step 1: Place the object to be inspected in the automated warehouse inspection area, collect real-time images of the object to be inspected from different angles and heights, and obtain the image to be processed by the industrial robot; Step 2: pre-process the collected images of industrial robots to be processed, use adaptive multi-layer denoising technology to process the images of industrial robots to be processed, remove the noise components through the denoising decomposition algorithm, and retain the edge and detail information of the images of industrial robots to be processed, calculate and evaluate the image denoising coefficient Ncj, and screen the images of industrial robots to be processed; Step 3: The denoised and screened industrial robot images to be processed are fused, and the images from multiple perspectives are converted into industrial robot fused images using a layered fusion algorithm based on deep learning. Then, grayscale features are generated based on the industrial robot fused images and the visibility coefficient Zjy of the occluded area is calculated. The visibility coefficient Zjy is evaluated by a preset threshold, and the current fused image is marked as "occlusion alarm" based on the evaluation content, and a prompt signal is issued. Step 4: Perform color and shape feature analysis on the industrial robot fusion image, construct the color diversity feature coefficient Ssc and shape complexity coefficient Xtx through the color histogram and shape geometric feature extraction method, and preset the color feature threshold E and shape complexity threshold R respectively, and compare them with the color diversity feature coefficient Ssc and shape complexity coefficient Xtx to evaluate the compliance of the current object in the diversity dimension; Step 5: Comprehensively analyze the visibility coefficient Zjy, image denoising coefficient Ncj, color diversity feature coefficient Ssc and shape complexity coefficient Xtx of the objects in the fusion image of the industrial robot, calculate and evaluate the feature matching degree Tzpd of the object to be detected in the storage environment. If the evaluation result meets the preset conditions, generate an identification report and perform sorting operations; otherwise, the system marks it as unrecognized and triggers a re-recognition instruction; Step 6. When object recognition does not meet the standards, the system will automatically correct and re-identify the object to be detected by optimizing the object feature matching algorithm in real time based on historical data and previously identified object information; incremental learning technology is used in the process to continuously update and adjust the object feature model.

2. The industrial robot image processing method based on image fusion according to claim 1, characterized in that: Step 1 specifically includes: First, the object to be detected is placed in the automated warehouse detection area, and the lighting conditions, background interference factors and sensor status in the automated warehouse detection area are adjusted to the preset standards; then, the three-dimensional lidar and depth camera are used to obtain the initial spatial contour of the detection area, and the position, size and posture of the object to be detected in the detection area are determined through the spatial point cloud analysis algorithm; and multi-angle and multi-height acquisition paths are generated according to the information corresponding to the posture; finally, real-time image acquisition is performed on the object to be detected to obtain the image to be processed by the industrial robot.

3. The industrial robot image processing method based on image fusion according to claim 1, characterized in that: Step 2 specifically includes: Firstly, the noise components in the image to be processed by the industrial robot are removed by using the denoising decomposition algorithm, and the edge and detail information of the image to be processed by the industrial robot is retained by adaptive adjustment; then the image denoising coefficient Ncj is calculated, and the image processing algorithm, including edge detection, noise estimation and statistical analysis, is used to obtain the image gradient Grd, image noise standard deviation Noi and image signal-to-noise ratio Sig in real time, and the image denoising coefficient Ncj is calculated by combining the following formula: Then, the image denoising coefficient Ncj is compared and evaluated by the preset denoising threshold Q, and the denoised industrial robot images to be processed are screened. The specific evaluation contents are as follows: If the image denoising coefficient Ncj ≥ the denoising threshold Q, it means that the denoising effect of the current industrial robot image to be processed meets the preset standard. At this time, the current image is taken as a valid image and further analyzed; If the image denoising coefficient Ncj is less than the denoising threshold Q, it means that the denoising effect of the current image to be processed by the industrial robot does not meet the preset standard. At this time, the current image is discarded and further processed or the image is re-collected.

4. The industrial robot image processing method based on image fusion according to claim 1, characterized in that: Step three specifically includes: Before converting images from multiple perspectives into industrial robot fusion images, the image data processing method from bitmap to non-bitmap is used to convert the bitmap image into a non-bitmap form with richer features, and the frequency component information and texture features of the image are extracted by Fourier transform and discrete wavelet transform in the internal frequency domain processing technology; the hierarchical fusion algorithm extracts and synthesizes the industrial robot images to be processed from different perspectives, and synthesizes the industrial robot images to be processed from multiple perspectives into an industrial robot fusion image; based on the industrial robot fusion image, the grayscale features and the occlusion area related data are extracted, including the image grayscale value Gra, the occlusion area Occ and the image perspective difference value Vie, the visibility of the occlusion area is analyzed, and the image grayscale value Gra, the occlusion area Occ and the image perspective difference value Vie are extracted, and the visibility coefficient Zjy of the occlusion area is calculated by combining the following formula:

5. The industrial robot image processing method based on image fusion according to claim 1, characterized in that: Step three specifically includes: By comparing and evaluating the preset occlusion threshold W with the visibility coefficient Zjy, the industrial robot fusion image is marked and a prompt signal is issued. The specific evaluation content is as follows: If the visibility coefficient Zjy ≥ the occlusion degree threshold W, it means that the occlusion area in the industrial robot fusion image has a normal impact on visibility, and it is marked as "qualified"; If the visibility coefficient Zjy is less than the occlusion degree threshold W, it means that the occlusion area in the industrial robot fusion image has an abnormal impact on visibility. At this time, it is marked as "occlusion alarm" and a prompt signal is sent to the management personnel.

6. The industrial robot image processing method based on image fusion according to claim 1, characterized in that: Step 4 specifically includes: First, the color space conversion of the industrial robot fusion image is performed to convert the original industrial robot fusion image into RGB color space; then the color histogram is used to extract the relevant data of the color distribution frequency in the fusion image, including the red channel mean MeaA, the green channel mean MeaB, the blue channel mean MeaC, the red channel variance VarA, the green channel variance VarB, the blue channel variance VarC and the color saturation Sat, and then the color diversity characteristic coefficient Ssc is calculated using the following formula: Then, the contour shape of the object is extracted through edge detection and image segmentation technology; the geometric morphological analysis method is used to describe the complexity of the shape, and the shape-related data of the object is collected and extracted in real time to calculate the shape complexity coefficient Xtx. The specific calculation formula is as follows: Wherein, Per represents the contour length in the object shape related data, Are represents the contour area in the object shape related data, Com represents the contour compactness in the object shape related data, and Irr represents the shape irregularity in the object shape related data.

7. The industrial robot image processing method based on image fusion according to claim 1, characterized in that: Step 4 specifically includes: The specific evaluation contents of the color diversity characteristic coefficient Ssc and the shape complexity coefficient Xtx are as follows: Compare the color diversity characteristic coefficient Ssc with the preset color characteristic threshold E: If the color diversity feature coefficient Ssc ≥ color feature threshold E, it means that the color diversity of objects in the current industrial robot fusion image is in compliance, and the color features of the objects are rich and diverse, which meets the requirements of the diversity dimension; If the color diversity characteristic coefficient Ssc is less than the color characteristic threshold E, it means that the color diversity of the objects in the current industrial robot fusion image does not meet the requirements and the color characteristics are single, and further adjustments are required at this time; Compare the shape complexity coefficient Xtx with the preset shape complexity threshold R: If the shape complexity coefficient Xtx ≥ shape complexity threshold R, it means that the shape complexity of the object in the current industrial robot fusion image meets the requirements of diversified dimensions; If the shape complexity coefficient Xtx is less than the shape complexity threshold R, it means that the shape complexity of the object in the current industrial robot fusion image does not meet the requirements and the shape of the object is simple, and further adjustments are made at this time.

8. The industrial robot image processing method based on image fusion according to claim 7, characterized in that: Step 4 specifically includes: Finally, the evaluation results of color and shape are combined to determine the degree of conformity of objects in the industrial robot fusion image in various dimensions: If both the color diversity assessment and the shape complexity assessment meet their respective thresholds, the objects in the industrial robot fusion image meet the diversity requirements; If any evaluation of the color diversity feature coefficient Ssc and the shape complexity coefficient Xtx does not meet the requirements, it means that the objects in the industrial robot fusion image do not meet the diversity requirements and need further adjustment.

9. The industrial robot image processing method based on image fusion according to claim 1, characterized in that: Step 5 specifically includes: Extract the visibility coefficient Zjy, image denoising coefficient Ncj, color diversity feature coefficient Ssc and shape complexity coefficient Xtx, and calculate the feature matching degree Tzpd by the following formula: In the formula, It is a preset matching degree calculation function, which comprehensively considers the influence of four lower parameters: visibility coefficient Zjy, image denoising coefficient Ncj, color diversity characteristic coefficient Ssc and shape complexity coefficient Xtx; Step 5 specifically includes: The feature matching threshold T is preset, and the feature matching degree Tzpd is compared and evaluated with the feature matching threshold T to determine whether the recognition requirements are met. The specific evaluation contents are as follows: If the feature matching degree Tzpd ≥ the feature matching threshold T, it means that the object recognition meets the standard, and an identification report is generated and the sorting operation is triggered; If the feature matching degree Tzpd is less than the feature matching threshold T, it means that the object recognition does not meet the standard. At this time, the system marks it as "unrecognized" and triggers the re-recognition instruction.

10. The industrial robot image processing method based on image fusion according to claim 1, characterized in that: Step 6 specifically includes: After the re-identification command is triggered, the image data of the currently unrecognized object is first extracted and compared with the historical recognition data for analysis. The key feature differences of similar objects are identified through comparative learning technology, and a feature correction parameter set is generated. Secondly, the object feature model is updated using incremental learning technology, the feature weight distribution of the image denoising coefficient Ncj, the color diversity feature coefficient Ssc and the shape complexity coefficient Xtx is adjusted, and the feature matching algorithm is optimized. At the same time, in the image data processing link, the image to be detected is converted from a bitmap form to a non-bitmap form, and the texture features and frequency component information of the image are extracted through the frequency domain processing method to enhance the recognition accuracy of the object to be detected under complex occlusion or noise conditions; then the non-bitmap data is combined with the deep learning model to regenerate the grayscale features and visibility coefficient Zjy of the fused image, and the feature matching degree Tzpd of the object in the storage environment is evaluated through a dynamically adjusted feature matching algorithm; if the re-evaluation result meets the preset standard, the system generates a recognition report and completes the sorting operation; otherwise, the current image is marked as an unrecognized state and the system-level optimization mechanism is further triggered.

Images