Image multi-channel enhanced identification method and system used in mesh shielding scene, and medium
By employing multi-channel image processing and multi-branch neural networks, the problems of artifact elimination and illumination changes in mesh occlusion scenarios were solved, achieving high-precision target recognition and data fusion, and improving recognition performance and stability.
Patent Information
- Application Number
- CN202511362643.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-09-23
- Publication Date
- 2026-02-10
AI Technical Summary
In scenarios where meshes are occluded, existing technologies struggle to effectively eliminate artifacts at mesh edges. During dynamic shooting, continuous frame registration is easily affected by changes in illumination. Furthermore, the lack of a quantitative compensation mechanism for spatial alignment during multi-sensor data fusion leads to insufficient target recognition accuracy.
A multi-channel image processing method is adopted, which combines the RGB main image with auxiliary channel images (such as infrared/depth data). Through feature point matching and registration, an auxiliary matrix is constructed for image enhancement, and a multi-branch neural network is used for environmental information recognition.
It effectively eliminates mesh artifacts, improves recognition accuracy and stability in dynamic scenes, reduces false recognition rate, and ensures sub-pixel background filling accuracy and information integrity of multi-source data fusion.
Smart Images
Figure CN121504737A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of image multi-channel enhancement technology, and in particular to an image multi-channel enhancement recognition method, system and medium for mesh occlusion scenarios. Background Technology
[0002] In fields such as industrial inspection, security monitoring, and medical imaging, target recognition in scenarios with mesh occlusion has long faced technical bottlenecks. When traditional monocular vision systems acquire images in scenarios such as metal screens and grid protective covers, the mesh structure generates periodic noise interference, leading to problems such as broken target textures and missing feature points. Existing technologies mostly employ image inpainting algorithms or frequency domain filtering, but these struggle to overcome registration errors in dynamic scenes and cannot effectively utilize the complementarity of multimodal data.
[0003] Current deep learning-based solutions have three limitations: First, the single RGB channel input makes it difficult to eliminate mesh edge artifacts; second, continuous frame registration during dynamic shooting is easily affected by changes in lighting; and third, there is a lack of quantitative compensation mechanisms for spatial alignment when fusing multi-sensor data. Especially in industrial quality inspection scenarios, the inspection of precision parts after metal mesh screening requires sub-pixel accuracy, and existing methods have not yet formed an effective solution for the collaborative optimization of mesh parameter modeling and environmental information reconstruction. Summary of the Invention
[0004] This application provides a method, system, and medium for image multi-channel enhancement and recognition in mesh occlusion scenarios, to solve the problems of existing solutions, such as the difficulty in eliminating mesh edge artifacts due to single RGB channel input, the susceptibility of continuous frame registration to changes in illumination during dynamic shooting, and the lack of a quantitative compensation mechanism for spatial alignment when fusing data from multiple sensors.
[0005] In a first aspect, this application provides an image multi-channel enhancement and recognition method for mesh occlusion scenarios, the method comprising: When a target region is detected, the system acquires the corresponding target main image; identifies the mesh occlusion area and mesh parameters in the target main image; captures continuous image frames in the target region while simultaneously acquiring auxiliary channel images; performs feature point matching and registration on several consecutive image frames to obtain background supplementation for the mesh occlusion area corresponding to the target main image; fills in the mesh occlusion area in the target main image using the background supplementation and mesh parameters; aligns the auxiliary channel image with the target main image to obtain the aligned auxiliary channel image; performs preset image enhancement processing on the target main image and auxiliary channel images; constructs an auxiliary matrix using the auxiliary channel image; and inputs the auxiliary matrix and the target main image into a trained multi-branch neural network to obtain the environmental information recognition result of the target main image in the mesh occlusion scene.
[0006] In one implementation of this application, when a target region is detected, a target main image corresponding to the target region is acquired; the mesh occlusion region and mesh parameters in the target main image are identified, specifically including: When the robot detects the target area, it acquires an initial image and checks whether there are any mesh occlusions in the initial image; if there are no mesh occlusions, the process ends. When there is mesh occlusion, the orientation and center line of the mesh occlusion are detected, and the robot is moved to the center line of the mesh occlusion at a preset distance in front, and the target main image corresponding to the target area is collected. Collect repetitive structures with mesh occlusion in the target main image, perform template matching on the repetitive structures to determine the corresponding mesh template, and obtain the mesh density corresponding to the mesh template; Collect the occlusion area of the mesh, the mesh size of the occlusion area, and the occlusion ratio through the target main image; store the mesh density, mesh size, and occlusion ratio as mesh parameters.
[0007] In one implementation of this application, feature point matching and registration are performed on several consecutive image frames to obtain background supplementation for the mesh occlusion area corresponding to the target main image, specifically including: Feature point matching and registration are performed on several consecutive image frames and the target main image respectively to obtain the environmental information corresponding to the feature points behind the mesh occlusion area of the target main image; Use environmental information as background supplement.
[0008] In one implementation of this application, background filling and mesh parameters are used to fill in the mesh occlusion area in the target main image, specifically including: An affine transformation matrix is established based on the mesh size in the mesh parameters, and the background supplementation of different frames is registered to the coordinate system of the target main image. The weighted average method is used to fuse the background supplementation of multiple frames to form a background supplementation layer. A two-dimensional periodic mesh model is constructed based on mesh parameters; the effective light-transmitting area ratio is calculated by the occlusion ratio, and a mask matrix consistent with the physical occlusion characteristics is generated on the two-dimensional periodic mesh model using a non-uniform sampling algorithm; Through the formula: The final output = background supplement layer × mask matrix + target main image × (1 - mask matrix) to obtain the filled mesh occlusion area.
[0009] In one implementation of this application, a preset image enhancement process is performed on the target main image and the auxiliary channel image, specifically including: Perform contrast-limited adaptive histogram equalization on the target main image; The target main image is input into an edge-preserving filter to process the structure and texture of the target main image; The auxiliary channel image is normalized and then converted into a grayscale or pseudo-color image.
[0010] In one implementation of this application, before inputting the auxiliary matrix and the target main image into a trained multi-branch neural network to obtain the environmental information recognition result of the target main image under the mesh occlusion scene, the method further includes: The historical main image, auxiliary matrix, and environmental information recognition results under mesh occlusion scenarios are obtained as sample data. Using sample data, a multi-branch neural network is trained to obtain a well-trained multi-branch neural network.
[0011] Secondly, this application provides an image multi-channel enhancement and recognition system for mesh occlusion scenarios, the system comprising: The recognition module is used to acquire the target main image corresponding to the target area when the target area is detected; and to identify the mesh occlusion area and mesh parameters in the target main image. The acquisition module is used to capture continuous image frames in the target area, while simultaneously acquiring images from auxiliary channels; The filling module is used to perform feature point matching and registration on several consecutive image frames to obtain background supplementation for the mesh occlusion area corresponding to the target main image; using the background supplementation and mesh parameters, the mesh occlusion area in the target main image is filled in. The construction module is used to align the auxiliary channel image with the target main image to obtain the aligned auxiliary channel image; perform preset image enhancement processing on the target main image and the auxiliary channel image; and construct an auxiliary matrix using the auxiliary channel image. The results module is used to input the auxiliary matrix and the target main image into the trained multi-branch neural network to obtain the environmental information recognition results of the target main image under the mesh occlusion scene.
[0012] In one implementation of this application, the identification module includes an identification unit. This is used to acquire an initial image when the robot detects a target area, and then detect whether there is any mesh occlusion in the initial image; if there is no mesh occlusion, the process ends. When there is mesh occlusion, the orientation and center line of the mesh occlusion are detected, and the robot is moved to the center line of the mesh occlusion at a preset distance in front, and the target main image corresponding to the target area is collected. Collect repetitive structures with mesh occlusion in the target main image, perform template matching on the repetitive structures to determine the corresponding mesh template, and obtain the mesh density corresponding to the mesh template; Collect the occlusion area of the mesh, the mesh size of the occlusion area, and the occlusion ratio through the target main image; store the mesh density, mesh size, and occlusion ratio as mesh parameters.
[0013] In one implementation of this application, the filling module includes filling units. This is used to establish an affine transformation matrix based on the grid size in the mesh parameters, and to register the background supplementation of different frames to the coordinate system of the target main image; a weighted average method is used to fuse the background supplementation of multiple frames to form a background supplementation layer. A two-dimensional periodic mesh model is constructed based on mesh parameters; the effective light-transmitting area ratio is calculated by the occlusion ratio, and a mask matrix consistent with the physical occlusion characteristics is generated on the two-dimensional periodic mesh model using a non-uniform sampling algorithm; Through the formula: The final output = background supplement layer × mask matrix + target main image × (1 - mask matrix) to obtain the filled mesh occlusion area.
[0014] Thirdly, this application provides a non-volatile computer storage medium, characterized in that it stores computer instructions thereon, which, when executed, implement an image multi-channel enhancement and recognition method for mesh occlusion scenarios as described in any one of claims 1-6.
[0015] As can be seen from the above technical solutions, this application has the following advantages: 1. Mesh artifact elimination mechanism based on multimodal data collaborative processing: A dual-channel input source is constructed by simultaneously acquiring RGB main images and auxiliary channel images (such as infrared / depth data). This design directly solves the mesh edge artifact problem caused by traditional single RGB channels: the non-visible spectral information provided by the auxiliary channel can effectively distinguish between the real scene and mesh occlusion. Combined with the multi-branch feature extraction capability of neural networks, spectral separation between physical occlusion and the real scene is achieved at the input layer. Compared with traditional single-channel solutions, this technology reduces the mesh false recognition rate.
[0016] 2. Spatiotemporal consistency compensation system in dynamic shooting: In the continuous frame registration process, an auxiliary channel is introduced as an illumination-invariant reference, improving stability through two dimensions: 1) Utilizing the radiometric consistency characteristics of the auxiliary channel (such as the temperature gradient stability of thermal imaging) as a registration benchmark overcomes the matching failure problem of the RGB channel during sudden illumination changes; 2) The establishment of a dynamic mesh parameter library enables the system to automatically adapt to occlusions of different apertures / densities, achieving sub-pixel-level background filling accuracy. This technology improves the effective frame rate in dynamic scenes.
[0017] 3. Spatial alignment scheme for quantitative cross-modal data: Precise alignment across three dimensions is achieved by establishing an auxiliary matrix: 1) Geometric alignment employs a perspective transformation model based on mesh parameters to eliminate multi-sensor parallax; 2) Radiation alignment maintains enhanced consistency through histogram migration between channels; 3) Temporal alignment relies on auxiliary channel feature tracking of consecutive frames to ensure the conservation of spatial relationships in dynamic scenes. This quantitative compensation mechanism reduces the information loss rate during multi-source data fusion compared to traditional methods. Attached Figure Description
[0018] To more clearly illustrate the technical solution of the present invention, the accompanying drawings used in the description will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0019] Figure 1 This is a flowchart of an image multi-channel enhancement and recognition method for mesh occlusion scenarios provided in this application embodiment.
[0020] Figure 2 This is a schematic diagram of the internal structure of an image multi-channel enhancement and recognition system for mesh occlusion scenarios provided in this application embodiment. Detailed Implementation
[0021] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0022] Those skilled in the art should understand that the embodiments described below are merely preferred embodiments of this disclosure and do not imply that this disclosure can only be implemented through these preferred embodiments. These preferred embodiments are merely used to explain the technical principles of this disclosure and are not intended to limit the scope of protection of this disclosure. Based on the preferred embodiments provided by this disclosure, all other embodiments obtained by those skilled in the art without creative effort should still fall within the scope of protection of this disclosure.
[0023] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.
[0024] The technical solutions proposed in the embodiments of this application will be described in detail below with reference to the accompanying drawings.
[0025] The embodiment provides an image multi-channel enhancement and recognition method for mesh occlusion scenarios, such as Figure 1 As shown in the embodiments of this application, the method mainly includes the following steps: Step 110: When a target area is detected, acquire the target main image corresponding to the target area; identify the mesh occlusion area and mesh parameters in the target main image.
[0026] In some embodiments, when a target region is detected, a target main image corresponding to the target region is acquired; the mesh occlusion region and mesh parameters in the target main image are identified, specifically including: When the robot detects the target area, it acquires an initial image and checks whether there are any mesh occlusions in the initial image; if there are no mesh occlusions, the process ends. When there is mesh occlusion, the orientation and center line of the mesh occlusion are detected, and the robot is moved to the center line of the mesh occlusion at a preset distance in front, and the target main image corresponding to the target area is collected. Collect repetitive structures with mesh occlusion in the target main image, perform template matching on the repetitive structures to determine the corresponding mesh template, and obtain the mesh density corresponding to the mesh template; Collect the occlusion area of the mesh, the mesh size of the occlusion area, and the occlusion ratio through the target main image; store the mesh density, mesh size, and occlusion ratio as mesh parameters.
[0027] Based on the above description, this step achieves automated acquisition of mesh parameters through a robot vision system. Its core technological advantages are as follows: The initial image pre-screening mechanism effectively distinguishes between mesh occlusion and non-mesh occlusion, avoiding invalid operations on areas without mesh. When mesh features are detected, the system uses orientation detection and centerline positioning algorithms to control the robot to move to a standard acquisition position, ensuring the standardization of the main image acquisition. Template matching technology is used to identify repetitive mesh structures, automatically matching preset mesh template types and calculating the corresponding mesh density parameters. The system can simultaneously measure the actual physical size of the mesh and the proportion of occluded areas, forming a complete parameter set including density, size, and occlusion ratio. This structured data can be directly used for mesh quality assessment and production process control. In practical applications, this technical solution solves the efficiency bottleneck and subjective error problems of manual inspection, providing a feasible technical path for automated inspection of mesh products. It should be noted that the system's ability to identify non-standard mesh structures depends on the completeness of the preset template library; algorithm optimization is required based on the specific product type during actual deployment.
[0028] Step 120: Capture consecutive image frames in the target area, and simultaneously acquire images from the auxiliary channel.
[0029] It should be noted that continuous image frames can be: capturing continuous image frames in the target area using a visible light camera; and controlling exposure compensation and white balance parameters during the recording process to ensure image consistency.
[0030] The acquisition of auxiliary channel images can include: acquiring infrared images, edge maps, depth maps, and other auxiliary channels (such as thermal imaging and pseudo-color images); all images are processed synchronously using timestamps.
[0031] Step 130: Perform feature point matching and registration on several consecutive image frames to obtain background supplementation for the mesh occlusion area corresponding to the target main image; use background supplementation and mesh parameters to fill in the mesh occlusion area in the target main image.
[0032] Feature point matching and registration are performed on several consecutive image frames to obtain background supplementation for the mesh occlusion area corresponding to the target main image, specifically including: Feature point matching and registration are performed on several consecutive image frames and the target main image respectively to obtain the environmental information corresponding to the feature points behind the mesh occlusion area of the target main image; Use environmental information as background supplement.
[0033] By using background filling and mesh parameters, fill in the mesh-occluded areas in the target main image, specifically including: An affine transformation matrix is established based on the mesh size in the mesh parameters, and the background supplementation of different frames is registered to the coordinate system of the target main image. The weighted average method is used to fuse the background supplementation of multiple frames to form a background supplementation layer. A two-dimensional periodic mesh model is constructed based on mesh parameters; the effective light-transmitting area ratio is calculated by the occlusion ratio, and a mask matrix consistent with the physical occlusion characteristics is generated on the two-dimensional periodic mesh model using a non-uniform sampling algorithm; Through the formula: The final output = background supplement layer × mask matrix + target main image × (1 - mask matrix) to obtain the filled mesh occlusion area.
[0034] Based on the above description, this step achieves accurate filling of mesh occlusion areas through multi-frame image processing. Its technical value is mainly reflected in three aspects: In the feature matching stage, the system can extract environmental information after the mesh occlusion area by registering feature points between consecutive image frames and the target main image. This process uses SIFT feature descriptors to achieve cross-frame matching, and the matching success rate reaches the benchmark level required by industrial application standards. In the background reconstruction stage, the coordinate system method based on the affine transformation matrix ensures the spatial consistency of background information in the target main image coordinate system across multiple frames. Combined with the weighted average algorithm, it can effectively suppress noise interference in a single frame image. In the final synthesis stage, the mask matrix generated by the two-dimensional periodic grid model can accurately reflect the physical occlusion characteristics of the mesh. Its mathematical expression is: Final output = Background filling layer × Mask matrix + Target main image × (1 - Mask matrix). This formula has been verified in the field of image processing to achieve pixel-level natural transitions. Practical application data shows that, under standard testing conditions, the structural similarity index (SSIM) of this method for filling metal meshes, as measured by professional image quality assessment tools, reaches the expected range, and edge artifacts are controlled to a visually indistinguishable level. It should be noted that the processing effect is affected by the original image resolution, the regularity of the mesh openings, and ambient lighting conditions. Additional compensation algorithms are required in extremely low-light or high-motion scenes.
[0035] Step 140: Align the auxiliary channel image with the target main image to obtain the aligned auxiliary channel image; perform preset image enhancement processing on the target main image and the auxiliary channel image; construct an auxiliary matrix using the auxiliary channel image.
[0036] In some embodiments, a preset image enhancement process is performed on the target main image and the auxiliary channel image, specifically including: Perform contrast-limited adaptive histogram equalization on the target main image; The target main image is input into an edge-preserving filter to process the structure and texture of the target main image; The auxiliary channel image is normalized and then converted into a grayscale or pseudo-color image.
[0037] Constructing an auxiliary matrix using auxiliary channel images: Constructing a multi-channel image input matrix: such as RGB channel image + depth channel image + edge channel image + infrared channel image.
[0038] Based on the above description, this step achieves multi-dimensional quality improvement through a systematic image processing workflow: 1) Alignment processing between the auxiliary channel and the main image effectively eliminates spatial displacement errors between multi-source images, ensuring geometric consistency in subsequent analysis; 2) When using Contrast-Limited Adaptive Histogram Equalization (CLAHE) to process the main image, it significantly improves the detail recognition of low-contrast areas while avoiding excessive noise enhancement; 3) The application of edge-preserving filters ensures that the main image fully preserves key structural features during noise reduction, providing more accurate structural texture information for subsequent analysis; 4) Normalization processing of the auxiliary channel eliminates grayscale deviations caused by differences in lighting conditions, while grayscale / pseudo-color conversion enhances the visual distinguishability of specific features; 5) The construction of the auxiliary matrix provides quantifiable data support for multimodal image analysis, facilitating subsequent feature fusion and quantitative analysis. This entire solution can improve the accuracy of feature extraction and the reliability of data analysis in scenarios requiring multi-channel image collaborative analysis, such as medical imaging and remote sensing monitoring.
[0039] Step 150: Input the auxiliary matrix and the target main image into the trained multi-branch neural network to obtain the environmental information recognition result of the target main image under the mesh occlusion scene.
[0040] Before inputting the auxiliary matrix and the target main image into the trained multi-branch neural network to obtain the environmental information recognition result of the target main image in the mesh occlusion scene, the method also includes: The historical main image, auxiliary matrix, and environmental information recognition results under mesh occlusion scenarios are obtained as sample data. Using sample data, a multi-branch neural network is trained to obtain a well-trained multi-branch neural network.
[0041] Based on the above description, this step, by inputting the auxiliary matrix and the target main image into a fully trained multi-branch neural network, achieves the following verifiable technical effects: First, training with sample data composed of historical main images, auxiliary matrices, and corresponding environmental information recognition results enables the network to extract effective features from mesh occlusion interference, and its output environmental information recognition results can accurately distinguish between mesh structures and real environmental information. Second, the multi-branch structure design allows the network to process image spatial features and geometric constraint information provided by the auxiliary matrix in parallel, avoiding misidentification at mesh edges caused by traditional single-branch networks through feature layer fusion. Experiments show that, while maintaining baseline computational efficiency, this method achieves consistent recognition results with manually labeled ground truth data for test samples with mesh densities ranging from 20 to 60 meshes. Especially in industrial scenarios with varying lighting conditions, it can stably output environmental depth information unaffected by mesh projection interference. Furthermore, the sample data used during training covers mesh types with different materials and aperture ratios, ensuring the model has generalization recognition capabilities for unknown mesh samples. This characteristic has been verified in practical applications such as textile quality inspection and construction site fencing inspection.
[0042] In addition, this application Figure 2 This application provides an embodiment of an image multi-channel enhancement and recognition system for mesh occlusion scenarios. For example... Figure 2 As shown in the embodiments of this application, the system mainly includes: The recognition module 210 is used to acquire the target main image corresponding to the target area when the target area is detected; and to identify the mesh occlusion area and mesh parameters in the target main image.
[0043] The recognition module 210 includes a recognition unit. This is used to acquire an initial image when the robot detects a target area, and then detect whether there is any mesh occlusion in the initial image; if there is no mesh occlusion, the process ends. When there is mesh occlusion, the orientation and center line of the mesh occlusion are detected, and the robot is moved to the center line of the mesh occlusion at a preset distance in front, and the target main image corresponding to the target area is collected. Collect repetitive structures with mesh occlusion in the target main image, perform template matching on the repetitive structures to determine the corresponding mesh template, and obtain the mesh density corresponding to the mesh template; Collect the occlusion area of the mesh, the mesh size of the occlusion area, and the occlusion ratio through the target main image; store the mesh density, mesh size, and occlusion ratio as mesh parameters.
[0044] The acquisition module 220 is used to capture continuous image frames in the target area and simultaneously acquire images from auxiliary channels.
[0045] The filling module 230 is used to perform feature point matching and registration on several consecutive image frames to obtain background supplementation for the mesh occlusion area corresponding to the target main image; and to fill the mesh occlusion area in the target main image using background supplementation and mesh parameters.
[0046] The filling module 230 includes filling units. This is used to establish an affine transformation matrix based on the grid size in the mesh parameters, and to register the background supplementation of different frames to the coordinate system of the target main image; a weighted average method is used to fuse the background supplementation of multiple frames to form a background supplementation layer. A two-dimensional periodic mesh model is constructed based on mesh parameters; the effective light-transmitting area ratio is calculated by the occlusion ratio, and a mask matrix consistent with the physical occlusion characteristics is generated on the two-dimensional periodic mesh model using a non-uniform sampling algorithm; Through the formula: The final output = background supplement layer × mask matrix + target main image × (1 - mask matrix) to obtain the filled mesh occlusion area.
[0047] The construction module 240 is used to align the auxiliary channel image with the target main image to obtain the aligned auxiliary channel image; perform preset image enhancement processing on the target main image and the auxiliary channel image; and construct an auxiliary matrix through the auxiliary channel image.
[0048] The result module 250 is used to input the auxiliary matrix and the target main image into the trained multi-branch neural network to obtain the environmental information recognition result of the target main image under the mesh occlusion scene.
[0049] In addition, this application embodiment also provides a non-volatile computer storage medium storing executable instructions, which, when executed, implement the above-described image multi-channel enhancement and recognition method for mesh occlusion scenarios.
[0050] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims
1. A method for multi-channel image enhancement and recognition in mesh occlusion scenarios, characterized in that, The method includes: When a target region is detected, the target main image corresponding to the target region is acquired; the mesh occlusion area and mesh parameters in the target main image are identified. Capture consecutive image frames in the target area while simultaneously acquiring images from auxiliary channels; Feature point matching and registration are performed on several consecutive image frames to obtain background supplementation for the mesh occlusion area corresponding to the target main image; Using background filling and mesh parameters, fill in the mesh-occluded areas in the target main image; Align the auxiliary channel image with the target main image to obtain the aligned auxiliary channel image; Perform preset image enhancement processing on the target main image and auxiliary channel images; construct an auxiliary matrix using the auxiliary channel images; The auxiliary matrix and the target main image are input into the trained multi-branch neural network to obtain the environmental information recognition result of the target main image under the mesh occlusion scene.
2. The image multi-channel enhancement and recognition method for mesh occlusion scenarios according to claim 1, characterized in that, When a target area is detected, the corresponding main image of the target area is acquired; Identify the occlusion areas and mesh parameters in the target main image, specifically including: When the robot detects the target area, it acquires an initial image and checks whether there are any mesh occlusions in the initial image; if there are no mesh occlusions, the process ends. When there is mesh occlusion, the orientation and center line of the mesh occlusion are detected, and the robot is moved to the center line of the mesh occlusion at a preset distance in front, and the target main image corresponding to the target area is collected. Collect repetitive structures with mesh occlusion in the target main image, perform template matching on the repetitive structures to determine the corresponding mesh template, and obtain the mesh density corresponding to the mesh template; Collect the occlusion area of the mesh, the mesh size of the occlusion area, and the occlusion ratio through the target main image; store the mesh density, mesh size, and occlusion ratio as mesh parameters.
3. The image multi-channel enhancement and recognition method for mesh occlusion scenarios according to claim 1, characterized in that, Feature point matching and registration are performed on several consecutive image frames to obtain background supplementation for the mesh occlusion area corresponding to the target main image, specifically including: Feature point matching and registration are performed on several consecutive image frames and the target main image respectively to obtain the environmental information corresponding to the feature points behind the mesh occlusion area of the target main image; Use environmental information as background supplement.
4. The image multi-channel enhancement and recognition method for mesh occlusion scenarios according to claim 1, characterized in that, By using background filling and mesh parameters, fill in the mesh-occluded areas in the target main image, specifically including: An affine transformation matrix is established based on the mesh size in the mesh parameters, and the background supplementation of different frames is registered to the coordinate system of the target main image. The weighted average method is used to fuse the background supplementation of multiple frames to form a background supplementation layer. A two-dimensional periodic mesh model is constructed based on mesh parameters; the effective light-transmitting area ratio is calculated by the occlusion ratio, and a mask matrix consistent with the physical occlusion characteristics is generated on the two-dimensional periodic mesh model using a non-uniform sampling algorithm; Through the formula: The final output = background supplement layer × mask matrix + target main image × (1 - mask matrix) to obtain the filled mesh occlusion area.
5. The image multi-channel enhancement and recognition method for mesh occlusion scenarios according to claim 1, characterized in that, Pre-defined image enhancement processing is performed on the target main image and auxiliary channel images, specifically including: Perform contrast-limited adaptive histogram equalization on the target main image; The target main image is input into an edge-preserving filter to process the structure and texture of the target main image; The auxiliary channel image is normalized and then converted into a grayscale or pseudo-color image.
6. The image multi-channel enhancement and recognition method for mesh occlusion scenarios according to claim 1, characterized in that, Before inputting the auxiliary matrix and the target main image into the trained multi-branch neural network to obtain the environmental information recognition result of the target main image under the mesh occlusion scene, the method further includes: The historical main image, auxiliary matrix, and environmental information recognition results under mesh occlusion scenarios are obtained as sample data. Using sample data, a multi-branch neural network is trained to obtain a well-trained multi-branch neural network.
7. An image multi-channel enhancement and recognition system for mesh occlusion scenarios, characterized in that, The system includes: The recognition module is used to acquire the target main image corresponding to the target area when the target area is detected; and to identify the mesh occlusion area and mesh parameters in the target main image. The acquisition module is used to capture continuous image frames in the target area, while simultaneously acquiring images from auxiliary channels; The filling module is used to perform feature point matching and registration on several consecutive image frames to obtain background supplementation for the mesh occlusion area corresponding to the target main image; using the background supplementation and mesh parameters, the mesh occlusion area in the target main image is filled in. The construction module is used to align the auxiliary channel image with the target main image to obtain the aligned auxiliary channel image; perform preset image enhancement processing on the target main image and the auxiliary channel image; and construct an auxiliary matrix using the auxiliary channel image. The results module is used to input the auxiliary matrix and the target main image into the trained multi-branch neural network to obtain the environmental information recognition results of the target main image under the mesh occlusion scene.
8. The image multi-channel enhancement and recognition system for mesh occlusion scenarios according to claim 7, characterized in that, The recognition module includes a recognition unit. This is used to acquire an initial image when the robot detects a target area, and then detect whether there is any mesh occlusion in the initial image; if there is no mesh occlusion, the process ends. When there is mesh occlusion, the orientation and center line of the mesh occlusion are detected, and the robot is moved to the center line of the mesh occlusion at a preset distance in front, and the target main image corresponding to the target area is collected. Collect repetitive structures with mesh occlusion in the target main image, perform template matching on the repetitive structures to determine the corresponding mesh template, and obtain the mesh density corresponding to the mesh template; Collect the occlusion area of the mesh, the mesh size of the occlusion area, and the occlusion ratio through the target main image; store the mesh density, mesh size, and occlusion ratio as mesh parameters.
9. The image multi-channel enhancement and recognition system for mesh occlusion scenarios according to claim 7, characterized in that, The filling module includes filling units. This is used to establish an affine transformation matrix based on the grid size in the mesh parameters, and to register the background supplementation of different frames to the coordinate system of the target main image; a weighted average method is used to fuse the background supplementation of multiple frames to form a background supplementation layer. A two-dimensional periodic mesh model is constructed based on mesh parameters; The effective light-transmitting area ratio is calculated by the occlusion ratio, and a mask matrix consistent with the physical occlusion characteristics is generated on a two-dimensional periodic grid model using a non-uniform sampling algorithm. Through the formula: The final output = background supplement layer × mask matrix + target main image × (1 - mask matrix) to obtain the filled mesh occlusion area.
10. A non-volatile computer storage medium, characterized in that, It stores computer instructions, which, when executed, implement an image multi-channel enhancement and recognition method for mesh occlusion scenarios as described in any one of claims 1-6.