Method, apparatus, storage medium and program product for gripping a side wall outer panel

Abstract

The application provides a side wall outer plate grabbing method and device, a storage medium and a program product, relates to the technical field of vehicles, and is used for improving the precision of a grabbing device in grabbing a side wall outer plate and comprises the following steps: acquiring multiple images of a to-be-detected area of a side wall outer plate of a vehicle; wherein the exposure degrees of different images are different; fusing the multiple images to obtain a fused image; determining the deviation between the image extraction pose of the side wall outer plate in the fused image and the real pose of the side wall outer plate; and controlling the grabbing device to grab the side wall outer plate based on the deviation.

Description

Technical Field

[0001] This invention relates to the field of vehicle technology, and specifically to a method, device, storage medium, and program product for gripping a side panel. Background Technology

[0002] As a key body panel component of a vehicle, the side panel is a core component that determines the structural strength, appearance flatness, and subsequent welding and assembly precision of the vehicle body. To improve the efficiency and stability of side panel material handling, vision technology uses industrial cameras to replace manual vision, combined with gripping devices such as robotic systems, to automate the gripping and handling of side panel materials. This significantly reduces manpower input in welding lines and other scenarios while improving material handling efficiency. In this process, machine vision needs to accurately locate key areas of the side panel, such as the rigid B-pillar support area, the front A-pillar area, and the freely suspended triangular window area, providing the robot with pose data for each area. This allows the robot to adjust the posture of its end effector, ensuring precise contact between the gripping points and the side panel surface.

[0003] However, during actual grasping, the positioning accuracy of the side panel is often affected by imaging interference caused by changes in external lighting. This may cause key features on the surface of the side panel to appear overexposed and white, with blurred dark areas or reflective halos in the image, as well as deformation of the side panel itself. These problems will directly cause the visual positioning result to be inconsistent with the actual pose of the side panel, resulting in gaps or misalignments between the robot's grasping point and the surface of the panel, ultimately leading to unstable grasping or even grasping failure.

[0004] Therefore, improving the accuracy of the gripping device in gripping the outer side panel is an urgent problem to be solved. Summary of the Invention

[0005] The purpose of this invention is to provide a method, device, storage medium, and program product for gripping side panel outer panels, so as to solve the problem of low accuracy in gripping side panel outer panels by gripping devices in related technologies.

[0006] To achieve the above objectives, the technical solution adopted by the present invention is as follows: In a first aspect, the present invention provides a method for grasping a side panel, comprising: acquiring multiple images of a detection area of ​​a vehicle side panel; wherein the different images have different exposures; fusing the multiple images to obtain a fused image; determining the deviation between the image extraction pose of the side panel in the fused image and the actual pose of the side panel; and controlling a grasping device to grasp the side panel based on the deviation.

[0007] Based on the aforementioned technical methods, the surface of the side panel is mostly made of highly reflective steel plates with complex shapes. A single exposure image is prone to overexposure in bright areas (brightening of reflective areas) or underexposure in dark areas (blurring of shadow areas), leading to the loss of key features of the side panel. By acquiring images at different exposure levels, effective information from each area of ​​the side panel can be captured separately, avoiding feature loss due to changes in lighting. Furthermore, by fusing images at different exposure levels, the information from these images is integrated. The resulting fused image retains most of the effective information from the initial image. Therefore, when the machine vision algorithm recognizes the fused image, it can more accurately extract the key features of the side panel, reducing feature extraction errors caused by image blurring and providing a reliable basis for subsequent pose calculations.

[0008] Furthermore, since the fused image retains the complete features of each region of the side panel, the image extraction pose can be closer to the actual state of the panel. As a result, the deviation between the image extraction pose and the actual pose is calculated more accurately. Based on this more accurate deviation, the gripping device (such as a robot) can grip the side panel more accurately, thereby significantly reducing the risk of gripping instability and improving the success rate of automatic feeding of the side panel.

[0009] Furthermore, multiple images are fused to obtain a fused image, including: for each image, performing binarization processing to obtain a binarized image; adding multiple binarized images to obtain a mask matrix; and fusing multiple images based on the mask matrix to obtain a fused image.

[0010] Based on the aforementioned techniques, binarizing each image removes redundant color information, retaining only the black-and-white contrast between the target area and the background. This highlights the characteristic contours of the monitoring area on the outer side panel, reducing the amount of data and interference in subsequent processing. Adding multiple binarized images yields a mask matrix, which cumulatively strengthens the common effective areas across multiple images while weakening noise or random interference in individual images, resulting in more reliable area marking. When fusing multiple images based on the mask matrix, key area information marked by the mask can be accurately preserved, while irrelevant background or interfering areas are filtered out. This ensures the fused image integrates the effective features of multiple images while avoiding noise superposition, ultimately yielding a clearer fusion result with more prominent key information.

[0011] Furthermore, for each image, the image is binarized to obtain a binarized image, including: for each pixel in the image, if the pixel brightness is greater than or equal to a preset brightness threshold, the value corresponding to the pixel is set to a first value; or, if the pixel brightness in the image matrix of the image is less than the preset brightness threshold, the value corresponding to the pixel is set to a second value; the value of each pixel in the binarized image is either the first value or the second value.

[0012] Based on the aforementioned technical methods, by setting a preset brightness threshold, the brightness value of each pixel in the image is compared with this threshold, thereby simplifying the pixel values ​​into two results: pixels that meet the brightness threshold are assigned a first value (usually 1), and pixels that do not meet the threshold are assigned a second value (usually 0). The final binarized image consists only of these two values, simplifying the image from multi-grayscale or color information to black-and-white binary information. This removes redundant details, enhances the contrast between the target area and the background, and facilitates the extraction of key features from the image.

[0013] Furthermore, the multiple binarized images are added together to obtain a mask matrix, including: pixel-level superposition of multiple binarized images to obtain an accumulation matrix; setting all elements in the accumulation matrix except the second value to the first value to obtain the mask matrix.

[0014] Based on the aforementioned techniques, multiple binarized images (with pixel values ​​of 0 or 1) are superimposed pixel-wise. In the resulting accumulation matrix, each pixel value represents the total number of times that location is 1 in all binarized images (potentially an overexposed area, i.e., the target area). This ensures that the mask matrix covers all possible target areas (as long as a location is a target in any image, it will be marked as 1), avoiding target loss due to missed detection in a single image. All non-zero elements in the accumulation matrix are uniformly set to 1. The final mask matrix only retains target areas that have appeared in at least one image, thus simplifying the mask matrix and reducing interference in subsequent processing.

[0015] Furthermore, the multiple images include: a first image, a second image, and a third image; the exposure of the first image, the second image, and the third image increases sequentially; the multiple images are fused based on the mask matrix to obtain a fused image, including: multiplying the image matrix of the first image and the mask matrix, adding the image matrix of the second image, and then subtracting the product of the second image and the mask matrix to obtain an initial fused image; multiplying the image matrix of the initial fused image and the mask matrix, adding the image matrix of the third image, and then subtracting the product of the third image and the mask matrix to obtain the fused image.

[0016] Based on the aforementioned technical methods, potentially overexposed areas are locked using a mask matrix, and the first image with the lowest exposure is always used to cover these areas. This avoids overexposure interference from high-exposure images in these areas, preserving the relatively clear details of the first image. For non-overexposed areas outside the mask, a second image with medium exposure is used for transition, followed by a third image with the highest exposure to enhance the brightness of the shadows, making the originally dark areas clearer. This achieves the goal of correcting overexposed areas with low exposure and enhancing shadow areas with high exposure, balancing the preservation of bright details with the enhancement of shadow brightness, ultimately resulting in an image without overexposure, with complete details, and balanced brightness.

[0017] Furthermore, the areas to be detected include: the A-pillar area of ​​the side wall, the B-pillar area of ​​the side wall, and the triangular window area of ​​the side wall; the fused images include: a first fused image of the B-pillar area of ​​the side wall, a second fused image of the A-pillar area of ​​the side wall, and a third fused image of the triangular window area of ​​the side wall; determining the deviation between the image extraction pose of the side wall outer panel and the true pose of the side wall outer panel in the fused images includes: determining a first deviation between the image extraction pose of the side wall outer panel and the true pose of the side wall outer panel in the first fused image based on the B-pillar area of ​​the side wall; determining a second deviation between the image extraction pose of the side wall outer panel and the true pose of the side wall outer panel in the second fused image based on the A-pillar area of ​​the side wall; determining a third deviation between the image extraction pose of the side wall outer panel and the true pose of the side wall outer panel in the third fused image based on the triangular window area of ​​the side wall; and determining the deviation between the image extraction pose of the side wall outer panel and the true pose of the side wall outer panel in the fused images based on the first deviation, the second deviation, and the third deviation.

[0018] Based on the above technical means, by calculating the deviations of the three key areas of the side panel, namely the A-pillar, B-pillar, and triangular window, and then combining the deviations of multiple areas to determine the overall pose deviation, the characteristic information of different areas can be fully utilized, avoiding misjudgment of deviations caused by a single area, improving the comprehensiveness and reliability of deviation calculation, and more realistically reflecting the overall pose state of the side panel.

[0019] Furthermore, based on the first deviation, the second deviation, and the third deviation, the deviation between the image extraction pose of the side outer plate in the fused image and the true pose of the side outer plate is determined, including: based on the difference between the first deviation and the second deviation, and the difference between the first deviation and the third deviation, the deviation between the image extraction pose of the side outer plate in the fused image and the true pose of the side outer plate is determined.

[0020] Based on the aforementioned technical methods, the B-pillar area typically exhibits a more stable structure and more prominent features, resulting in higher accuracy in detecting its deviation (first deviation), making it suitable as a benchmark. By calculating the difference between the first deviation and the deviations of other areas, it can be determined whether the deviations of the A-pillar and triangular window areas are consistent with the benchmark deviation. If the difference is within a reasonable range, it indicates that the deviation trends of each area are unified, and the overall deviation can be fine-tuned based on the benchmark deviation combined with the difference, taking into account the actual state of each area. The resulting overall deviation more closely matches the true pose of the side panel, improving the accuracy and robustness of pose judgment.

[0021] Secondly, the present invention provides a gripping device for a side panel, comprising: an acquisition unit for acquiring multiple images of a detection area of ​​a vehicle side panel; wherein the different images have different exposures; a processing unit for fusing the multiple images to obtain a fused image; determining the deviation between the image extraction pose of the side panel in the fused image and the actual pose of the side panel; and controlling the gripping device to grip the side panel based on the deviation.

[0022] Thirdly, the present invention provides an electronic device comprising: a processor and a memory configured to store processor-executable instructions; wherein the processor is configured to execute the instructions to implement any of the optional side panel gripping methods described in the first aspect above.

[0023] Fourthly, a computer-readable storage medium is provided, on which instructions are stored, such that when the instructions in the computer-readable storage medium are executed by a device, the device is able to perform any of the optional side panel grasping methods described in the first aspect.

[0024] Fifthly, a computer program product is provided, the computer program product including computer instructions that, when executed on a processor of a device, enable the device to perform a gripping method for the side panel as optionally described in any of the first aspects above.

[0025] The beneficial effects of this invention are: (1) The surface of the side panel is mostly made of highly reflective steel plate with complex shape. A single exposure image is prone to overexposure of bright areas (whitening of reflective areas) or underexposure of dark areas (blurring of shadow areas), resulting in the loss of key features of the side panel. By acquiring images with different exposures, effective information of each area of ​​the side panel can be captured separately, avoiding feature loss due to changes in lighting. By fusing images with different exposures, the information of different exposure images can be integrated. The final fused image can retain most of the effective information of the initial image. Thus, when the machine vision algorithm recognizes the fused image, it can extract the key features of the side panel more accurately, reduce the feature extraction error caused by image blurring, and provide a reliable basis for subsequent pose calculation.

[0026] Furthermore, since the fused image retains the complete features of each region of the side panel, the image extraction pose can be closer to the actual state of the panel. As a result, the deviation between the image extraction pose and the actual pose is calculated more accurately. Based on this more accurate deviation, the gripping device (such as a robot) can grip the side panel more accurately, thereby significantly reducing the risk of gripping instability and improving the success rate of automatic feeding of the side panel.

[0027] (2) Binarizing each image can remove redundant color information, retain only the black-and-white contrast between the target area and the background, highlight the feature contours of the monitoring area of ​​the outer side panel, and reduce the amount of data and interference in subsequent processing. Adding multiple binarized images to obtain a mask matrix can strengthen the effective areas that coexist in multiple images by accumulation, while weakening noise or random interference in a single image, forming a more reliable basis for area marking. When fusing multiple images based on the mask matrix, the key area information of the mask marking can be accurately retained, and irrelevant background or interference areas can be filtered out, so that the fused image integrates the effective features of multiple images and avoids noise superposition, ultimately obtaining a clearer fusion result with more prominent key information.

[0028] (3) By setting a preset brightness threshold, the brightness value of each pixel in the image is compared with the threshold, thereby simplifying the pixel value into two results: pixels that meet the brightness standard are assigned a first value (usually 1), and pixels that do not meet the standard are assigned a second value (usually 0). The final binarized image consists of only these two values, which simplifies the image from multiple grayscale or color information to black and white binary information, thereby removing redundant details, enhancing the contrast between the target area and the background, and making it easier to extract key features from the image.

[0029] (4) Multiple binarized images (pixel values ​​of 0 or 1) are superimposed at the pixel level. In the resulting accumulation matrix, each pixel value represents the total number of times that position is 1 in all binarized images (possibly an overexposed area, i.e., the target area). This ensures that the mask matrix covers all possible target areas (as long as a position is a target in any image, it will be marked as 1), avoiding target loss due to missed detection in a single image. All non-zero elements in the accumulation matrix are uniformly set to 1. In the final mask matrix, only the target area that has appeared in at least one image is retained, thereby simplifying the mask matrix and reducing interference in subsequent processing.

[0030] (5) By locking potentially overexposed areas using a mask matrix, the first image with the lowest exposure is always used to cover these areas, avoiding overexposure interference from high-exposure images and preserving the relatively clear details of the first image. For non-overexposed areas outside the mask, a second image with medium exposure is used for transition, and then a third image with the highest exposure is used to enhance the brightness of the dark areas, making the originally dark areas clearer. This achieves the goal of correcting overexposed areas with low exposure and enhancing dark areas with high exposure, balancing the preservation of bright details with the enhancement of dark brightness, and finally obtaining an image without overexposure, with complete details and balanced brightness.

[0031] (6) By calculating the deviations of the three key areas of the side panel, namely the A-pillar, B-pillar and triangular window, and then combining the deviations of multiple areas to determine the overall pose deviation, the characteristic information of different areas can be fully utilized, avoiding misjudgment of deviation caused by a single area, improving the comprehensiveness and reliability of deviation calculation, and more realistically reflecting the overall pose state of the side panel.

[0032] (7) The B-pillar area is usually more stable in structure and more distinctive in features, and its deviation (first deviation) has higher detection accuracy, making it suitable as a benchmark. By calculating the difference between the first deviation and the deviations of other areas, it can be determined whether the deviations of the A-pillar and triangular window areas are consistent with the benchmark deviation. If the difference is within a reasonable range, it indicates that the deviation trends of each area are consistent, and the overall deviation can be finely adjusted based on the benchmark deviation and the difference, taking into account the actual state of each area. The final overall deviation is more in line with the true pose of the side panel, improving the accuracy and robustness of the pose judgment. Attached Figure Description

[0033] Figure 1 This is a schematic diagram of the structure of a gripping system for a side panel proposed in this invention; Figure 2 This is a schematic diagram showing the posture of the side panel in a container according to the present invention; Figure 3 This is a schematic diagram of the area to be detected in a side panel according to the present invention; Figure 4 This is a flowchart illustrating a method for gripping a side panel according to the present invention. Figure 5 This is a flowchart illustrating another method for gripping the side panel proposed in this invention. Figure 6 This is a schematic diagram of an image fusion method proposed in this invention; Figure 7 This is a schematic diagram of the structure of a gripping device for a side panel according to the present invention; Figure 8 This is a schematic diagram of the structure of an electronic device proposed in this invention. Detailed Implementation

[0034] The embodiments of the present invention will be described below with reference to the accompanying drawings and preferred embodiments. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be understood that the preferred embodiments are only for illustrating the present invention and not for limiting the scope of protection of the present invention.

[0035] It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Therefore, the drawings only show the components related to the present invention and are not drawn according to the actual number, shape and size of the components in the actual implementation. In the actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0036] As a key body panel component of a vehicle, the side panel is a core component that determines the structural strength, appearance flatness, and subsequent welding and assembly precision of the vehicle body. To improve the efficiency and stability of side panel material handling, vision technology uses industrial cameras to replace manual vision, combined with gripping devices such as robotic systems, to automate the gripping and handling of side panel materials. This significantly reduces manpower input in welding lines and other scenarios while improving material handling efficiency. In this process, machine vision needs to accurately locate key areas of the side panel, such as the rigid B-pillar support area, the front A-pillar area, and the freely suspended triangular window area, providing the robot with pose data for each area. This allows the robot to adjust the posture of its end effector, ensuring precise contact between the gripping points and the side panel surface.

[0037] However, during actual grasping, the positioning accuracy of the side panel is often affected by imaging interference caused by changes in external lighting. This may cause key features on the surface of the side panel to appear overexposed and white, with blurred dark areas or reflective halos in the image, as well as deformation of the side panel itself. These problems will directly cause the visual positioning result to be inconsistent with the actual pose of the side panel, resulting in gaps or misalignments between the robot's grasping point and the surface of the panel, ultimately leading to unstable grasping or even grasping failure.

[0038] Therefore, improving the accuracy of the gripping device in gripping the outer side panel is an urgent problem to be solved.

[0039] Based on this, the present invention proposes a method, device, storage medium, and program product for grasping a side panel, comprising: acquiring multiple images of the area to be detected of the vehicle's side panel; wherein the different images have different exposures; fusing the multiple images to obtain a fused image; determining the deviation between the image extraction pose of the side panel in the fused image and the actual pose of the side panel; and controlling the grasping device to grasp the side panel based on the deviation. By acquiring images with different exposures, effective information of each area of ​​the side panel can be captured separately, avoiding feature loss due to changes in illumination. By fusing images with different exposures, the information of the different exposure images is integrated, and the final fused image retains most of the effective information of the initial image. Therefore, when the machine vision algorithm recognizes the fused image, it can more accurately extract the key features of the side panel, reduce feature extraction errors caused by image blurring, and provide a reliable basis for subsequent pose calculation.

[0040] Furthermore, since the fused image retains the complete features of each region of the side panel, the image extraction pose can be closer to the actual state of the panel. As a result, the deviation between the image extraction pose and the actual pose is calculated more accurately. Based on this more accurate deviation, the gripping device (such as a robot) can grip the side panel more accurately, thereby significantly reducing the risk of gripping instability and improving the success rate of automatic feeding of the side panel.

[0041] In some embodiments, such as Figure 1 As shown, this invention proposes a side panel grasping system, comprising: a controller 10, a vision acquisition device 20, and a grasping device 30. The vision acquisition device 20 acquires images of multiple areas to be detected on the side panel. The controller 10 fuses the multiple images to obtain a fused image. Based on the fused image, the deviation between the image extraction pose of the side panel in the fused image and the actual pose of the side panel is determined. The grasping device 30 is then controlled to grasp the side panel based on this deviation.

[0042] As one possible implementation, the vision acquisition device 20 can be an industrial camera that supports dynamic adjustment of exposure parameters. By adjusting the shutter speed and image sensor gain, a low-exposure image can be obtained with a short shutter speed and low gain, and a high-exposure image can be obtained with a long shutter speed and high gain (which can fully present the outline details of dark areas such as the shadow of the triangular window), thereby covering the exposure range from low to high, and thus acquiring images from low exposure to high exposure.

[0043] The gripping device 30 can be an industrial robot (with an end effector). The robot body is mounted next to the container, and its range of motion covers the gripping position of the outer panel and the subsequent loading position. The end effector includes multiple vacuum suction cups, each equipped with a negative pressure sensor and a floating joint to ensure that it adheres to the surface of the outer panel during gripping without damaging the workpiece. The robot control cabinet receives deviation commands (translation / rotation parameters) sent by the controller and drives the robot to adjust the posture of the end effector to complete the gripping.

[0044] In some embodiments, the side panel's posture within the container is as follows: Figure 2 As shown, the container serves as the carrier for transferring the side panel from the stamping workshop to the welding line. The side panel will be fixed in a nearly vertical or slightly tilted (tilt angle 5°-15°) posture according to the gripping requirements.

[0045] Because the rigidity of different areas of the side panel varies (the B-pillar area is the most rigid, the triangular window area is the least rigid, and the A-pillar area is in between), the supporting structure is specifically designed for each area. This results in significant differences in the posture of each area. When the gripping device grasps the side panel, the gripping points need to be adjusted according to the posture characteristics of each area to avoid deformation of the panel or gripping failure due to uneven force. Therefore, the vision acquisition device needs to include multiple industrial cameras to acquire images of different areas.

[0046] In some embodiments, the detection area of ​​the side panel is as follows: Figure 3 As shown, it includes: the side wall A-pillar area, the side wall B-pillar area, and the side wall triangular window area.

[0047] The A-pillar area of ​​the side panel is located at the frontmost end of the outer side panel, corresponding to the pillar area between the lower edge of the windshield and the upper edge of the front door. It connects to the front longitudinal beam of the vehicle body forward, the front door frame backward, and the door sill beam downward.

[0048] The B-pillar area, located in the middle of the outer side panel, corresponds to the pillar area between the rear edge of the front door and the front edge of the rear door. It connects upwards to the roof side beam and downwards through the sill beam. Structurally, the B-pillar area has an internal B-pillar reinforcement plate, which is spot-welded to the outer panel to form a double-layer rigid structure. This structure has the strongest overall rigidity (it is the only area of ​​the outer side panel that can withstand significant external forces without deformation). Because of this characteristic, when calculating visual deviations, the B-pillar area is usually used as a benchmark to calculate the overall pose deviation, serving as a reference for calculating deviations in other areas.

[0049] The side triangular window area is located at the rear end of the side outer panel, corresponding to the triangular area between the rear edge of the rear door and the C-pillar (rear windshield side pillar). It connects upward to the roof side beam and downward to the rear fender. Inward, it is the side and rear view area for rear passengers. Structurally, the side triangular window area is a thin-shell frame with only a few small clip mounting positions on the inner side of the frame, making it the weakest in overall rigidity.

[0050] Understandably, the B-pillar area of ​​the side panel is the main support for the outer panel, bearing most of its weight. Grasping the B-pillar area prevents the panel from denting in the middle or tilting entirely due to insufficient rigidity at the gripping point. The A-pillar area, located at the front of the outer panel, is only fixed by a single support block within the container, but its posture is relatively controllable. Grasping the A-pillar area, along with the B-pillar area, creates a two-point positioning system, restricting the panel's rotational freedom around the B-pillar area. Combined with the triangular window area, the B-pillar becomes the main gripping point for bearing weight, while the A-pillar and triangular window gripping points restrict offset and deformation. The gripping points in these three areas together form a stable triangular structure, preventing the outer panel from swaying, rotating, or sagging during transport, thus improving gripping stability.

[0051] In other embodiments, the side panel gripping system of the present invention can also grip other components that can be gripped by visual positioning, such as the front fender of an automobile. The front fender is a thin shell covering the wheels at the front of the vehicle body. Referring to the gripping logic of the side panel, multiple gripping points can be set on the front fender. Combined with the wheel arch contour and front mounting hole features extracted by visual positioning, stable gripping can be achieved, which is suitable for automatic feeding of the front fender.

[0052] like Figure 4 As shown, the side panel gripping method of the present invention can be applied to the controller of the above-mentioned side panel gripping system, and includes the following steps: S401. Acquire multiple images of the area to be detected on the outer side panel of the vehicle.

[0053] Different images have different exposure levels.

[0054] As one possible implementation, different exposure parameters are preset based on the lighting characteristics of different areas of the side panel. These parameters include shutter speed (exposure time) and sensor gain. The lower the exposure, the shorter the shutter speed, and the lower the sensor gain. The controller sends an image capture command to the camera, triggering the camera to capture the first image (low exposure). The camera then sequentially switches to medium and high exposure parameters according to a preset gradient. Each time the parameters are switched, the camera captures an image, until all exposure levels are completed, resulting in multiple images of the area to be detected.

[0055] It should be understood that the side panel is a large, thin-shell component with a surface mostly made of highly reflective galvanized steel sheet and complex shapes. If the exposure is too high, the highly reflective areas will be overexposed and appear white due to excessive light, resulting in blurred key features such as positioning holes and edges. If the exposure is too low, the dark areas will be underexposed and appear black due to insufficient light, leading to the loss of frame outline and detailed features. By acquiring images of the area to be inspected at different exposure levels, the feature details of each area of ​​the side panel can be comprehensively captured, avoiding information loss caused by single exposure.

[0056] S402. Fuse multiple images to obtain a fused image.

[0057] One possible approach is to perform positional calibration on multiple images to ensure that key features in the multiple images can overlap. For each pixel in the calibrated image, the brightness values ​​in the images with different exposures are weighted and averaged to obtain the weighted brightness value of each pixel, which is also the brightness value of each pixel in the fused image.

[0058] In one possible implementation, positional calibration is performed on multiple images to ensure that key features in the multiple images can overlap. This can be achieved by using one of the images as a reference, identifying common key features in other exposed images (the positions of these points remain unchanged under different exposures) through a feature point matching algorithm, calculating the deviation of key features in other exposed images relative to the low-exposure image, and aligning multiple images to the same coordinate system based on the deviation of key features to ensure that the positions of key features overlap.

[0059] In one possible implementation, for each pixel in the calibrated image, the brightness values ​​of the images at different exposure levels are weighted and averaged. This can be achieved by determining the weights of the images at different exposure levels. For low-exposure images, bright pixels that are not overexposed are retained first. For high-exposure images, dark pixels that are not underexposed are retained first. The weight of the medium-exposure image is 1 minus the weights of the other two types of images. After obtaining the weights of each type of image, the brightness values ​​of the pixels in each exposure image are weighted and averaged according to their respective weights.

[0060] It should be understood that each of the multi-exposure images has its limitations (such as underexposure in short exposures and overexposure in long exposures). By using image fusion algorithms to filter and integrate the pixel information of multiple images, the fused image can retain the clear features of the highly reflective areas in the low-exposure image, the complete details of the dark areas in the high-exposure image, and the continuous surface information of the transition areas in the medium-exposure image. This achieves a distortion-free presentation of the full range of features of the side panel to be detected, avoiding overexposure and whitening or underexposure and blackening caused by single exposure, and providing a complete and clear image basis for subsequent pose extraction.

[0061] S403. Determine the deviation between the image extraction pose of the side outer panel in the fused image and the true pose of the side outer panel.

[0062] Among them, the image extraction pose representation is based on the fused image, and the digital pose parameters of the side panel in the image coordinate system are obtained through feature recognition and algorithm calculation. The real pose refers to the physical space pose actually occupied by the side panel in the container, which is affected by placement deviation (such as container positioning error) and its own deformation (such as the triangular window hanging down).

[0063] As one possible implementation, based on fused images, the pose parameters of the detection areas (A-pillar, B-pillar, triangular window) of the side panel are extracted using a feature matching algorithm. For example, the coordinates of the B-pillar positioning holes in the image coordinate system are identified, their translation and rotation parameters are calculated, the image-extracted pose and the true pose are aligned to a unified coordinate system, and the coordinates of the image-extracted pose and the true pose are compared to obtain the translational deviation and rotational deviation. These deviations directly reflect the gap between the image positioning and the physical reality.

[0064] As another possible implementation, the detection area includes: the side A-pillar area, the side B-pillar area, and the side triangular window area; the fused image includes: a first fused image of the side B-pillar area, a second fused image of the side A-pillar area, and a third fused image of the side triangular window area; determining the deviation between the image extraction pose of the side outer panel in the fused image and the true pose of the side outer panel includes: based on the first fused image of the side B-pillar area, determining a first deviation between the image extraction pose of the side outer panel in the first fused image and the true pose of the side outer panel; based on the second fused image of the side A-pillar area, determining a second deviation between the image extraction pose of the side outer panel in the second fused image and the true pose of the side outer panel; based on the third fused image of the side triangular window area, determining a third deviation between the image extraction pose of the side outer panel in the third fused image and the true pose of the side outer panel; and based on the first deviation, the second deviation, and the third deviation, determining the deviation between the image extraction pose of the side outer panel in the fused image and the true pose of the side outer panel.

[0065] For example, point cloud data acquired by a camera based on the side B-pillar region with good overall rigidity is compared and calculated with the established feature template to obtain the pose deviation Δ2 of the object, which is the first deviation. Point cloud data acquired by a camera based on the features of the side A-pillar region is compared and calculated with the established feature template to obtain the pose deviation Δ1 of the object, which is the second deviation. Point cloud data acquired by a camera based on the features of the side triangular window region is best fitted with the established feature model to obtain the pose deviation Δ3 of the object, which is the third deviation.

[0066] It should be understood that by calculating the deviations of the three key areas of the side panel's outer wall—the A-pillar, B-pillar, and triangular window—separately, and then combining the deviations of multiple areas to determine the overall pose deviation, the characteristic information of different areas can be fully utilized, avoiding misjudgment of deviations caused by a single area, improving the comprehensiveness and reliability of the deviation calculation, and more realistically reflecting the overall pose state of the side panel's outer wall.

[0067] In one possible implementation, the deviation between the image-extracted pose of the side panel in the fused image and the true pose of the side panel is determined based on the first deviation, the second deviation, and the third deviation, including: determining the deviation between the image-extracted pose of the side panel in the fused image and the true pose of the side panel based on the difference between the first deviation and the second deviation, and the difference between the first deviation and the third deviation.

[0068] For example, the pose correction deviation Δm=Δ1-Δ2 caused by deformation in the A-pillar region and the pose correction deviation Δn=Δ3-Δ2 caused by deformation in the triangular window region are calculated, and the mean value Δ=(Δm+Δn) / 2 is calculated to obtain the deviation between the final image extraction pose and the true pose.

[0069] It should be understood that the B-pillar area typically has a more stable structure and more prominent features, resulting in higher accuracy in detecting its deviation (the first deviation), making it suitable as a benchmark. By calculating the difference between the first deviation and the deviations of other areas, it can be determined whether the deviations of the A-pillar and triangular window areas are consistent with the benchmark deviation. If the difference is within a reasonable range, it indicates that the deviation trends of each area are unified, and the overall deviation can be fine-tuned based on the benchmark deviation combined with the difference, taking into account the actual state of each area. The final overall deviation more closely matches the true pose of the side panel, improving the accuracy and robustness of pose judgment.

[0070] S404, The control gripping device grips the side panel based on the deviation.

[0071] As one possible implementation, the distance between the gripping device and the side panel is adjusted based on the deviation, and the side panel is gripped based on the adjusted distance.

[0072] As another possible implementation, the turning device is first adjusted based on the first deviation, and then the distance between the gripping device and the side panel is adjusted based on the deviation.

[0073] Understandably, when the side panel has significant deformation, the deviation between the image-extracted pose and the actual pose will be large. In this case, the gripping device cannot grip the side panel based on the deviation. Therefore, before controlling the gripping device to grip the side panel based on the deviation, it is necessary to determine whether the deviation between the image-extracted pose and the actual pose is greater than a preset deviation threshold. If it is greater than the preset deviation threshold, the gripping device should be stopped, and an early warning mechanism should be triggered. At the same time, the pose deviation data of the current side panel and the corresponding workstation information should be recorded, waiting for the operator to inspect, adjust, or replace the deformed workpiece. If it is less than the deviation threshold, the gripping device should be controlled to grip the side panel based on the deviation to ensure the reliability of the gripping action.

[0074] Therefore, the surface of the side panel is mostly made of highly reflective steel plates with complex shapes. A single exposure image is prone to overexposure in bright areas (reflective areas appear washed out) or underexposure in dark areas (shadow areas become blurred), resulting in the loss of key features of the side panel. By acquiring images at different exposure levels, effective information of each area of ​​the side panel can be captured separately, avoiding feature loss due to changes in lighting. Furthermore, by fusing images at different exposure levels, the information from different exposure images is integrated. The final fused image retains most of the effective information from the initial image. Consequently, when the machine vision algorithm recognizes the fused image, it can more accurately extract the key features of the side panel, reducing feature extraction errors caused by image blurring and providing a reliable basis for subsequent pose calculations.

[0075] Furthermore, since the fused image retains the complete features of each region of the side panel, the image extraction pose can be closer to the actual state of the panel. As a result, the deviation between the image extraction pose and the actual pose is calculated more accurately. Based on this more accurate deviation, the gripping device (such as a robot) can grip the side panel more accurately, thereby significantly reducing the risk of gripping instability and improving the success rate of automatic feeding of the side panel.

[0076] In some embodiments, binarization can simplify single-image information and highlight the contrast between the target region and the background. Then, multiple binarized images are added together to form a mask matrix to integrate effective features from multiple images and reduce noise interference. Finally, multiple images are fused based on this mask matrix, resulting in a fused image with more prominent key information and less redundant interference. This provides a more reliable foundation for subsequent image analysis, feature extraction, and other processing. Therefore, as... Figure 5 As shown, the above S402 includes the following steps: S501. For each image, perform binarization processing to obtain a binarized image.

[0077] As one possible implementation, for each pixel in the image, if the pixel brightness is greater than or equal to a preset brightness threshold, the value corresponding to the pixel is set to a first value; or, if the pixel brightness in the image matrix of the image is less than the preset brightness threshold, the value corresponding to the pixel is set to a second value; the value of each pixel in the binarized image is either the first value or the second value.

[0078] For example, starting with low exposure and progressing to high exposure, the image matrix of the three images can be as follows (255 indicates that this area of ​​the image is overexposed):

[0079] Binarize all images, setting overexposed images to 1 and underexposed images to 0. The binarized results are as follows:

[0080] It should be understood that by setting a preset brightness threshold, the brightness value of each pixel in the image is compared with this threshold, thereby simplifying the pixel values ​​into two results: pixels that meet the brightness threshold are assigned a first value (usually 1), and pixels that do not meet the threshold are assigned a second value (usually 0). The final binarized image consists only of these two values, simplifying the image from multi-grayscale or color information to black-and-white binary information, thereby removing redundant details, enhancing the contrast between the target area and the background, and facilitating the extraction of key features in the image.

[0081] S502. Add the multiple binarized images together to obtain the mask matrix.

[0082] As one possible implementation, multiple binarized images are superimposed pixel by pixel to obtain an accumulation matrix; all elements in the accumulation matrix except the second value are set to the first value to obtain a mask matrix.

[0083] For example, the binarized images in S501 are summed, and pixels greater than or equal to 1 are set to 1, while others are set to 0, resulting in the following mask matrix: .

[0084] It should be understood that pixel-level overlay of multiple binarized images (pixel values ​​of 0 or 1) results in an accumulation matrix where each pixel value represents the total number of times that location is 1 across all binarized images (potentially an overexposed area, i.e., the target area). This ensures the mask matrix covers all possible target areas (if a location is a target in any image, it will be marked as 1), avoiding target loss due to missed detection in a single image. By uniformly setting all non-zero elements in the accumulation matrix to 1, the final mask matrix only retains target areas that have appeared in at least one image, thus simplifying the mask matrix and reducing interference in subsequent processing.

[0085] S503. Based on the mask matrix, multiple images are fused to obtain a fused image.

[0086] As one possible implementation, multiple images include: a first image, a second image, and a third image; the exposure of the first image, the second image, and the third image increases sequentially. Fusing multiple images based on a mask matrix to obtain a fused image can be achieved by multiplying the image matrix of the first image and the mask matrix, adding the image matrix of the second image, and then subtracting the product of the second image and the mask matrix to obtain an initial fused image; then, multiplying the image matrix of the initial fused image and the mask matrix, adding the image matrix of the third image, and then subtracting the product of the third image and the mask matrix to obtain the fused image.

[0087] For example, the fusion process includes two steps: First fusion:

[0088] Second fusion:

[0089] It should be understood that the exposure of the first, second, and third images increases sequentially: the first image has the lowest exposure, with clear details in the dark areas but insufficient information in the bright areas; the third image has the highest exposure, possibly with overexposure in the bright areas (even medium-brightness areas), but clear details in the dark areas; the second image has a medium exposure, representing a transitional state, with some areas possibly having moderate brightness, but the dark areas are not as stable as in the third image, and the bright areas are not as stable as in the first image. To overcome the interference of lighting, it is necessary to preserve the unexposed bright details in the low-exposure image (first image), enhance the brightness of the dark areas using the high-exposure image (third image), and simultaneously avoid interference from overexposed areas caused by high exposure.

[0090] In the first step, when a pixel in the mask matrix is ​​1: that area uses the pixel value of the first image. A high-exposure image might be overexposed in this area (loss of detail); using a low-exposure first image can preserve image detail and eliminate the effects of overexposure. When a pixel in the mask matrix is ​​0: that area uses the pixel value of the second image. These areas might be too dark in the low-exposure image; using a medium-exposure second image can appropriately increase brightness and enhance details in the shadows.

[0091] In the second step, when a pixel in the mask matrix is ​​1: the pixel value of the first image in the initial fused image is retained. When a pixel in the mask matrix is ​​0: the pixel value of the third image is used for that area. Using the third image with the highest exposure can maximize the brightness of dark areas and enhance the details of originally dark areas.

[0092] Understandably, by using a mask matrix to lock potentially overexposed areas, the first image with the lowest exposure is always used to cover these areas, preventing overexposure interference from high-exposure images and preserving the relatively clear details of the first image. For non-overexposed areas outside the mask, a second image with medium exposure is used for transition, followed by a third image with the highest exposure to enhance the brightness of the shadows, making the originally dark areas clearer. This achieves the goal of correcting overexposed areas with low exposure and enhancing shadow areas with high exposure, balancing the preservation of bright details with the enhancement of shadow brightness, ultimately resulting in an image without overexposure, with complete details, and balanced brightness.

[0093] It should be noted that this application does not limit the number of images. For N images, the process of determining the fusion image can be as follows: Figure 6 As shown, multiple images, including image A, image B, ... image N, are binarized to obtain images a, b, ... image n. The binarized images are then superimposed to obtain a mask matrix. Then, the images are fused: fused image 1 = image A * mask matrix + image B - mask matrix * image B; fused image 2 = fused image 1 * mask matrix + image C - mask matrix * image C; and so on, fused image N-1 = fused image N-2 * mask matrix + image N - mask matrix * image N.

[0094] Therefore, binarizing each image removes redundant color information, retaining only the black-and-white contrast between the target area and the background, highlighting the characteristic contours of the monitoring area on the outer side panel, and reducing the amount of data and interference in subsequent processing. Adding multiple binarized images to obtain a mask matrix strengthens the common effective areas in the multiple images while weakening noise or random interference in individual images, forming a more reliable basis for region marking. When fusing multiple images based on the mask matrix, key region information marked by the mask can be accurately preserved, while irrelevant background or interference areas are filtered out. This ensures that the fused image integrates the effective features of multiple images while avoiding noise superposition, ultimately resulting in a clearer fusion result with more prominent key information.

[0095] The foregoing mainly describes the solutions provided by the embodiments of the present invention from a methodological perspective. To achieve the above functions, the gripping device or electronic device of the side panel includes hardware structures and / or software modules corresponding to the execution of each function. Those skilled in the art should readily recognize that, in conjunction with the units and algorithm steps of the various examples described in the embodiments disclosed herein, the present invention can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed in hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present invention.

[0096] According to the above method, the gripping device or electronic device for the side panel can be functionally divided into modules. For example, the gripping device or electronic device for the side panel may include functional modules corresponding to each functional division, or two or more functions may be integrated into one processing module. The integrated module can be implemented in hardware or as a software functional module. It should be noted that the module division in this embodiment is illustrative and only represents one logical functional division; other division methods may be used in actual implementation.

[0097] Reference Figure 7 The side panel gripping device 700 provided in this embodiment of the invention includes: an acquisition unit 701 and a processing unit 702.

[0098] The acquisition unit 701 is used to acquire multiple images of the area to be detected on the outer side panel of the vehicle; wherein the exposure of different images is different.

[0099] The processing unit 702 is used to fuse multiple images to obtain a fused image; determine the deviation between the image extraction pose of the side panel in the fused image and the actual pose of the side panel; and control the grasping device to grasp the side panel based on the deviation.

[0100] In some embodiments, the processing unit 702 is specifically configured to: perform binarization processing on each image to obtain a binarized image; add multiple binarized images to obtain a mask matrix; and fuse multiple images based on the mask matrix to obtain a fused image.

[0101] In some embodiments, the processing unit 702 is specifically configured to, for each pixel in the image, set the value corresponding to the pixel to a first value if the pixel brightness is greater than or equal to a preset brightness threshold; or, set the value corresponding to the pixel to a second value if the pixel brightness in the image matrix of the image is less than the preset brightness threshold; the value of each pixel in the binarized image is either the first value or the second value.

[0102] In some embodiments, the processing unit 702 is specifically used to perform pixel-level superposition of multiple binarized images to obtain an accumulation matrix; and to set all elements in the accumulation matrix except for the second value to the first value to obtain a mask matrix.

[0103] In some embodiments, the multiple images include: a first image, a second image, and a third image; the exposure of the first image, the second image, and the third image increases sequentially; the processing unit 702 is specifically configured to multiply the image matrix and the mask matrix of the first image, add the image matrix of the second image, and then subtract the product of the second image and the mask matrix to obtain an initial fused image; multiply the image matrix and the mask matrix of the initial fused image, add the image matrix of the third image, and then subtract the product of the third image and the mask matrix to obtain the fused image.

[0104] In some embodiments, the region to be detected includes: a side A-pillar region, a side B-pillar region, and a side triangular window region; the fused image includes: a first fused image of the side B-pillar region, a second fused image of the side A-pillar region, and a third fused image of the side triangular window region; the processing unit 702 is specifically configured to, based on the first fused image of the side B-pillar region, determine a first deviation between the image extraction pose of the side outer plate in the first fused image and the true pose of the side outer plate; based on the second fused image of the side A-pillar region, determine a second deviation between the image extraction pose of the side outer plate in the second fused image and the true pose of the side outer plate; based on the third fused image of the side triangular window region, determine a third deviation between the image extraction pose of the side outer plate in the third fused image and the true pose of the side outer plate; and based on the first deviation, the second deviation, and the third deviation, determine the deviation between the image extraction pose of the side outer plate in the fused image and the true pose of the side outer plate.

[0105] In some embodiments, the processing unit 702 is specifically used to determine the deviation between the image extraction pose of the side panel in the fused image and the true pose of the side panel based on the difference between the first deviation and the second deviation, and the difference between the first deviation and the third deviation.

[0106] like Figure 8 As shown, the electronic device 800 provided in this embodiment of the invention includes, but is not limited to, a processor 801 and a memory 802.

[0107] The memory 802 described above is used to store the executable instructions of the processor 801. It is understood that the processor 801 is configured to execute instructions to implement the side panel gripping method in the above embodiment.

[0108] It should be noted that those skilled in the art will understand that Figure 8 The electronic device structure shown does not constitute a limitation on the electronic device; the electronic device may include, but is not limited to, other electronic devices. Figure 8 This may indicate more or fewer components, or combinations of certain components, or different component arrangements.

[0109] The processor 801 is the control center of the electronic device. It connects various parts of the electronic device via various interfaces and lines. By running or executing software programs and / or modules stored in the memory 802, and by calling data stored in the memory 802, it performs various functions and processes data, thereby providing overall monitoring of the electronic device. The processor 801 may include one or more processing units. Optionally, the processor 801 may integrate an application processor and a modem processor. The application processor mainly handles the operating system, user interface, and applications, while the modem processor mainly handles wireless communication. It is understood that the modem processor may not be integrated into the processor 801.

[0110] The memory 802 can be used to store software programs and various data. The memory 802 may primarily include a program storage area and a data storage area. The program storage area may store the operating system, application programs required by at least one functional module (such as a determination unit, processing unit, etc.), etc. Furthermore, the memory 802 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0111] In an exemplary embodiment, a vehicle is also provided, including the electronic equipment described above.

[0112] In an exemplary embodiment, a computer-readable storage medium including instructions is also provided, such as a memory 802 including instructions, which can be executed by a processor 801 of an electronic device 800 to implement the methods in the above embodiments.

[0113] In actual implementation, Figure 7 The functions of each module can be provided by Figure 8 The processor 801 calls the computer program stored in the memory 802 to implement the process. The specific execution process can be found in the method section of the previous embodiment, and will not be repeated here.

[0114] Optionally, the computer-readable storage medium may be a non-transitory computer-readable storage medium, such as a read-only memory (ROM), random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device.

[0115] In an exemplary embodiment, the present invention also provides a computer program product including one or more instructions, which can be executed by a processor 801 of an electronic device to perform the methods described above.

[0116] It should be noted that when one or more instructions in the computer-readable storage medium or computer program product are executed by the processor of an electronic device, they implement the various processes of the above method embodiments and achieve the same technical effect as the above method. To avoid repetition, they will not be described again here.

[0117] Through the above description of the embodiments, those skilled in the art can clearly understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0118] In the several embodiments provided by this invention, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0119] The units described as separate components may or may not be physically separate. A component shown as a unit can be one or more physical units; that is, it can be located in one place or distributed in multiple different locations. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0120] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0121] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiments of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods of the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, ROM, RAM, magnetic disks, or optical disks.

[0122] The above embodiments are merely preferred embodiments provided to fully illustrate the present invention, and the scope of protection of the present invention is not limited thereto. Equivalent substitutions or modifications made by those skilled in the art based on the present invention are all within the scope of protection of the present invention.

Claims

1. A method for gripping a side panel, characterized in that, include: Multiple images of the area to be detected on the outer side panel of the vehicle are acquired; the exposure of each image is different. The multiple images are fused to obtain a fused image; Determine the deviation between the image-extracted pose of the side panel in the fused image and the true pose of the side panel; The gripping device is controlled to grip the outer side panel based on the deviation; The area to be detected includes: the side A-pillar area, the side B-pillar area, and the side triangular window area; the fused image includes: a first fused image of the side B-pillar area, a second fused image of the side A-pillar area, and a third fused image of the side triangular window area. The step of determining the deviation between the image-extracted pose of the side panel in the fused image and the true pose of the side panel includes: Based on the first fused image of the side B-pillar region, a first deviation is determined between the image extraction pose of the side outer panel in the first fused image and the true pose of the side outer panel; Based on the second fused image of the side A-pillar region, a second deviation is determined between the image extraction pose of the side outer panel in the second fused image and the true pose of the side outer panel; Based on the third fused image of the side triangular window region, a third deviation is determined between the image extraction pose of the side outer panel in the third fused image and the true pose of the side outer panel; Based on the first deviation, the second deviation, and the third deviation, the deviation between the image extraction pose of the side outer panel in the fused image and the actual pose of the side outer panel is determined.

2. The method according to claim 1, characterized in that, The process of fusing the multiple images to obtain a fused image includes: For each of the images, the image is binarized to obtain a binarized image; The multiple binarized images are added together to obtain a mask matrix; The multiple images are fused based on the mask matrix to obtain the fused image.

3. The method according to claim 2, characterized in that, For each of the images, binarization processing is performed to obtain a binary image, including: For each pixel in the image, if the pixel brightness is greater than or equal to a preset brightness threshold, the value corresponding to the pixel is set to a first value; or, If the pixel brightness in the image matrix of the image is less than the preset brightness threshold, the value corresponding to the pixel is set to the second value; the value of each pixel in the binarized image is either the first value or the second value.

4. The method according to claim 2, characterized in that, The step of adding the multiple binarized images to obtain a mask matrix includes: The multiple binarized images are superimposed pixel by pixel to obtain an accumulation matrix; Set all elements in the accumulated matrix except for the second value to the first value to obtain the mask matrix.

5. The method according to claim 2, characterized in that, The plurality of images includes: a first image, a second image, and a third image; the exposure of the first image, the second image, and the third image increases sequentially; the fusion of the plurality of images based on the mask matrix to obtain the fused image includes: The initial fused image is obtained by multiplying the image matrix of the first image and the mask matrix, adding the image matrix of the second image, and then subtracting the product of the second image and the mask matrix. The product of the image matrix of the initial fused image and the mask matrix is ​​added to the image matrix of the third image, and then the product of the third image and the mask matrix is ​​subtracted to obtain the fused image.

6. The method according to claim 1, characterized in that, The step of determining the deviation between the image extraction pose of the side panel in the fused image and the true pose of the side panel, based on the first deviation, the second deviation, and the third deviation, includes: Based on the difference between the first deviation and the second deviation, and the difference between the first deviation and the third deviation, the deviation between the image extraction pose of the side outer panel in the fused image and the true pose of the side outer panel is determined.

7. An electronic device, characterized in that, include: processor; Memory used to store the processor's executable instructions; The processor is configured to execute the instructions to implement the method as described in any one of claims 1 to 6.

8. A computer-readable storage medium, characterized in that, When the computer-executable instructions stored in the computer-readable storage medium are executed by the processor of the electronic device, the electronic device is capable of performing the method as described in any one of claims 1 to 6.

9. A computer program product, characterized in that, The computer program product includes computer instructions that, when executed on an electronic device, cause the electronic device to perform the method as described in any one of claims 1 to 6.

Images