Battery cell appearance detection method and device based on multi-modal image fusion

Abstract

The invention provides a battery cell appearance detection method and device based on multi-modal image fusion, and the method comprises the steps: obtaining a single-channel gray-scale map and a single-channel height map as input, carrying out the validity verification, generating a target fusion image through multi-modal fusion, and constructing a training set to train a deep learning model, and finally, accurate detection and result output of the appearance defects of the battery cell are realized. According to the invention, through combination of multi-modal image fusion and deep learning, the cell appearance detection precision and efficiency are significantly improved, and efficient and accurate intelligent quality control is realized.

Description

Technical Field

[0001] The present application relates to the field of industrial visual inspection, and in particular to a method and device for inspecting the appearance of battery cells based on multimodal image fusion. Background Art

[0002] In the field of industrial visual inspection, detecting defects in battery cell appearance is a critical step in ensuring the quality of lithium-ion battery products. Current mainstream technical solutions still rely heavily on single-channel 8-bit grayscale images (pixel value range 0-255) for image processing and analysis. These core processes typically include traditional algorithms such as grayscale threshold segmentation, edge detection, and morphological filtering.

[0003] However, existing technical solutions have significant limitations: single-channel grayscale images can only reflect the texture information of the battery cell surface, but cannot characterize topographical features (such as height variations such as bulges and warpage). The morphology of such defects often exhibits different characteristics with height variations (for example, a slight scratch may be buried by noise in a grayscale image, while a significant bulge will only appear as a slight grayscale fluctuation), making traditional algorithms prone to missed detections or false detections. Summary of the Invention

[0004] In view of this, the embodiments of the present application provide a method and device for battery cell appearance inspection based on multimodal image fusion, which can significantly improve the accuracy and efficiency of battery cell appearance inspection by combining multimodal image fusion with deep learning, and realize efficient and accurate intelligent quality control.

[0005] The technical solution of the embodiment of the present application is implemented as follows: In a first aspect, an embodiment of the present application provides a method for detecting the appearance of a battery cell based on multimodal image fusion, the method comprising: Obtain a first input image and a second input image; wherein the first input image is a single-channel grayscale image, the second input image is a single-channel height image, and the number of bits of the single-channel height image is higher than the number of bits of the single-channel grayscale image; Performing validity verification on the first input image and the second input image, terminating the process and outputting an exception code when either input image is empty or the size does not match; performing a fusion process on the first input image and the second input image to obtain a target fused image; A training set is constructed based on the target fusion image, a target deep learning model is constructed through the training set, and a battery cell appearance inspection result is output based on the target deep learning model.

[0006] In a second aspect, an embodiment of the present application further provides a battery cell appearance inspection device based on multimodal image fusion, the device comprising: An acquisition module is configured to acquire a first input image and a second input image; wherein the first input image is a single-channel grayscale image, and the second input image is a single-channel height image, and the number of bits of the single-channel height image is higher than the number of bits of the single-channel grayscale image; a verification module, configured to verify the validity of the first input image and the second input image, and terminate the process and output an exception code when either input image is empty or has a mismatched size; a fusion module, configured to fuse the first input image and the second input image to obtain a target fused image; A training module is used to construct a training set based on the target fusion image, and to construct a target deep learning model through the training set, and to output a cell appearance inspection result based on the target deep learning model.

[0007] In a third aspect, an embodiment of the present application further provides an electronic device comprising: a processor, a storage medium and a bus, wherein the storage medium stores machine-readable instructions executable by the processor. When the electronic device is running, the processor communicates with the storage medium through the bus, and the processor executes the machine-readable instructions to execute the battery cell appearance detection method based on multimodal image fusion as described in any one of the first aspects.

[0008] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the battery cell appearance detection method based on multimodal image fusion as described in any one of the first aspects is executed.

[0009] The embodiments of the present application have the following beneficial effects: Through multimodal image fusion technology, the texture details of a single-channel grayscale image and the morphological features of a single-channel height map are efficiently integrated into a multi-channel fused image, breaking through the information limitations of single-modal data and significantly improving the detection accuracy and robustness of battery cell appearance defects (such as scratches, bulges, and cracks). The construction of fused images effectively solves the problem of information loss during direct analysis of cross-bit depth data. At the same time, the integrity and consistency of the input data are ensured through a validity verification mechanism, avoiding model misjudgments caused by invalid data. The training set constructed based on fused images provides the deep learning model with multimodal data with high semantic relevance, enabling the trained model to simultaneously capture texture anomalies and geometric deformations on the battery cell surface, improving detection accuracy while reducing the false detection rate. It also supports real-time reasoning, providing an efficient, accurate, and intelligent solution for quality control of lithium battery cell production lines, significantly reducing the cost of manual inspection and the risk of defective product outflow. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without creative work.

[0011] Figure 1 10 is a flow chart of steps S101-S104 provided in an embodiment of the present application; Figure 2 It is a flowchart of steps S201-S202 provided in an embodiment of the present application; Figure 3 Schematic diagram of the process of steps S301-S302 provided in the embodiment of the present application; Figure 4 It is a flowchart of steps S401-S402 provided in an embodiment of the present application; Figure 5 Schematic diagram of the structure of a battery cell appearance inspection device based on multimodal image fusion provided in an embodiment of the present application; Figure 6 It is a schematic diagram of the composition structure of the electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0012] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. It should be understood that the drawings in the present application only serve the purpose of illustration and description and are not used to limit the scope of protection of the present application. In addition, it should be understood that the schematic drawings are not drawn to scale. The flowcharts used in this application illustrate the operations implemented according to some embodiments of the present application. It should be understood that the operations of the flowcharts can be implemented out of sequence, and steps without logical context can be reversed or implemented simultaneously. In addition, those skilled in the art, under the guidance of the contents of this application, can add one or more other operations to the flowchart, or remove one or more operations from the flowchart.

[0013] In the following description, reference is made to “some embodiments”, which describes a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0014] In addition, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. The components of the embodiments of the present application generally described and shown in the drawings here can be arranged and designed in various configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the claimed application, but merely represents selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work are within the scope of protection of the present application.

[0015] In the following description, the terms "first\second\third" involved are merely used to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that "first\second\third" can be interchanged with a specific order or sequence where permitted, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0016] It should be noted that the term "comprising" will be used in the embodiments of the present application to indicate the existence of the features declared thereafter, but does not exclude the addition of other features.

[0017] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this application belongs. The terms used herein are for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0018] See also Figure 1 , Figure 1 This is a flow chart of steps S101-S104 of the cell appearance detection method based on multimodal image fusion provided in the embodiment of the present application, which will be combined with Figure 1 Steps S101-S104 are shown for explanation.

[0019] In step S101, a first input image and a second input image are obtained; wherein the first input image is a single-channel grayscale image, the second input image is a single-channel height image, and the number of bits of the single-channel height image is higher than the number of bits of the single-channel grayscale image.

[0020] Here, the single-channel grayscale image is an 8-bit single-channel grayscale image, which is used to capture the surface texture features of the battery cell (such as scratches, pits, and printing defects). The 8-bit depth (0-255 grayscale levels) can meet the contrast requirements of common industrial inspections. For example, the grayscale mutation area corresponds to the surface scratch; The single-channel height map is a 16-bit single-channel height map. It obtains the three-dimensional morphology of the battery cell through laser scanning or structured light technology. The 16-bit depth (0-65535) can quantify micron-level deformations (such as bulges and depressions). For example, areas with sudden changes in height values ​​correspond to battery cell packaging defects.

[0021] It should be noted that, in this embodiment, the number of bits of the single-channel grayscale image is not limited to 8 bits, and the number of bits of the single-channel height map is not limited to 16 bits. It only needs to have a higher number of bits of the single-channel grayscale image than the single-channel height map.

[0022] In step S102 , the validity of the first input image and the second input image is verified. When any input image is empty or the size does not match, the process is terminated and an exception code is output.

[0023] Here, validity verification includes: empty image detection, size matching check and exception handling. Specifically, empty image detection is used to prevent invalid data caused by sensor failure or transmission interruption. Size matching check is used to ensure pixel-level alignment of grayscale image and height map. It checks whether the first input image, the second input image, and the final fused target fusion image are empty. At the same time, it checks whether the image sizes match. If any image is empty or any size parameter does not match, the operation is terminated and an exception code is reported.

[0024] In step S103, the first input image and the second input image are fused to obtain a target fused image.

[0025] Here, the texture information of the grayscale image and the topography of the height map are integrated to generate a richer RGB fused image. The complementary RGB channels enhance defect detection sensitivity and are compatible with the common input format of subsequent deep learning models (without requiring additional channel processing). Furthermore, the target fused image can also be a fused image of other multi-channels, such as a four-channel fused image. This is achieved by adding an alpha channel to the RGB fused image to indicate transparency, where alpha = 0 indicates full transparency.

[0026] In step S104, a training set is constructed based on the target fusion image, a target deep learning model is constructed through the training set, and a battery cell appearance inspection result is output based on the target deep learning model.

[0027] Here, a training set is constructed based on the RGB three-channel fusion image to build a target deep learning model for detecting battery cell appearance defects. The target deep learning model can be used in actual production on the production line. Specifically, the first input image and the second input image are input into the target deep learning model to obtain the target detection result.

[0028] In some embodiments, see Figure 2 , Figure 2This is a flow chart of steps S201-S202 provided in an embodiment of the present application. The first input image is an 8-bit single-channel grayscale image, and the second input image is a 16-bit single-channel height image. The first input image and the second input image are fused to obtain a target fused image. This can be achieved through steps S201-S202, which will be explained in combination with each step.

[0029] In step S201, the pixel values ​​of the first input image are mapped from the range of [0, 255] to the target normalization interval through a dynamic bit depth conversion algorithm, and the pixel values ​​of the second input image are mapped from the range of [0, 65535] to the target normalization interval.

[0030] In step S202, channel fusion is performed on the first input image and the second input image after bit depth conversion based on a preset weight distribution ratio to generate the RGB three-channel fused image.

[0031] Here, the pixel value ranges of the grayscale image (8-bit, 0-255) and the heightmap (16-bit, 0-65535) differ significantly. Direct fusion would result in information imbalance (for example, small changes in the heightmap would be masked by large fluctuations in the grayscale image). Therefore, for the grayscale image, all pixel values ​​are divided by 255 to compress them to the target normalization range, that is, between 0 and 1. For example, a grayscale value of 128 becomes 0.5, and 255 becomes 1.0. For the heightmap, all pixel values ​​are divided by 65535 to similarly compress them to the range of 0 to 1. For example, a height value of 32768 becomes approximately 0.5. This consistent value range of the two images allows for more equitable weight distribution during subsequent fusion while preserving the contrast information of the original data (for example, both dark patterns in the grayscale image and tiny bumps in the heightmap can be recognized).

[0032] Then determine the weight distribution to fuse the images according to the weight distribution ratio, for example: Grayscale dominant texture: Assign a higher weight (such as 60%) to detect defects such as scratches and printing offsets.

[0033] Height map dominant topography: Assign a medium weight (such as 30%) to detect defects such as bulges and dents.

[0034] Height map enhancement assistance: Assign a lower weight (such as 10%) and highlight local morphological changes by calculating the gradient of the height map (such as edge changes).

[0035] The above method can adjust the weights according to the specific defect types and further improve the adaptability of the model by introducing a dynamic weight allocation strategy.

[0036] In some embodiments, see Figure 3 , Figure 3This is a flow chart of steps S301-S302 provided in an embodiment of the present application. The pixel values ​​of the first input image are mapped from the range of [0, 255] to the target normalized interval through the dynamic bit depth conversion algorithm, and the pixel values ​​of the second input image are mapped from the range of [0, 65535] to the target normalized interval. This can be achieved through steps S301-S302 and will be explained in combination with each step.

[0037] In step S301 , a first pixel maximum value and a first pixel minimum value of the first input image are obtained, and a second pixel maximum value and a second pixel minimum value of the second input image are obtained.

[0038] In step S302 , the first pixel maximum value and the first pixel minimum value are compressed into a target normalization interval, and the second pixel maximum value and the second pixel minimum value are compressed into a target normalization interval.

[0039] Here, the pixel value ranges of input images with different bit depths (8-bit grayscale images and 16-bit height images) are uniformly mapped to the interval [0,1] to eliminate the dimension difference and facilitate subsequent fusion processing. Specifically: For grayscale images (8-bit), scan the entire image and find the maximum value (e.g., 255) and minimum value (e.g., 0) among all pixels. For height images (16-bit), scan the entire image and find the maximum value (e.g., 65535) and minimum value (e.g., 0) among all pixels.

[0040] Map all pixel values ​​in the grayscale image to the range [0,1]. For example, if the maximum value is 255 and the minimum value is 0, a pixel value of 128 will be mapped to 0.5 (128 / 255). If the minimum value is non-zero (such as 50), subtract the minimum value first, then divide by (maximum value - minimum value).

[0041] Map all pixel values ​​in the heightmap to the range [0,1]. For example, if the maximum value is 65535 and the minimum value is 0, then a pixel value of 32768 will be mapped to approximately 0.5 (32768 / 65535). If the minimum value is non-zero (such as 1000), subtract the minimum value first, then divide by (maximum value - minimum value).

[0042] If the minimum value of the image is not zero (due to sensor noise or background offset), the offset can be eliminated by subtracting the minimum value to ensure a reasonable range after mapping.

[0043] With this approach, the numerical range of the grayscale image and height map is no longer limited by the sensor bit depth, resulting in a fairer weight distribution during fusion. Extreme value compression preserves the contrast characteristics of the original image (such as the light and dark changes in scratches and the height differences in bulges), while also enhancing robustness and making it insensitive to sensor noise or background offset (through minimum value correction).

[0044] In some embodiments, see Figure 4 , Figure 4 This is a flow chart of steps S401-S402 provided in an embodiment of the present application. The channel fusion of the first input image and the second input image after bit depth conversion based on a preset weight distribution ratio to generate the RGB three-channel fused image can be achieved through steps S401-S402, which will be explained in combination with each step.

[0045] In step S401, the first input image and the second input image after bit depth conversion are mapped to the RGB space based on linear mapping; wherein the grayscale image data is mapped to the G channel, the depth image data is mapped to the R channel, and the blue channel is left blank.

[0046] In step S402, the first input image and the second input image are fused based on the weight parameter to obtain the RGB three-channel fused image; wherein, the RGB three-channel fused image = (1-α) × G channel + α × R channel, α is the weight parameter, and the weight parameters include at least a first weight parameter, a second weight parameter and a third weight parameter, the first weight parameter is used to detect surface scratches, the second weight parameter is used to detect concave and convex deformation, and the third weight parameter is used for general detection.

[0047] Here, the normalized grayscale image (G channel) and height map (R channel) are fused into an RGB image through linear mapping and weight distribution strategy to highlight the characteristics of different defect types (such as scratches or concave-convex deformations).

[0048] First, the normalized grayscale image data is directly mapped to the G channel (green channel) of the RGB image. The green channel is sensitive to texture detail and is suitable for detecting texture defects such as scratches and print offsets. Next, the normalized height map data is directly mapped to the R channel (red channel) of the RGB image. The red channel is sensitive to topographical changes and is suitable for detecting height anomalies such as bulges and dents. Blue channel processing: The blue channel (B channel) is left blank (i.e., no data is mapped) to avoid color interference.

[0049] The fused RGB image = (1-α)×G channel + α×R channel.

[0050] Among them, α is the weight parameter that controls the fusion ratio of the grayscale image and the height map.

[0051] Detecting surface scratches: α=0.3 The grayscale image (G channel) has a higher weight (70%), highlighting texture details.

[0052] Detecting bump deformation: α=0.7 The height map (R channel) has a higher weight (70%), highlighting the morphological changes.

[0053] General detection: α = 0.5, the grayscale image and height map are weighted equally (50% each), taking into account both texture and morphology information.

[0054] For example, in a scratch detection scenario, the scratch is highlighted in the G channel (green), while the R channel (red) may not change significantly. In the fused image, the scratch appears as a green area.

[0055] In a bump detection scenario, a bump is highlighted in the R channel (red), while the G channel (green) may not show any noticeable changes. In the fused image, the bump appears as a red area.

[0056] In a general inspection scenario, texture and topography information are balanced and integrated, and complex defects (such as scratches and bulges) can be displayed as a green and red mixed color at the same time.

[0057] This approach quickly adapts to different detection tasks (such as scratches or uneven deformations) by adjusting the alpha value. The intuitive color mapping (green = texture, red = topography) facilitates manual review or model debugging. The resulting RGB image can be directly fed into standard deep learning models (such as YOLO and ResNet).

[0058] In some embodiments, after obtaining a first pixel maximum value and a first pixel minimum value of the first input image, and obtaining a second pixel maximum value and a second pixel minimum value of the second input image, the method further includes: The pixel values ​​are screened based on mathematical statistics, wherein the screening process includes but is not limited to screening by setting a screening threshold, and the screening threshold is determined based on different battery cell models.

[0059] Here, mathematical statistical methods can be used to filter the pixel values ​​of the grayscale image and height map to eliminate the influence of noise or outliers and improve the robustness of subsequent fusion and detection.

[0060] Based on historical data or process specifications for different battery cell models, set reasonable pixel value ranges (such as the bulge threshold for height maps and the scratch contrast threshold for grayscale images). For example, the bulge threshold for a certain battery cell model's height map is set to 10μm (approximately 3277 pixels in a 16-bit image). The scratch contrast threshold for a grayscale image is set to 30 (the difference from the background in an 8-bit image).

[0061] For grayscale image screening, pixels below the contrast threshold can be filtered out (for example, areas with grayscale value changes less than 30 are considered background noise), thereby reducing the impact of sensor noise or uneven lighting.

[0062] For height map screening, pixels below the height threshold can be filtered out (for example, areas with height variations less than 10μm are considered normal morphology), thereby eliminating small fluctuations allowed by the process and focusing on significant defects.

[0063] Different battery cell models have different process requirements, so the thresholds need to be adjusted dynamically (for example, the bulge threshold for large-capacity cells is higher than that for small-capacity cells). In practical applications, the thresholds can also be automatically determined through histogram analysis or clustering algorithms (such as K-means), reducing manual intervention.

[0064] The above method filters out invalid pixels and improves the accuracy of subsequent fusion and detection. The threshold can be flexibly adjusted according to the battery cell model or process requirements to adapt to the production of multiple varieties. Combined with statistical methods (such as histogram analysis), the threshold can be automatically set to reduce labor costs.

[0065] In some embodiments, the method further comprises: Control parameters are added when generating the target fused image; the control parameters include contrast and / or reference offset value, the contrast is used to control the degree of distinction between different colors, and the reference offset value is used to select a reference plane for plane correction.

[0066] Here, by introducing control parameters such as contrast and reference offset value, the visual effect and geometric accuracy of the fused image are optimized, thereby improving the reliability and consistency of defect detection. As examples, the following can be included: Parameter 1: Contrast.

[0067] Function: Controls the degree of distinction between different colors (such as green texture and red appearance) in the RGB fusion image.

[0068] High Contrast: Enhances color differences and highlights defects (e.g., scratches appear greener and bumps appear redder).

[0069] Low contrast: reduces color differences and is suitable for observing complex defects (such as mixed areas of scratches and bulges).

[0070] Adjustment method: Dynamically adjust the contrast of RGB channels through linear stretching or nonlinear transformation (such as gamma correction).

[0071] Parameter 2: Base offset value.

[0072] Function: Select the reference surface for plane correction to eliminate geometric distortion caused by tilted sensor installation or uneven workpiece placement.

[0073] Reference plane selection: For example, a plane area in the height map is selected as the reference (offset value = 0), and the height difference of other areas is calculated relative to this plane to ensure the geometric alignment of the height map and the grayscale image, and avoid fusion misalignment caused by tilted viewing angle.

[0074] In some embodiments, the training set is annotated in the following manner: Annotating the training set based on a dichotomy method; Alternatively, the training set is labeled based on a pre-trained model; Alternatively, the training set is labeled based on a supervised learning algorithm.

[0075] Here, an efficient and accurate annotation method can be used to generate labeled training sets to provide supervision signals for the defect detection model and improve the model's ability to recognize defects such as scratches, concave and convex deformations.

[0076] The present invention provides three marking methods, see Table 1.

[0077] Table 1

[0078] In summary, the embodiments of the present application have the following beneficial effects: (1) Accuracy and integrity assurance of cross-depth data fusion.

[0079] The dynamic bit-depth conversion algorithm uses adaptive scaling factors and quantization space unification to losslessly map 16-bit height data (0-65535) and 8-bit texture data (0-255) to the same quantization range, effectively avoiding the information truncation and noise amplification problems caused by bit-depth differences in traditional methods. The introduction of a color mapping function further strengthens the semantic relevance of multimodal data, making height information (such as bulges / dents) and texture information (such as scratches / cracks) visually distinguishable in the fused image, providing a high-fidelity data foundation for subsequent defect detection.

[0080] (2) Improvement of multi-scenario defect detection capabilities.

[0081] The geometric and texture consistency of the fused image is achieved through reference plane correction and contrast control parameter optimization, ensuring spatial alignment and visual contrast balance between height and texture data. This significantly improves the model's ability to identify complex defects (such as scratches and bulges). Dynamic control parameters (such as contrast and reference offset) allow for flexible adjustment of the fusion strategy for different cell models or process requirements, enhancing the solution's adaptability to high-variety, low-volume production scenarios.

[0082] (3) Dual optimization of training set construction efficiency and model performance.

[0083] Annotation solutions that combine automated annotation with manual verification (e.g., pre-trained model annotation + manual correction) improve data preparation efficiency while ensuring annotation accuracy, significantly shortening model iteration cycles. High-quality training sets, through multimodal data fusion and refined defect annotation, significantly improve the detection accuracy of trained deep learning models on test sets, reduce false positives, and maintain inference speeds that meet real-time detection requirements.

[0084] (4) Robustness and economy of industrial deployment.

[0085] Noise and interference resistance are effectively suppressed through pixel value screening and dynamic bit depth conversion algorithms, suppressing interference factors such as sensor noise and uneven lighting, ensuring the model maintains stable performance in complex industrial environments. The resource-friendly design, through a lightweight model architecture and hardware acceleration optimization, reduces computing resource requirements, enabling the solution to be deployed on edge computing devices, significantly reducing deployment costs.

[0086] Based on the same inventive concept, the embodiments of the present application also provide a battery cell appearance detection device based on multimodal image fusion corresponding to the battery cell appearance detection method based on multimodal image fusion in the first embodiment. Since the principle of solving the problem by the device in the embodiments of the present application is similar to the above-mentioned battery cell appearance detection method based on multimodal image fusion, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be repeated.

[0087] like Figure 5 As shown, Figure 5 : is a schematic diagram of the structure of a battery cell appearance inspection device 500 based on multimodal image fusion provided in an embodiment of the present application. The battery cell appearance inspection device 500 based on multimodal image fusion includes: An acquisition module 501 is configured to acquire a first input image and a second input image; wherein the first input image is a single-channel grayscale image, and the second input image is a single-channel height image, and the number of bits of the single-channel height image is higher than the number of bits of the single-channel grayscale image; A verification module 502 is configured to verify the validity of the first input image and the second input image, and terminate the process and output an exception code when either input image is empty or has a mismatched size. A fusion module 503 is configured to fuse the first input image and the second input image to obtain a target fused image; The training module 504 is used to construct a training set based on the target fusion image, build a target deep learning model through the training set, and output a battery cell appearance inspection result based on the target deep learning model.

[0088] It should be understood by those skilled in the art that Figure 5The functions of each unit in the battery cell appearance inspection device 500 based on multimodal image fusion can be understood by referring to the related description of the battery cell appearance inspection method based on multimodal image fusion. Figure 5 The functions of the various units in the battery cell appearance inspection device 500 based on multimodal image fusion shown can be implemented by a program running on a processor, or by a specific logic circuit.

[0089] In a possible implementation, the first input image is an 8-bit single-channel grayscale image, and the second input image is a 16-bit single-channel height image. The fusion module 503 performs a fusion process on the first input image and the second input image to obtain a target fused image, including: Mapping pixel values ​​of the first input image from a range of [0, 255] to a target normalized interval, and mapping pixel values ​​of the second input image from a range of [0, 65535] to a target normalized interval, using a dynamic bit depth conversion algorithm; Channel fusion is performed on the first input image and the second input image after bit depth conversion based on a preset weight distribution ratio to generate the RGB three-channel fused image.

[0090] In one possible implementation, the fusion module 503 maps pixel values ​​of the first input image from a range of [0, 255] to a target normalized interval, and maps pixel values ​​of the second input image from a range of [0, 65535] to a target normalized interval, using a dynamic bit depth conversion algorithm, including: Obtaining a first pixel maximum value and a first pixel minimum value of the first input image, and obtaining a second pixel maximum value and a second pixel minimum value of the second input image; The first pixel maximum value and the first pixel minimum value are compressed into a target normalization interval, and the second pixel maximum value and the second pixel minimum value are compressed into a target normalization interval.

[0091] In a possible implementation, the fusion module 503 performs channel fusion on the first input image and the second input image after bit depth conversion based on a preset weight distribution ratio to generate the RGB three-channel fused image, including: Mapping the first input image and the second input image after bit depth conversion to the RGB space based on a linear mapping; wherein the grayscale image data is mapped to the G channel, the depth image data is mapped to the R channel, and the blue channel is left blank; Based on the weight parameter, the first input image and the second input image are fused to obtain the RGB three-channel fused image; wherein, the RGB three-channel fused image = (1-α)×G channel + α×R channel, α is the weight parameter, and the weight parameters include at least a first weight parameter, a second weight parameter, and a third weight parameter, the first weight parameter is used to detect surface scratches, the second weight parameter is used to detect concave and convex deformation, and the third weight parameter is used for general detection.

[0092] In a possible implementation, after obtaining the first pixel maximum value and the first pixel minimum value of the first input image and obtaining the second pixel maximum value and the second pixel minimum value of the second input image, the fusion module 503 further includes: The pixel values ​​are screened based on mathematical statistics, wherein the screening process includes but is not limited to screening by setting a screening threshold, and the screening threshold is determined based on different battery cell models.

[0093] In a possible implementation, the fusion module 503 further includes: Control parameters are added when generating the target fused image; the control parameters include contrast and / or reference offset value, the contrast is used to control the degree of distinction between different colors, and the reference offset value is used to select a reference plane for plane correction.

[0094] In one possible implementation, the training set is labeled in the following manner: Annotating the training set based on a dichotomy method; Alternatively, the training set is labeled based on a pre-trained model; Alternatively, the training set is labeled based on a supervised learning algorithm.

[0095] The battery cell appearance inspection device based on multimodal image fusion has the following beneficial effects: (1) Accuracy and integrity assurance of cross-depth data fusion.

[0096] The dynamic bit-depth conversion algorithm uses adaptive scaling factors and quantization space unification to losslessly map 16-bit height data (0-65535) and 8-bit texture data (0-255) to the same quantization range, effectively avoiding the information truncation and noise amplification problems caused by bit-depth differences in traditional methods. The introduction of a color mapping function further strengthens the semantic relevance of multimodal data, making height information (such as bulges / dents) and texture information (such as scratches / cracks) visually distinguishable in the fused image, providing a high-fidelity data foundation for subsequent defect detection.

[0097] (2) Improvement of multi-scenario defect detection capabilities.

[0098] The geometric and texture consistency of the fused image is achieved through reference plane correction and contrast control parameter optimization, ensuring spatial alignment and visual contrast balance between height and texture data. This significantly improves the model's ability to identify complex defects (such as scratches and bulges). Dynamic control parameters (such as contrast and reference offset) allow for flexible adjustment of the fusion strategy for different cell models or process requirements, enhancing the solution's adaptability to high-variety, low-volume production scenarios.

[0099] (3) Dual optimization of training set construction efficiency and model performance.

[0100] Annotation solutions that combine automated annotation with manual verification (e.g., pre-trained model annotation + manual correction) improve data preparation efficiency while ensuring annotation accuracy, significantly shortening model iteration cycles. High-quality training sets, through multimodal data fusion and refined defect annotation, significantly improve the detection accuracy of trained deep learning models on test sets, reduce false positives, and maintain inference speeds that meet real-time detection requirements.

[0101] (4) Robustness and economy of industrial deployment.

[0102] Noise and interference resistance are effectively suppressed through pixel value screening and dynamic bit depth conversion algorithms, suppressing interference factors such as sensor noise and uneven lighting, ensuring the model maintains stable performance in complex industrial environments. The resource-friendly design, through a lightweight model architecture and hardware acceleration optimization, reduces computing resource requirements, enabling the solution to be deployed on edge computing devices, significantly reducing deployment costs.

[0103] like Figure 6 As shown, Figure 6 This is a schematic diagram of the structure of an electronic device 600 provided in an embodiment of the present application. The electronic device 600 includes: A processor 601, a storage medium 602 and a bus 603, wherein the storage medium 602 stores machine-readable instructions executable by the processor 601. When the electronic device 600 is running, the processor 601 communicates with the storage medium 602 via the bus 603, and the processor 601 executes the machine-readable instructions to perform the steps of the battery cell appearance detection method based on multimodal image fusion described in the embodiment of the present application.

[0104] In actual application, the various components in the electronic device 600 are coupled together via bus 603. It is understood that bus 603 is used to realize the connection and communication between these components. In addition to the data bus, bus 603 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, Figure 6 Various buses are labeled as bus 603.

[0105] The electronic device has the following beneficial effects: (1) Accuracy and integrity assurance of cross-depth data fusion.

[0106] The dynamic bit-depth conversion algorithm uses adaptive scaling factors and quantization space unification to losslessly map 16-bit height data (0-65535) and 8-bit texture data (0-255) to the same quantization range, effectively avoiding the information truncation and noise amplification problems caused by bit-depth differences in traditional methods. The introduction of a color mapping function further strengthens the semantic relevance of multimodal data, making height information (such as bulges / dents) and texture information (such as scratches / cracks) visually distinguishable in the fused image, providing a high-fidelity data foundation for subsequent defect detection.

[0107] (2) Improvement of multi-scenario defect detection capabilities.

[0108] The geometric and texture consistency of the fused image is achieved through reference plane correction and contrast control parameter optimization, ensuring spatial alignment and visual contrast balance between height and texture data. This significantly improves the model's ability to identify complex defects (such as scratches and bulges). Dynamic control parameters (such as contrast and reference offset) allow for flexible adjustment of the fusion strategy for different cell models or process requirements, enhancing the solution's adaptability to high-variety, low-volume production scenarios.

[0109] (3) Dual optimization of training set construction efficiency and model performance.

[0110] Annotation solutions that combine automated annotation with manual verification (e.g., pre-trained model annotation + manual correction) improve data preparation efficiency while ensuring annotation accuracy, significantly shortening model iteration cycles. High-quality training sets, through multimodal data fusion and refined defect annotation, significantly improve the detection accuracy of trained deep learning models on test sets, reduce false positives, and maintain inference speeds that meet real-time detection requirements.

[0111] (4) Robustness and economy of industrial deployment.

[0112] Noise and interference resistance are effectively suppressed through pixel value screening and dynamic bit depth conversion algorithms, suppressing interference factors such as sensor noise and uneven lighting, ensuring the model maintains stable performance in complex industrial environments. The resource-friendly design, through a lightweight model architecture and hardware acceleration optimization, reduces computing resource requirements, enabling the solution to be deployed on edge computing devices, significantly reducing deployment costs.

[0113] An embodiment of the present application further provides a computer-readable storage medium storing executable instructions. When the executable instructions are executed by at least one processor 601, the battery cell appearance detection method based on multimodal image fusion described in the embodiment of the present application is implemented.

[0114] In some embodiments, the storage medium can be a magnetic random access memory (FRAM), a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a flash memory, a magnetic surface storage, an optical disc, or a compact disc read-only memory (CD-ROM); it can also be various devices including one or any combination of the above memories.

[0115] In some embodiments, executable instructions may be in the form of a program, software, software module, script, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment.

[0116] As an example, executable instructions may, but do not necessarily, correspond to a file in a file system, may be stored as part of a file that stores other programs or data, for example, in one or more scripts in a HyperText Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple coordinated files (for example, files storing one or more modules, subroutines, or code portions).

[0117] By way of example, executable instructions may be deployed to be executed on one computing device, or on multiple computing devices at one site, or on multiple computing devices distributed across multiple sites and interconnected by a communication network.

[0118] The computer-readable storage medium has the following advantages: (1) Accuracy and integrity assurance of cross-depth data fusion.

[0119] The dynamic bit-depth conversion algorithm uses adaptive scaling factors and quantization space unification to losslessly map 16-bit height data (0-65535) and 8-bit texture data (0-255) to the same quantization range, effectively avoiding the information truncation and noise amplification problems caused by bit-depth differences in traditional methods. The introduction of a color mapping function further strengthens the semantic relevance of multimodal data, making height information (such as bulges / dents) and texture information (such as scratches / cracks) visually distinguishable in the fused image, providing a high-fidelity data foundation for subsequent defect detection.

[0120] (2) Improvement of multi-scenario defect detection capabilities.

[0121] The geometric and texture consistency of the fused image is achieved through reference plane correction and contrast control parameter optimization, ensuring spatial alignment and visual contrast balance between height and texture data. This significantly improves the model's ability to identify complex defects (such as scratches and bulges). Dynamic control parameters (such as contrast and reference offset) allow for flexible adjustment of the fusion strategy for different cell models or process requirements, enhancing the solution's adaptability to high-variety, low-volume production scenarios.

[0122] (3) Dual optimization of training set construction efficiency and model performance.

[0123] Annotation solutions that combine automated annotation with manual verification (e.g., pre-trained model annotation + manual correction) improve data preparation efficiency while ensuring annotation accuracy, significantly shortening model iteration cycles. High-quality training sets, through multimodal data fusion and refined defect annotation, significantly improve the detection accuracy of trained deep learning models on test sets, reduce false positives, and maintain inference speeds that meet real-time detection requirements.

[0124] (4) Robustness and economy of industrial deployment.

[0125] Noise and interference resistance are effectively suppressed through pixel value screening and dynamic bit depth conversion algorithms, suppressing interference factors such as sensor noise and uneven lighting, ensuring the model maintains stable performance in complex industrial environments. The resource-friendly design, through a lightweight model architecture and hardware acceleration optimization, reduces computing resource requirements, enabling the solution to be deployed on edge computing devices, significantly reducing deployment costs.

[0126] In the several embodiments provided in this application, it should be understood that the disclosed methods and electronic devices can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0127] The modules described as separate components may or may not be physically separate, and the components shown as modules may or may not be physical units, that is, they may be located in one place or distributed across multiple network elements. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0128] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0129] If the functions are implemented as software functional units and sold or used as independent products, they can be stored in a processor-executable, non-volatile, computer-readable storage medium. Based on this understanding, the technical solution of this application, or the portion that contributes to the prior art, or the portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, platform server, or network device, etc.) to execute all or part of the steps of the methods described in various embodiments of this application. The aforementioned storage media include various media that can store program code, such as USB flash drives, mobile hard drives, ROM, RAM, magnetic disks, or optical disks.

[0130] The above are only specific embodiments of the present application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A method for detecting the appearance of a battery cell based on multimodal image fusion, characterized in that: The method comprises: Obtain a first input image and a second input image; wherein the first input image is a single-channel grayscale image, the second input image is a single-channel height image, and the number of bits of the single-channel height image is higher than the number of bits of the single-channel grayscale image; Performing validity verification on the first input image and the second input image, terminating the process and outputting an exception code when either input image is empty or the size does not match; performing a fusion process on the first input image and the second input image to obtain a target fused image; A training set is constructed based on the target fusion image, a target deep learning model is constructed through the training set, and a battery cell appearance inspection result is output based on the target deep learning model.

2. The method according to claim 1, characterized in that The first input image is an 8-bit single-channel grayscale image, the second input image is a 16-bit single-channel height image, and the fusion processing of the first input image and the second input image to obtain a target fused image includes: Mapping pixel values ​​of the first input image from a range of [0, 255] to a target normalized interval, and mapping pixel values ​​of the second input image from a range of [0, 65535] to a target normalized interval, using a dynamic bit depth conversion algorithm; Channel fusion is performed on the first input image and the second input image after bit depth conversion based on a preset weight distribution ratio to generate an RGB three-channel fused image.

3. The method according to claim 2, characterized in that Mapping the pixel values ​​of the first input image from the range of [0, 255] to the target normalized interval by a dynamic bit depth conversion algorithm, and mapping the pixel values ​​of the second input image from the range of [0, 65535] to the target normalized interval, comprises: Obtaining a first pixel maximum value and a first pixel minimum value of the first input image, and obtaining a second pixel maximum value and a second pixel minimum value of the second input image; The first pixel maximum value and the first pixel minimum value are compressed into a target normalization interval, and the second pixel maximum value and the second pixel minimum value are compressed into a target normalization interval.

4. The method according to claim 2, characterized in that The performing channel fusion on the first input image and the second input image after bit depth conversion based on a preset weight distribution ratio to generate the RGB three-channel fused image includes: Mapping the first input image and the second input image after bit depth conversion to the RGB space based on a linear mapping; wherein the grayscale image data is mapped to the G channel, the depth image data is mapped to the R channel, and the blue channel is left blank; Based on the weight parameter, the first input image and the second input image are fused to obtain the RGB three-channel fused image; wherein, the RGB three-channel fused image = (1-α)×G channel + α×R channel, α is the weight parameter, and the weight parameters include at least a first weight parameter, a second weight parameter, and a third weight parameter, the first weight parameter is used to detect surface scratches, the second weight parameter is used to detect concave and convex deformation, and the third weight parameter is used for general detection.

5. The method according to claim 3, characterized in that After obtaining a first pixel maximum value and a first pixel minimum value of the first input image, and obtaining a second pixel maximum value and a second pixel minimum value of the second input image, the method further includes: The pixel values ​​are screened based on mathematical statistics, wherein the screening process includes but is not limited to screening by setting a screening threshold, and the screening threshold is determined based on different battery cell models.

6. The method according to claim 1, characterized in that The method further comprises: Control parameters are added when generating the target fused image; the control parameters include contrast and / or reference offset value, the contrast is used to control the degree of distinction between different colors, and the reference offset value is used to select a reference plane for plane correction.

7. The method according to claim 1, characterized in that The training set is annotated in the following way: Annotating the training set based on a dichotomy method; Alternatively, the training set is labeled based on a pre-trained model; Alternatively, the training set is labeled based on a supervised learning algorithm.

8. A battery cell appearance inspection device based on multimodal image fusion, characterized in that: The device comprises: An acquisition module is configured to acquire a first input image and a second input image; wherein the first input image is a single-channel grayscale image, and the second input image is a single-channel height image, and the number of bits of the single-channel height image is higher than the number of bits of the single-channel grayscale image; a verification module, configured to verify the validity of the first input image and the second input image, and terminate the process and output an exception code when either input image is empty or has a mismatched size; a fusion module, configured to fuse the first input image and the second input image to obtain a target fused image; A training module is used to construct a training set based on the target fusion image, and to construct a target deep learning model through the training set, and to output a cell appearance inspection result based on the target deep learning model.

9. An electronic device, characterized in that: include: A processor, a storage medium, and a bus, wherein the storage medium stores machine-readable instructions executable by the processor. When the electronic device is running, the processor and the storage medium communicate via the bus, and the processor executes the machine-readable instructions to perform the battery cell appearance detection method based on multimodal image fusion as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by the processor, the battery cell appearance detection method based on multimodal image fusion according to any one of claims 1 to 7 is executed.

Images