Thermoelectric battery assembly defect online detection method fusing visual semantics and knowledge reasoning

Abstract

This invention discloses an online detection method for hot battery assembly defects that integrates visual semantics and knowledge reasoning, belonging to the field of online hot battery assembly defect detection. The method includes real-time acquisition of hot battery stack images; dividing the hot battery stack images into tail stack assembly images, head stack assembly images, and several individual battery assembly images; segmenting the tail stack assembly images, head stack assembly images, and individual battery assembly images into sub-component images to obtain several hot battery sub-component images; extracting the assembly order of the sub-components based on the tail stack assembly images, head stack assembly images, and individual battery sub-component images; constructing a standard coding matrix and a real-time hot battery coding matrix; and identifying the type and location of assembly defects based on the real-time hot battery coding matrix and the standard coding matrix. This invention solves the problems of existing CT-based penetration detection methods, which rely on post-packaging imaging, cannot be integrated into assembly lines, suffer from delays in the detection process leading to non-repairability, have high radiation risks, and are costly.

Description

Technical Field

[0001] This invention belongs to the field of online detection of defects in thermal battery assembly, and particularly relates to an online detection method for defects in thermal battery assembly that integrates visual semantics and knowledge reasoning. Background Technology

[0002] The core of the thermal battery assembly process is to stack several individual battery cells in a specific order to form a battery pack. If cells are missing, over-assembled, or assembled in the wrong order during the assembly process, it will lead to abnormal internal electrochemical reactions, causing localized high temperatures, high pressures, and even the risk of explosion. Therefore, assembly quality inspection of thermal batteries before and after packaging is a critical step in ensuring product reliability and safety.

[0003] Existing methods for detecting assembly defects in hot batteries primarily rely on imaging analysis of the internal structure after packaging, often depending on CT or X-ray imaging techniques for non-destructive testing. For example, one existing approach proposes an assembly defect identification method based on hot battery X-ray image analysis. This method uses template matching to segment the battery stack area, employs Shi-Tomasi corner detection for target tilt correction, and designs a self-contrast defect classification method based on grayscale scan line features. Finally, it uses peak and trough information from the grayscale curve to identify and classify battery defects. Another existing approach, targeting the internal structure of hot batteries after packaging, designs an X-ray-based detection algorithm and uses a YOLOv5s deep learning model to detect typical defects. Yet another approach combines X-ray images with local features of the grayscale co-occurrence matrix, using an improved random forest algorithm to identify assembly defects such as flip-chipping, misordering, and missing current collectors. The aforementioned CT-based detection methods demonstrate some effectiveness in identifying internal structural defects in packaged batteries.

[0004] Existing methods for detecting defects in thermal batteries primarily rely on post-packaging CT imaging analysis of the battery stack to determine internal structural anomalies or assembly defects through tomographic images. While this approach can identify internal defects to some extent, it typically only identifies battery abnormalities and cannot accurately pinpoint specific assembly sequence errors or missing components, thus limiting both detection accuracy and localization capabilities. Furthermore, CT image-based defect identification suffers from long detection cycles, high system costs, and significant imaging noise interference. Moreover, the bulky size and stringent radiation safety requirements of CT inspection equipment make integration with automated thermal battery assembly lines difficult, hindering real-time online detection. Summary of the Invention

[0005] To address the aforementioned shortcomings in existing technologies, this invention provides an online detection method for hot battery assembly defects that integrates visual semantics and knowledge reasoning. This method solves the problems of existing CT-based penetration detection methods, which rely on post-packaging imaging, cannot be integrated into assembly lines, suffer from delays in the detection process leading to non-repairability, have high radiation risks, and are costly.

[0006] To achieve the aforementioned objectives, the technical solution adopted by this invention is: an online detection method for assembly defects in thermal batteries that integrates visual semantics and knowledge reasoning, comprising: Real-time acquisition of images of the thermal battery stack during the assembly process; acquisition of design knowledge semantic information; the design knowledge semantic information includes the number of components in the first thermal battery stack, the number of components in the last thermal battery stack, the number of components in the middle thermal battery stack, the color characteristics of each component of the thermal battery, and the standard thermal battery assembly sequence. Based on the semantic information of design knowledge, the thermal battery stack images are divided into tail stack component images, first stack component images and intermediate battery stack component images using visual semantic segmentation. Highlight region detection is performed on the image of the intermediate battery stack component, and segmentation is performed based on the highlight region detection results to divide the image of the intermediate battery stack component into several individual battery component images. Highlight region detection is performed on the images of the tail stack module, the first stack module, and each individual cell module. Based on the highlight region detection results, the modules are segmented to obtain several thermal cell sub-component images corresponding to each module image. Based on the images of each component and the corresponding images of each thermal battery sub-component, the actual assembly order of each component and sub-component is extracted based on the color features of each component of the thermal battery. Based on the semantic information of design knowledge, a standard coding matrix that can represent the assembly order of standard thermal batteries is constructed. A real-time coding matrix for the thermal battery is generated by utilizing the actual assembly sequence of each component and sub-component. Based on the real-time coding matrix of the thermal battery and the standard coding matrix, the assembly sequence defect type is identified.

[0007] Furthermore, the division of the thermal battery stack image into tail stack assembly image, first stack assembly image, and intermediate battery stack assembly image is specifically as follows: Based on the color characteristics of each component of the thermal battery in the semantic information of the design knowledge, the lead-out pieces in the thermal battery assembly image are located using a color recognition algorithm; and the thermal battery assembly image is divided into tail stack assembly image, first stack assembly image and intermediate battery stack assembly image by the position of each lead-out piece.

[0008] Furthermore, the process of performing highlight region detection and segmentation based on the highlight region detection results specifically includes: The image to be processed is converted to the YUV luminance space and based on the luminance channel. The grayscale distribution features are used to construct a saliency submap of the image to be processed; Pixels with a significance value greater than the significance threshold in the significance submap are marked as highlight pixels; Connectivity analysis was used to aggregate the regions of the highlight pixels, resulting in several highlight regions. Isolated noise in each highlight region is removed, and morphological operations are used to enhance the continuity of the boundaries of each highlight region, resulting in several continuous highlight regions. The image to be processed is segmented based on each continuous highlight region; the continuous highlight regions on both sides of the segmentation line are equidistant.

[0009] Furthermore, the saliency value of each pixel in the saliency sub-image is:

[0010] in, For pixels The significance value; For pixels; The brightness value is Pixels in the brightness channel Frequency of occurrence in; For pixels Except for pixels External brightness is The grayscale distance function of the pixels; For pixels Brightness value; This represents the brightness value.

[0011] Furthermore, the expression for the significance threshold is:

[0012] in, The significance threshold; For pixels The significance values ​​are mapped to integers in the interval [0, 255]. The number of pixels; For pixels; It is a set of pixels.

[0013] Furthermore, the expression for the standard coding matrix is:

[0014] in, For standard coding matrix; For the first batch of thermal battery components The assembly sequence of the first sub-component; For the first batch of thermal battery modules The assembly sequence of the second sub-component; For the first batch of thermal battery modules The Middle The assembly sequence of individual components; This refers to the assembly sequence of the first sub-component in the first unit of the intermediate battery stack. The assembly sequence of the second sub-component in the first unit of the intermediate battery stack; The first unit in the intermediate battery stack The assembly sequence of individual components; The first unit in the intermediate battery stack The assembly sequence of individual components; For the intermediate battery stack The assembly sequence of the first sub-component in a single unit; For the intermediate battery stack The assembly sequence of the second sub-component in a single unit; For the intermediate battery stack The first unit component The assembly sequence of individual components; For the intermediate battery stack The first unit component The assembly sequence of individual components; For the intermediate battery stack The assembly sequence of the first sub-component in a single unit; For the intermediate battery stack The assembly sequence of the second sub-component in a single unit; For the intermediate battery stack The first unit component The assembly sequence of individual components; For the intermediate battery stack The first unit component The assembly sequence of individual components; Tail stack components The assembly sequence of the first sub-component; Tail stack components The assembly sequence of the second sub-component; Tail stack components The Middle The assembly order of the sub-components; if the number of sub-components assembled in the current component is less than If the row number of the current component is greater than the number of sub-components assembled in the current component, then the element in the corresponding column is None; This represents the maximum number of sub-components in each individual unit of the first stack, the last stack, and the intermediate stack.

[0015] Furthermore, the column elements of the real-time encoding matrix of the thermal battery represent the component types, and the row elements represent the sub-component types, which correspond to the standard encoding matrix. If the sub-component type corresponding to the current element position does not exist in the thermal battery sub-component image corresponding to each component image, the current element position is set to 0.

[0016] Furthermore, the identification of assembly sequence defect types specifically includes: If there is an element with a value of 0 in the real-time coding matrix of the thermal battery, then the current thermal battery has a missing assembly defect. The defect location is the assembly position of the sub-component corresponding to the element with a value of 0. Real-time encoding matrix of thermal batteries The elements with a value of 0 in the matrix are updated to the corresponding elements in the standard encoding matrix, resulting in the corrected real-time encoding matrix of the thermal battery. ; If the real-time coding matrix of the thermal battery is corrected If the number of sub-components is greater than the number of sub-components in the standard coding matrix, then the current thermal battery has a multi-assembly defect. Generate a candidate matrix containing information on all redundant components. :

[0017] in, The candidate state vector for the first stack of components includes the corrected real-time encoding matrix of the thermal cells. The number of sub-component elements in the matrix exceeds the number in the standard coding matrix; The candidate state vector for intermediate battery stack components includes the corrected real-time encoding matrix of the thermal cells. The number of sub-component elements in the intermediate battery stack assembly exceeds the number of intermediate battery stack assemblies in the standard coding matrix; The candidate state vectors for the tail stack components include the corrected real-time encoding matrix of the thermal cell. The number of sub-component elements in the tail stack component is greater than the number of tail stack components in the standard coding matrix; Candidate matrix Each candidate element in the standard coding matrix Calculate sparsity by subtracting elements from corresponding labels. :

[0018] in, For the first Candidate state vectors of each component; This indicates the current component's position in the sequence. Candidate matrix The Middle One candidate element; For standard coding matrix The Middle One standard element; This is the index of the element currently participating in the matrix operation; It is the order of the norm; The total number of elements calculated for the matrix; If sparsity If the value is 0, then the component type corresponding to the current candidate element is correctly assembled; If sparsity If the value is not 0, then the assembly position of the sub-component corresponding to the current candidate element is a multi-assembly position; If there are multiple non-zero sparsity vectors, the vector with the smallest sparsity is selected as the actual position of the corresponding component, and the rest are regarded as multiple positions. The redundant elements corresponding to the multiple sub-components in the corrected real-time coding matrix of the thermal battery are deleted, and the row number of the sub-components whose row number in the column containing the redundant element is greater than that of the redundant element is reduced by 1 to obtain the adjusted real-time coding matrix of the thermal battery. The adjusted real-time coding matrix of the thermal battery is compared with the standard coding matrix. Perform element-wise subtraction to obtain a sparse matrix; If the second column of the sparse matrix is ​​the same as the first... If the sparse vector of a single-component column contains non-zero elements, then the second column and the first column... The components represented by the column have a reverse assembly defect.

[0019] The beneficial effects of this invention are as follows: This invention proposes an online defect detection method for the assembly process of thermal batteries that integrates visual semantic analysis and design knowledge reasoning. Compared with existing detection methods that rely on industrial CT imaging, it has significant innovation and practical value. Existing detection methods generally use CT or X-ray scanning to reconstruct and analyze the internal structure of packaged finished thermal batteries. This not only involves complex and time-consuming processes but also poses a risk of ionizing radiation, requiring a dedicated protective environment and making it difficult to integrate into production lines and perform real-time detection. In contrast, the method of this invention is directly based on an assembly line platform built with a high-resolution industrial camera, performing image acquisition and real-time analysis of semi-finished thermal batteries in the unpackaged stage. By introducing machine vision semantic segmentation technology and a design knowledge encoding matrix comparison mechanism, intelligent identification, location, and visualization of assembly order and component defects are achieved. Detection not only focuses on structural appearance but also reflects the correctness of assembly logic and process consistency. This cross-layer detection method from "geometric imaging" to "knowledge semantics" realizes a leap from visual recognition to intelligent assembly reasoning. At the same time, the computational load is low, achieving millisecond-level response and online parallel processing, significantly improving detection efficiency. This method requires no radiation protection facilities, is highly safe, and can be directly applied to the production site. It can also detect and report defects in real time during assembly, guiding operators to correct assembly errors promptly. This invention not only effectively reduces production costs and time losses caused by finished product scrap, but also achieves a fundamental shift in the testing process from "offline testing after packaging" to "online testing during assembly." Its testing process has advantages such as strong real-time performance, high environmental adaptability, and good integrability, providing a new, efficient, safe, and intelligent approach for the quality control of thermal battery assembly. Attached Figure Description

[0020] Figure 1 This is a flowchart of the visual semantic segmentation process for thermal batteries according to the present invention.

[0021] Figure 2 This is a flowchart for detecting defects in thermal battery assembly based on knowledge encoding matrix reasoning.

[0022] Figure 3 A schematic diagram of the design assembly structure model and knowledge encoding method.

[0023] Figure 4 This is a schematic diagram of the standard coding matrix.

[0024] Figure 5 A schematic diagram representing the sparse matrix characterization of assembly defects in various thermal batteries. Detailed Implementation

[0025] The specific embodiments of the present invention are described below to enable those skilled in the art to understand the present invention. However, it should be understood that the present invention is not limited to the scope of the specific embodiments. For those skilled in the art, various changes are obvious as long as they are within the spirit and scope of the present invention as defined and determined by the appended claims. All inventions utilizing the concept of the present invention are protected.

[0026] like Figure 1 As shown, in one embodiment of the present invention, an online detection method for hot battery assembly defects integrating visual semantics and knowledge reasoning includes: Real-time acquisition of images of the thermal battery stack during the assembly process; acquisition of design knowledge semantic information; the design knowledge semantic information includes the number of components in the first thermal battery stack, the number of components in the last thermal battery stack, the number of components in the middle thermal battery stack, the color characteristics of each component of the thermal battery, and the standard thermal battery assembly sequence. Based on the semantic information of design knowledge, the thermal battery stack images are divided into tail stack component images, first stack component images and intermediate battery stack component images using visual semantic segmentation. Highlight region detection is performed on the image of the intermediate battery stack component, and segmentation is performed based on the highlight region detection results to divide the image of the intermediate battery stack component into several individual battery component images. Highlight region detection is performed on the images of the tail stack module, the first stack module, and each individual cell module. Based on the highlight region detection results, the modules are segmented to obtain several thermal cell sub-component images corresponding to each module image. Based on the images of each component and the corresponding images of each thermal battery sub-component, the actual assembly order of each component and sub-component is extracted based on the color features of each component of the thermal battery. Based on the semantic information of design knowledge, a standard coding matrix that can represent the assembly order of standard thermal batteries is constructed. A real-time coding matrix for the thermal battery is generated by utilizing the actual assembly sequence of each component and sub-component. Based on the real-time coding matrix of the thermal battery and the standard coding matrix, the assembly sequence defect type is identified.

[0027] In this embodiment, the present invention mainly includes two stages: (1) Visual semantic segmentation of thermal battery images; (2) Assembly defect detection based on knowledge encoding. First, the acquired thermal battery assembly images are input into an adaptive visual segmentation algorithm to accurately extract the key areas of the battery stack and its structure, and obtain the image regions and spatial location information of each sub-component in real time. At the same time, the design assembly sequence and component spatial layout of the thermal battery are encoded and stored in the form of a position matrix. Subsequently, based on the semantic segmentation results, the identified thermal battery sub-components are further mapped to sub-components corresponding to the design assembly structure model, and the actual assembly sequence of each sub-component is extracted. A real-time knowledge encoding matrix is ​​constructed based on the assembly sequence information, and a standard encoding matrix is ​​constructed based on the design knowledge semantic information in the thermal battery design assembly structure model. By performing corresponding calculations between the real-time encoding matrix and the standard encoding matrix, a sparse matrix that can characterize assembly differences is generated, and the sparse matrix results are analyzed to finally achieve automatic identification and accurate positioning of thermal battery assembly sequence defects.

[0028] The process of dividing the thermal battery stack images into tail stack assembly images, first stack assembly images, and intermediate battery stack assembly images is as follows: Based on the color characteristics of each component of the thermal battery in the semantic information of the design knowledge, the lead-out pieces in the thermal battery assembly image are located using a color recognition algorithm; and the thermal battery assembly image is divided into tail stack assembly image, first stack assembly image and intermediate battery stack assembly image by the position of each lead-out piece.

[0029] In this embodiment, visual semantic segmentation of the thermal battery is a key step in obtaining the assembly location and assembly order features of its internal components. The overall process is as follows: Figure 1 As shown, the process mainly consists of two steps: thermal battery sub-assembly classification and thermal battery sub-component segmentation. The complete thermal battery is divided into multiple sub-assemblies, and each sub-assembly is further subdivided into independent sub-components. In the "thermal battery sub-assembly classification" stage, component classification is guided by semantic information based on design knowledge. This information guides the component division and individual unit identification in the visual segmentation stage, ensuring that the entire segmentation process is executed according to the thermal battery's design assembly structure model, dividing the thermal battery into first and last stack assemblies and intermediate battery stack sub-assemblies.

[0030] Figure 1 The “Design Assembly Structure Model (CAD)” refers to the structured expression based on the CAD assembly model of the thermal battery, which is used to describe the identification category of all sheet components of the thermal battery, the color characteristics of the components, the design order of the sheets in the battery stack, and the vertical relationship of adjacent sheet components.

[0031] Figure 1The “design knowledge semantic information” is a structural prior extracted from the design assembly structure model (CAD) that can be used for image segmentation, including the first / last stack discrimination rules, the number of sheet components that should be included in the intermediate battery stack, color information, sheet component arrangement characteristics, etc.

[0032] In the "thermal battery sub-component segmentation" stage, based on the salience of the highlight areas of the component features, it is divided into two sub-processes: salience component segmentation and non-salience component segmentation. This enables further segmentation of the internal structure of the thermal battery and extraction of the actual assembly sequence knowledge of the corresponding structure, providing a basis for subsequent assembly defect detection based on knowledge reasoning.

[0033] In this embodiment, by analyzing the design structural features of the thermal battery, its overall structure can be composed of a head and tail stack assembly and a middle battery stack assembly, and the middle battery stack assembly can be further divided into multiple individual components. Based on design knowledge semantic information, this invention uses the lead-out sheet as the dividing line between the head and tail stacks and the middle battery stack assembly. Because its shape and color characteristics are significantly different from the surrounding components, it can serve as an important basis for achieving precise segmentation of sub-components.

[0034] In the first and last stack assemblies, due to the significant color differences among the individual sheet components, the location and feature knowledge of their internal parts can be identified and extracted based on the assembly prior order information and color characteristics provided by the semantic information of the design knowledge. The specific extraction process includes: The assembly prior order information provided by the design knowledge semantic information includes the standard layer number and relative order of each sheet element in the design structure of the thermal battery. During component-level segmentation, this invention first uses a color recognition algorithm to locate and mark the individual heating elements in each sheet element; then, the sheet element image is grayscaled, and a grayscale nonlinear transformation is applied to enhance its color contrast, thereby extracting the assembly position and order features of each component. Finally, it is matched with the theoretical layer number range of the corresponding component in the prior order information to determine the structural level to which the sheet belongs, i.e., the first stack, the middle stack, or the last stack, and this guides the subsequent sub-component identification process, ensuring that the segmentation result conforms to the thermal battery design and assembly logic.

[0035] The process of detecting highlight areas and segmenting them based on the detection results is as follows: The image to be processed is converted to the YUV luminance space and based on the luminance channel. The grayscale distribution features are used to construct a saliency submap of the image to be processed; Pixels with a significance value greater than the significance threshold in the significance submap are marked as highlight pixels; Connectivity analysis was used to aggregate the regions of the highlight pixels, resulting in several highlight regions. Isolated noise in each highlight region is removed, and morphological operations are used to enhance the continuity of the boundaries of each highlight region, resulting in several continuous highlight regions. The image to be processed is segmented based on each continuous highlight region; the continuous highlight regions on both sides of the segmentation line are equidistant.

[0036] The saliency value of each pixel in the saliency sub-image is:

[0037] in, For pixels The significance value; For pixels; The brightness value is Pixels in the brightness channel Frequency of occurrence in; For pixels Except for pixels External brightness is The grayscale distance function of the pixels; For pixels Brightness value; This represents the brightness value.

[0038] The expression for the significance threshold is:

[0039] in, The significance threshold; For pixels The significance values ​​are mapped to integers in the interval [0, 255]. The number of pixels; For pixels; It is a set of pixels.

[0040] In this embodiment, for the segmentation of the internal structure of the intermediate battery stack assembly, since it is composed of multiple stacked units, it is necessary to further identify the axial boundaries of each unit. Considering that the single heating element in the unit exhibits obvious strong reflective characteristics under light source illumination, this specular highlight characteristic makes it stand out sharply from adjacent components. However, the color and texture information of the saturated areas in the specular highlight image will be destroyed or even completely lost. To address this problem, this invention proposes to apply the saliency model to a luminance space separate from the chromaticity space, thereby achieving the boundary localization of the thermal battery element. This method can overcome the problem that the saliency model itself is sensitive to color values, and can more specifically and accurately detect the specular highlight areas in the image, thus achieving accurate segmentation of the battery stack components.

[0041] The saliency detection method first converts the thermal battery image from the RGB color space to the YUV luminance space. This invention constructs a saliency sub-map based on the grayscale distribution characteristics of the luminance channel Y. Specifically, it calculates the saliency of pixels... The cumulative grayscale difference between the pixel and the brightness values ​​of all pixels in the entire image is used to characterize the pixel. The saliency of pixels. significance value It can be defined as the sum of the differences between its brightness value and the brightness values ​​of other pixels.

[0042] In this embodiment, after obtaining the saliency value and identifying the highlight pixels, the present invention further employs connected component analysis to perform region aggregation on the highlight pixels. By performing neighborhood connected component detection on pixels with saliency values ​​exceeding a threshold, adjacent highlight pixels are merged into highlight regions. Subsequently, the highlight regions are filtered using geometric features such as region area, aspect ratio, and positional distribution to remove isolated noise points, and morphological operations are used to enhance the continuity of region boundaries.

[0043] For the selected highlight regions, their circumscribed rectangles or central axes are calculated and sorted according to the vertical axis of the image. Since each individual battery cell contains a heating element with stable highlight reflection characteristics, the vertical distance between consecutive highlight regions can be used as the basis for sub-component segmentation. Based on the sorted highlight region boundaries, this invention divides the intermediate battery stack into multiple sub-components and obtains the precise spatial location range of each component, realizing the transformation process from pixel-level saliency detection to thermal battery structure-level component segmentation. Through this process, precise location information of the intermediate battery stack sub-components can be obtained.

[0044] Based on the above steps, a hierarchical segmentation process for thermal batteries was established: first, the overall thermal battery is classified into sub-modules, then the modules are further divided into sub-components, and finally the assembly position and assembly sequence features of the sub-components are extracted.

[0045] like Figure 4 As shown, the expression for the standard coding matrix is:

[0046] in, For standard coding matrix; For the first batch of thermal battery modules The assembly sequence of the first sub-component; For the first batch of thermal battery modules The assembly sequence of the second sub-component; For the first batch of thermal battery modules The Middle The assembly sequence of individual components; This refers to the assembly sequence of the first sub-component in the first unit of the intermediate battery stack. The assembly sequence of the second sub-component in the first unit of the intermediate battery stack; The first unit in the intermediate battery stack The assembly sequence of individual components; The first unit in the intermediate battery stack The assembly sequence of individual components; For the intermediate battery stack The assembly sequence of the first sub-component in a single unit; For the intermediate battery stack The assembly sequence of the second sub-component in a single unit; For the intermediate battery stack The first unit component The assembly sequence of individual components; For the intermediate battery stack The first unit component The assembly sequence of individual components; For the intermediate battery stack The assembly sequence of the first sub-component in a single unit; For the intermediate battery stack The assembly sequence of the second sub-component in a single unit; For the intermediate battery stack The first unit component The assembly sequence of individual components; For the intermediate battery stack The first unit component The assembly sequence of individual components; Tail stack components The assembly sequence of the first sub-component; Tail stack components The assembly sequence of the second sub-component; Tail stack components The Middle The assembly order of the sub-components; if the number of sub-components assembled in the current component is less than If the row number of the current component is greater than the number of sub-components assembled in the current component, then the element in the corresponding column is None; This represents the maximum number of sub-components in each individual unit of the first stack, the last stack, and the intermediate stack.

[0047] The column elements of the real-time encoding matrix of the thermal battery represent the component types, and the row elements represent the sub-component types, which correspond to the standard encoding matrix. If the sub-component type corresponding to the current element position does not exist in the thermal battery sub-component image corresponding to each component image, the current element position is set to 0.

[0048] In this embodiment, the obtained "sub-component assembly position and assembly sequence features" are used as the original feature data. These features are structured and stored according to the coding rules in the design assembly structure model to form a real-time knowledge coding matrix that can be compared with the standard design assembly structure model. The elements in the matrix represent entities or attributes, and adjacent elements are arranged according to the relationship between elements or the semantic association between entities.

[0049] Building upon the feature extraction and segmentation of thermal battery sub-components, this invention proposes a method for detecting assembly order defects based on a knowledge encoding matrix to further detect assembly defects at the assembly level. The process is as follows: Figure 2 As shown, firstly, based on the prior assembly knowledge provided by the semantic information of the design knowledge, a standard encoding matrix that can represent the assembly sequence of a standard thermal battery is constructed. Then, the assembly sequence features of each component are obtained from the real-time acquired thermal battery images using a feature extraction algorithm, generating the corresponding real-time encoding matrix. By calculating the difference between the real-time encoding matrix and the standard encoding matrix, a sparse matrix is ​​generated. Then, based on the calculation results of the sparse matrix, the type and location of assembly sequence defects are identified, achieving precise defect localization.

[0050] The overall structure of a thermal battery and its corresponding coding representation method are as follows: Figure 3 As shown, a thermal battery can be structurally divided into three main parts: the initial stack, the final stack, and the intermediate stack. The intermediate stack consists of multiple individual modules arranged in sequence. Each individual module contains different types of components assembled in different orders.

[0051] The knowledge encoding matrix method in this invention corresponds to the thermal battery design and assembly structure model, dividing the thermal battery into a first-stage assembly, a last-stage assembly, and multiple individual assemblies, each encoded separately. Therefore, the standard assembly sequence characteristics of the thermal battery can be represented by a standard encoding matrix. express.

[0052] Here, the dimension of the column vector equals the number of component types contained in the component. Each element in the column vector stores the positional order label of a certain type of component within the component, with the label representing its order in the assembly sequence in a regular form. If a component appears in multiple positions within the component, its positional labels are connected by an '&' symbol. The column vectors of all components are arranged according to the assembly order relationship defined by the semantic information of standard thermal cell design knowledge, together forming a standard encoding matrix. To maintain consistency in the dimension of column vectors across different components, column vectors with lower dimensions are filled with "None".

[0053] The real-time encoding matrix of the thermal battery also consists of three parts: the state vector of the first stack of components. Intermediate battery stack component state vector and tail heap component state vector The real-time encoding matrix corresponds structurally to the standard encoding matrix and is used to record the positional order information of each component extracted through features. Each element in this matrix is ​​arranged according to the coordinate order of the component in the position matrix. If a component does not contain a certain type of component defined by the state vector, the label of the corresponding matrix position is set to 0 to indicate that the component is missing.

[0054] The identification of assembly sequence defect types specifically includes: If there is an element with a value of 0 in the real-time coding matrix of the thermal battery, then the current thermal battery has a missing assembly defect. The defect location is the assembly position of the sub-component corresponding to the element with a value of 0. Real-time encoding matrix of thermal batteries The elements with a value of 0 in the matrix are updated to the corresponding elements in the standard encoding matrix, resulting in the corrected real-time encoding matrix of the thermal battery. ; If the real-time coding matrix of the thermal battery is corrected If the number of sub-components is greater than the number of sub-components in the standard coding matrix, then the current thermal battery has a multi-assembly defect. Generate a candidate matrix containing information on all redundant components. :

[0055] in, The candidate state vector for the first stack of components includes the corrected real-time encoding matrix of the thermal cells. The number of sub-component elements in the matrix exceeds the number in the standard coding matrix; The candidate state vector for intermediate battery stack components includes the corrected real-time encoding matrix of the thermal cells. The number of sub-component elements in the intermediate battery stack assembly exceeds the number of intermediate battery stack assemblies in the standard coding matrix; The candidate state vectors for the tail stack components include the corrected real-time encoding matrix of the thermal cell. The number of sub-component elements in the tail stack component is greater than the number of tail stack components in the standard coding matrix; Candidate matrix Each candidate element in the standard coding matrix Calculate sparsity by subtracting elements from corresponding labels. :

[0056] in, For the first Candidate state vectors of each component; This indicates the current component's position in the sequence. Candidate matrix The Middle One candidate element; For standard coding matrix The Middle One standard element; This is the index of the element currently participating in the matrix operation; It is the order of the norm; The total number of elements calculated for the matrix; If sparsity If the value is 0, then the component type corresponding to the current candidate element is correctly assembled; If sparsity If the value is not 0, then the assembly position of the sub-component corresponding to the current candidate element is a multi-assembly position; If there are multiple non-zero sparsity vectors, the vector with the smallest sparsity is selected as the actual position of the corresponding component, and the rest are regarded as multiple positions. The redundant elements corresponding to the multiple sub-components in the corrected real-time coding matrix of the thermal battery are deleted, and the row number of the sub-components whose row number in the column containing the redundant element is greater than that of the redundant element is reduced by 1 to obtain the adjusted real-time coding matrix of the thermal battery. The adjusted real-time coding matrix of the thermal battery is compared with the standard coding matrix. Perform element-wise subtraction to obtain a sparse matrix; If the second column of the sparse matrix is ​​the same as the first... If the sparse vector of a single-component column contains non-zero elements, then the second column and the first column... The components represented by the column have a reverse assembly defect.

[0057] In this embodiment, after obtaining the assembly position and assembly sequence characteristics of each component of the thermal battery, the present invention constructs a real-time knowledge coding matrix based on these characteristics according to the coding rules of the design knowledge model. This is used to characterize the actual assembly sequence of the current thermal battery. Subsequently, the real-time encoding matrix is ​​compared with the standard encoding matrix corresponding to standard design knowledge, and a sparse matrix is ​​generated based on the differences to achieve automatic identification, classification, and location of assembly defects. The core detection process is as follows: (1) Detection of missing parts In real-time encoding matrix In this process, each component is represented by an assembly order label (1, 2, 3…n) indicating its design position within a single component. If a component's state vector contains a label with a value of 0, it indicates that no component is matched at that position, and this is classified as a "missing component defect." Subsequently, the element with a label of 0 in the real-time coding matrix is ​​located, and its label is corrected to the standard label value of the corresponding component in the standard coding matrix, ensuring that missing components are filled in. The component position number to be retained is marked with a superscript *. For other components within the same component whose label values ​​are greater than their corresponding standard label values, all their label values ​​are incremented by 1 to restore the correct order structure. This yields the real-time coding matrix after the missing component defect has been corrected. This is used for subsequent defect assessment.

[0058] (2) Defect detection of multiple units Multiple defects are identified by comparing real-time encoding matrices. With standard coding matrix The number of component tags for the corresponding component is used to determine the presence of a "multiple-item defect". If the number of tags in the real-time matrix exceeds the expected number in the standard matrix, a "multiple-item defect" is identified.

[0059] If sparsity If the calculated value is 0, the candidate label matches the standard position and belongs to the correct assembly position. If it is not 0, it indicates that the label is "multiple assembly", and the multiple assembly component type is marked with a superscript *. If there are multiple non-zero sparsity vectors, the vector with the smallest sparsity is selected as the actual position of the component, and the rest are considered as multiple assemblies. Finally, the redundant labels of the multiple assembly components are deleted, and other components with indices greater than the label are decremented by 1 to maintain the integrity and consistency of the sequence.

[0060] (3) Detection of defects in reverse assembly of components The method of this invention involves correcting the real-time matrix. With standard matrix Element-wise subtraction is performed to obtain the difference between the state vectors of each component, and then a sparse matrix is ​​constructed. This matrix clearly characterizes the deviation between the actual assembly sequence and the designed assembly structure model, and its structure is as follows: Figure 5 As shown.

[0061] In a sparse matrix, if the second column is the same as the first column... If the sparse vectors of two individual components simultaneously contain non-zero elements, it indicates an assembly order anomaly, specifically a reverse assembly defect. If the battery stack has both missing and extra components, this invention will mark the defect locations and locate the extra components as anomaly points based on the states obtained from the aforementioned missing and extra component correction steps. Finally, this invention combines the sparse matrix with the component position matrix to map the actual location of the defective component back to the image of the thermal battery, directly marking the defective component in the digital image, thereby achieving automatic identification of assembly defects, automatic classification of defect types, and precise location of defective components in the image.

[0062] In this embodiment, the present invention extracts the spatial location and structural features of each component inside the thermal battery through visual semantic segmentation technology during the assembly stage. It further combines this with thermal battery assembly design knowledge to deduce the assembly sequence and construct a corresponding real-time knowledge matrix. This knowledge matrix is ​​compared with a standard assembly knowledge matrix generated based on a CAD design model to obtain a sparse matrix for identifying abnormal assembly states and potential defects. Finally, the system maps the defect analysis results back to the original image of the thermal battery, achieving an intuitive conversion from image segmentation results to knowledge-encoded information, thereby enabling high-precision, visualized defect location and judgment.

[0063] Compared to existing CT imaging-based inspection methods, this method eliminates the need for post-packaging inspection of the battery stack, enabling defect identification at the front end of the assembly process. This significantly shortens the inspection cycle and reduces costs. Traditional CT methods suffer from drawbacks such as complex image reconstruction, long inspection cycles, and inability to be integrated into assembly lines. In contrast, this invention achieves online inspection and intelligent decision-making through the fusion of visual semantics and knowledge encoding, overcoming the integration bottlenecks of traditional methods.

[0064] This invention proposes a novel approach to visualize assembly sequence using a knowledge encoding matrix. By representing the assembly status, position, and assembly sequence information of the thermal battery assembly in the form of an encoding matrix, the system can directly reflect assembly anomalies, such as missing components, misalignment, or incorrect assembly sequence. If an anomaly exists at a certain assembly location, this information will be encoded and visualized in real time, simultaneously mapped onto a real image of the thermal battery. This allows inspection personnel to intuitively perceive the location and type of defects, greatly improving the accuracy and response speed of inspection.

[0065] Compared with traditional image recognition or defect detection methods, this invention does not only stop at the appearance difference recognition at the image level, but also realizes the analysis and judgment of assembly logic based on knowledge reasoning, thus possessing a higher semantic level and intelligence level in defect detection.

[0066] In terms of inspection efficiency and cost control, this invention fully utilizes the fusion characteristics of machine vision and knowledge reasoning to achieve online real-time detection of defects in hot battery assembly, eliminating the need for additional complex inspection equipment and post-packaging processing environments. This method is suitable for production line integration, significantly reducing equipment costs and inspection latency, and improving the automation and intelligence level of the production process.

[0067] Compared to existing technologies that rely on expensive CT equipment and post-processing algorithms, this invention significantly improves detection speed and economy while maintaining detection accuracy, providing a widely applicable intelligent quality control solution for the hot battery assembly process.

[0068] This invention achieves automatic identification and intelligent analysis of the assembly sequence of thermal batteries, and directly projects the detection results onto the original thermal battery image through a knowledge matrix mapping mechanism, thereby providing assembly personnel with real-time and intuitive defect visualization prompts. This method not only improves the timeliness and accuracy of defect detection and correction, but also enables the assembly production line to possess self-sensing, self-diagnostic, and self-optimization capabilities. It is an innovative solution for thermal battery detection technology in intelligent manufacturing scenarios, specifically manifested as follows: (1) The detection stage and the system integration method are different.

[0069] Current technologies primarily rely on CT scans of the encapsulated thermal battery stack to reconstruct its internal structure and determine the presence of defects. This method is time-consuming, costly, and cannot be integrated into thermal battery assembly lines, limiting its application to offline inspection.

[0070] This invention performs defect detection at the front end of the thermal battery assembly process. It uses a high-resolution industrial camera to capture images of the assembly process and combines machine vision and knowledge encoding technologies to achieve online identification and analysis. This method can be embedded in the assembly line to achieve real-time detection, immediate feedback, and dynamic correction, overcoming the limitations of traditional CT inspection which cannot be integrated online.

[0071] (2) The detection principle and information expression method are different.

[0072] Traditional CT scans rely on physical imaging and density reconstruction algorithms, primarily identifying internal structural anomalies in materials, but are powerless to detect cognitive defects such as assembly logic and assembly sequence.

[0073] This invention extracts image features through machine vision semantic segmentation, then transforms the recognition results into an assembly knowledge encoding matrix, and analyzes the assembly order and component matching relationship through knowledge reasoning. The detection not only focuses on structural appearance but also reflects the correctness of assembly logic and process consistency. This cross-layer detection method, from "geometric imaging" to "knowledge semantics," achieves a leap from visual recognition to intelligent assembly reasoning.

[0074] (3) The expression and visualization of defects are different.

[0075] Current CT scan results require complex processing such as image reconstruction and tomographic analysis, resulting in defects that are not readily apparent and are difficult to locate, often requiring manual comparison and judgment by professionals.

[0076] This invention employs a two-way mapping mechanism between a knowledge matrix and the original image, directly visualizing the defect detection results in the form of an encoded matrix and simultaneously mapping them onto the original image of the thermal battery. Inspectors can intuitively see the specific location, type, and corresponding assembly unit of the assembly defect, achieving an integrated and intuitive presentation of "detection-identification-location."

[0077] (4) The detection efficiency and applicability are different.

[0078] Traditional CT scans are complex, computationally intensive, and time-consuming, making them unsuitable for mass production and online quality control.

[0079] The detection algorithm of this invention is based on semantic segmentation and knowledge matrix comparison, which has low computational cost, can achieve millisecond-level response and online parallel processing, and significantly improves detection efficiency. Furthermore, this solution has low requirements for equipment environment and can be directly applied to the production site, achieving low-cost and scalable online detection.

Claims

1. An online detection method for assembly defects in thermal batteries that integrates visual semantics and knowledge reasoning, characterized in that, include: Real-time acquisition of images of the thermal battery stack during the assembly process; Obtain semantic information about design knowledge; The design knowledge semantic information includes the number of components in the first thermal battery stack, the number of components in the last thermal battery stack, the number of components in the middle thermal battery stack, the color characteristics of each component of the thermal battery, and the standard thermal battery assembly order. Based on the semantic information of design knowledge, the thermal battery stack images are divided into tail stack component images, first stack component images and intermediate battery stack component images using visual semantic segmentation. Highlight region detection is performed on the image of the intermediate battery stack component, and segmentation is performed based on the highlight region detection results to divide the image of the intermediate battery stack component into several individual battery component images. Highlight region detection is performed on the images of the tail stack module, the first stack module, and each individual cell module. Based on the highlight region detection results, the modules are segmented to obtain several thermal cell sub-component images corresponding to each module image. Based on the images of each component and the corresponding images of each thermal battery sub-component, the actual assembly order of each component and sub-component is extracted based on the color features of each component of the thermal battery. Based on the semantic information of design knowledge, a standard coding matrix that can represent the assembly order of standard thermal batteries is constructed. A real-time coding matrix for the thermal battery is generated by utilizing the actual assembly sequence of each component and sub-component. Based on the real-time coding matrix of the thermal battery and the standard coding matrix, the assembly sequence defect type is identified.

2. The online detection method for hot battery assembly defects integrating visual semantics and knowledge reasoning according to claim 1, characterized in that, The process of dividing the thermal battery stack images into tail stack assembly images, first stack assembly images, and intermediate battery stack assembly images is as follows: Based on the color characteristics of each component of the thermal battery in the semantic information of the design knowledge, the lead-out pieces in the thermal battery assembly image are located using a color recognition algorithm; and the thermal battery assembly image is divided into tail stack assembly image, first stack assembly image and intermediate battery stack assembly image by the position of each lead-out piece.

3. The online detection method for hot battery assembly defects integrating visual semantics and knowledge reasoning according to claim 1, characterized in that, The process of detecting highlight areas and segmenting them based on the detection results is as follows: The image to be processed is converted to the YUV luminance space and based on the luminance channel. The grayscale distribution features are used to construct a saliency submap of the image to be processed; Pixels with a significance value greater than the significance threshold in the significance submap are marked as highlight pixels; Connectivity analysis was used to aggregate the regions of the highlight pixels, resulting in several highlight regions. Isolated noise in each highlight region is removed, and morphological operations are used to enhance the continuity of the boundaries of each highlight region, resulting in several continuous highlight regions. The image to be processed is segmented based on each continuous highlight region; the continuous highlight regions on both sides of the segmentation line are equidistant.

4. The online detection method for hot battery assembly defects integrating visual semantics and knowledge reasoning according to claim 3, characterized in that, The saliency value of each pixel in the saliency sub-image is: in, For pixels The significance value; For pixels; The brightness value The pixels in the brightness channel Frequency of occurrence in; For pixels Except for pixels External brightness is The grayscale distance function of the pixels; For pixels The brightness value; This represents the brightness value.

5. The online detection method for hot battery assembly defects integrating visual semantics and knowledge reasoning according to claim 3, characterized in that, The expression for the significance threshold is: in, The significance threshold; For pixels The significance values ​​are mapped to integers in the interval [0, 255]. The number of pixels; For pixels; It is a set of pixels.

6. The online detection method for hot battery assembly defects integrating visual semantics and knowledge reasoning according to claim 1, characterized in that, The expression for the standard coding matrix is: in, For standard coding matrix; For the first batch of thermal battery components The assembly sequence of the first sub-component; For the first batch of thermal battery components The assembly sequence of the second sub-component; For the first batch of thermal battery components The Middle The assembly sequence of individual components; This refers to the assembly sequence of the first sub-component in the first unit of the intermediate battery stack. The assembly sequence of the second sub-component in the first unit of the intermediate battery stack; The first unit in the intermediate battery stack The assembly sequence of individual components; The first unit in the intermediate battery stack The assembly sequence of individual components; For the intermediate battery stack The assembly sequence of the first sub-component in a single unit; For the intermediate battery stack The assembly sequence of the second sub-component in a single unit; For the intermediate battery stack The first unit component The assembly sequence of individual components; For the intermediate battery stack The first unit component The assembly sequence of individual components; For the intermediate battery stack The assembly sequence of the first sub-component in a single unit; For the intermediate battery stack The assembly sequence of the second sub-component in a single unit; For the intermediate battery stack The first unit component The assembly sequence of individual components; For the intermediate battery stack The first unit component The assembly sequence of individual components; Tail stack components The assembly sequence of the first sub-component; Tail stack components The assembly sequence of the second sub-component; Tail stack components The Middle The assembly order of the sub-components; if the number of sub-components assembled in the current component is less than If the row number of the current component is greater than the number of sub-components assembled in the current component, then the element in the corresponding column is None; This represents the maximum number of sub-components in each individual unit of the first stack, the last stack, and the intermediate stack.

7. The online detection method for hot battery assembly defects integrating visual semantics and knowledge reasoning according to claim 1, characterized in that, The column elements of the real-time encoding matrix of the thermal battery represent the component types, and the row elements represent the sub-component types, which correspond to the standard encoding matrix. If the sub-component type corresponding to the current element position does not exist in the thermal battery sub-component image corresponding to each component image, the current element position is set to 0.

8. The online detection method for hot battery assembly defects integrating visual semantics and knowledge reasoning according to claim 1, characterized in that, The identification of assembly sequence defect types specifically includes: If there is an element with a value of 0 in the real-time coding matrix of the thermal battery, then the current thermal battery has a missing assembly defect. The defect location is the assembly position of the sub-component corresponding to the element with a value of 0. Real-time encoding matrix of thermal batteries The elements with a value of 0 in the matrix are updated to the corresponding elements in the standard encoding matrix, resulting in the corrected real-time encoding matrix of the thermal battery. ; If the real-time coding matrix of the thermal battery is corrected If the number of sub-components is greater than the number of sub-components in the standard coding matrix, then the current thermal battery has a multi-assembly defect. Generate a candidate matrix containing information on all redundant components. : in, The candidate state vector for the first stack of components includes the corrected real-time encoding matrix of the thermal cells. The number of sub-component elements in the matrix exceeds the number in the standard coding matrix; The candidate state vector for intermediate battery stack components includes the corrected real-time encoding matrix of the thermal cells. The number of sub-component elements in the intermediate battery stack assembly exceeds the number of intermediate battery stack assemblies in the standard coding matrix; The candidate state vectors for the tail stack components include the corrected real-time encoding matrix of the thermal cell. The number of sub-component elements in the tail stack component is greater than the number of tail stack components in the standard coding matrix; Candidate matrix Each candidate element in the standard coding matrix Calculate sparsity by subtracting elements from corresponding labels. : in, For the first Candidate state vectors of each component; This indicates the current component's position in the sequence. Candidate matrix The Middle One candidate element; For standard coding matrix The Middle One standard element; This is the index of the element currently participating in the matrix operation; It is the order of the norm; The total number of elements calculated for the matrix; If sparsity If the value is 0, then the component type corresponding to the current candidate element is correctly assembled; If sparsity If the value is not 0, then the assembly position of the sub-component corresponding to the current candidate element is a multi-assembly position; If there are multiple non-zero sparsity vectors, the vector with the smallest sparsity is selected as the actual position of the corresponding component, and the rest are regarded as multiple positions. The redundant elements corresponding to the multiple sub-components in the corrected real-time coding matrix of the thermal battery are deleted, and the row number of the sub-components whose row number in the column containing the redundant element is greater than that of the redundant element is reduced by 1 to obtain the adjusted real-time coding matrix of the thermal battery. The adjusted real-time coding matrix of the thermal battery is compared with the standard coding matrix. Perform element-wise subtraction to obtain a sparse matrix; If the second column of the sparse matrix is ​​the same as the first... If the sparse vector of a single-component column contains non-zero elements, then the second column and the first column... The components represented by the column have a reverse assembly defect.

Images