Battery pack appearance detection method, device and system

Abstract

A battery pack appearance detection method, device and system applied to the computer vision technical field, the detection device comprises: an image acquisition unit configured to acquire an image of a first battery pack to be detected, the image presenting a partial or whole outer surface of the first battery pack; a processing unit configured to input the image into a detection model corresponding to a second battery pack to obtain a detection result for the first battery pack; wherein a similarity between the appearances of the first battery pack and the second battery pack is greater than or equal to a similarity threshold, the detection model corresponding to the second battery pack is used to detect a first type of defect; and the first detection result comprises at least one of the following: an indication of whether the first type of defect exists on the outer surface of the first battery pack, and an indication of a position of the first type of defect on the outer surface of the first battery pack. The scheme provided in the application can also achieve detection of a certain battery pack through a detection model corresponding to a battery pack similar to the certain battery pack when the detection model is not trained for the certain battery pack.

Description

Technical Field

[0001] This application relates to the field of computer vision technology, and more specifically, to a method, apparatus, and system for inspecting the appearance of a battery pack. Background Technology

[0002] With the development of optical hardware and artificial intelligence technologies, the technology of combining machine vision hardware and algorithm software systems has been further developed and applied; however, appearance inspection has always been a challenge in the vision industry due to interference from factors such as lighting, surface reflection, and scene.

[0003] In the current technological context, the deployment and use of visual inspection applications typically involve the following process: image collection, image annotation, detection model construction, detection model evaluation, deployment, and application. Real-world scenarios include a wide variety of visual inspection scenarios, such as the appearance inspection of various battery packs. For each type of battery pack, the above process must be repeated, resulting in high time and manpower costs for visual inspection from training to application. Summary of the Invention

[0004] This application provides a battery pack appearance inspection method, apparatus, and system. When the detection model has not been trained for the first battery pack, the first battery pack can still be detected by using the detection model corresponding to a battery pack similar to the first battery pack. This can improve the applicability of the detection method in different detection scenarios, eliminate the need for extensive offline image annotation, and help reduce the time and manpower costs required for the detection method from training to application.

[0005] In a first aspect, a battery pack appearance inspection device is provided, comprising: an image acquisition unit for acquiring an image of a first battery pack to be inspected, the image showing part or all of the outer surface of the first battery pack; a processing unit for inputting the image into a similarity model, determining information of a second battery pack from data of multiple battery packs in a training library, and determining a detection model corresponding to the second battery pack from detection models corresponding to the multiple battery packs based on the information of the second battery pack, the data of the multiple battery packs being used to train the detection models corresponding to the multiple battery packs; the processing unit is further configured to input the image into the detection model corresponding to the second battery pack to obtain a first detection result for the first battery pack; wherein the similarity between the appearance of the first battery pack and the second battery pack is greater than or equal to a similarity threshold, and the detection model corresponding to the second battery pack is used to detect a first type of defect; the first detection result includes at least one of the following: indicating whether the first type of defect exists on the outer surface of the first battery pack, and indicating the location of the first type of defect on the outer surface of the first battery pack.

[0006] In the above technical solution, the detection model can be transferred based on the similarity of the battery pack appearance. This improves the applicability of the battery pack appearance detection device in different battery pack detection scenarios, eliminating the need to repeat the offline image annotation process for each battery pack. This helps reduce the time and manpower costs required for training and applying the model used in the battery pack appearance detection device. Through detection model transfer, the appearance detection of similar battery packs can be achieved using the detection model corresponding to existing battery packs, improving the compatibility of the battery pack appearance detection device with the appearance detection of different battery packs.

[0007] It should be noted that the detection model corresponding to the second battery pack can be a detection model used to detect the appearance of multiple battery packs, including the second battery pack, or it can be a detection model among multiple detection models in the detection device used to detect the second battery pack.

[0008] For example, the first battery pack and the second battery pack can be battery packs with the same appearance but different sizes; or, the first battery pack and the second battery pack can be battery packs with similar appearance but different sizes; or, the first battery pack and the second battery pack can be battery packs with partially similar appearances.

[0009] For example, the first type of defect may include at least one of the following: dent, bulge, membrane rupture, screw defect (such as missing screw, screw not tightened, etc.).

[0010] For example, the similarity threshold can be 90%, or 95%, or other values.

[0011] In some possible implementations, the image of the first battery pack can be transformed by matrix transformation to convert the image of the first battery pack into a form compatible with the detection model corresponding to the second battery pack, thereby enabling the detection of the first battery pack through the detection model corresponding to the second battery pack.

[0012] In conjunction with the first aspect, in some implementations of the first aspect, the processing unit is also used for:

[0013] For example, the similarity model can be the Fréchet inception distance (FID) model, or it can be the structural similarity (SSM) model, or it can be other models.

[0014] For example, the similarity threshold mentioned above can be determined based on the specific similarity model used.

[0015] For example, the second battery pack can be the battery pack with the highest similarity to the first battery pack among a variety of battery packs, or it can be one of the battery packs with a similarity to the first battery pack that is greater than or equal to a similarity threshold.

[0016] In some possible implementations, the data for multiple battery packs in the training library does not include the data for the first battery pack.

[0017] In conjunction with the first aspect, in some implementations of the first aspect, the device communicates with the display device, and the processing unit is further configured to: when the first detection result indicates that the first type of defect exists on the outer surface of the first battery pack, input the first detection result into the defect location marking model to obtain a first mark indicating the location of the first type of defect on the outer surface of the first battery pack; and control the display device to display the first mark based on the image.

[0018] In one example, the battery pack appearance inspection device and the display device are located in the same entity, such as in an interactive device. The interactive device can display an image of the first battery pack to be inspected, and can also use the image to complete the inspection and marking of the battery pack to be inspected and display the inspection results and / or markings (such as the first marking).

[0019] In another example, the battery pack appearance inspection device and the display device are two different entities. For example, the battery pack appearance inspection device is an inspection device in a cloud server, used to inspect the object to be inspected; furthermore, the display device can also be controlled to display images and inspection results, etc.

[0020] In some possible implementations, the first mark can also be displayed on the first battery pack using augmented reality technology.

[0021] In the above technical solution, by controlling the display device to display the test results, users can more intuitively know the location of the defect, and then maintain the battery pack, or remotely guide the workers on the battery pack line to maintain the battery pack.

[0022] In conjunction with the first aspect, in some implementations of the first aspect, the processing unit is further configured to: train the detection model corresponding to the second battery pack using the first detection result.

[0023] In the above technical solution, the detection model is continuously trained by the detection results, which helps to continuously improve the detection capability of the battery pack appearance inspection device.

[0024] In conjunction with the first aspect, in some implementations of the first aspect, the processing unit is further configured to: train a detection model corresponding to the second battery pack using a second marker, the second marker being generated in response to a user's operation on the image, the second marker indicating a second type of defect present on the outer surface of the first battery pack.

[0025] For example, the second type of defect can be at least one of the following: dent, bulge, membrane rupture, screw defect (such as missing screw, screw not tightened, etc.), and the second type of defect is different from the first type of defect.

[0026] In some possible implementations, the second type of defect and the first type of defect may be of the same type, but they are not detected by the detection model corresponding to the second battery pack.

[0027] For example, an image of a first battery pack is displayed to a user via a display device, and a second mark is generated in response to the user's operation on the image.

[0028] In the above technical solution, when the detection device fails to detect certain defects, the detection results can be modified in response to user operations, enabling more accurate detection and analysis of various battery packs. Furthermore, the detection model can be trained in real-time using user-interactively labeled data, helping to continuously enhance detection performance and improve defect detection rate and accuracy.

[0029] In a second aspect, a battery pack appearance inspection device is provided, the device comprising a battery pack appearance inspection device as described in any possible implementation of the first aspect and a display device for displaying the first inspection result.

[0030] In the above technical solution, an interactive device is provided that enables users to more accurately complete various image processing and analysis, especially information display, remote positioning and interactive annotation, during the battery pack appearance inspection process.

[0031] In conjunction with the second aspect, in some implementations of the second aspect, the interactive device includes at least one of the following: a virtual reality (VR) device and an augmented reality (AR) device.

[0032] In the above technical solutions, VR and AR devices allow users to participate in interactive activities as if they were actually there during remote inspections, enabling new forms and experiences of remote maintenance and image annotation.

[0033] Thirdly, a battery pack appearance inspection system is provided, the system including an interactive device as described in any possible implementation of the second aspect, and a camera device; wherein the camera device is used to acquire the image, the camera device is located in the area where the first battery pack is located, the interactive device is located in the area where the user is located, and the user and the first battery pack are in different areas.

[0034] In the above technical solution, images of the battery pack to be tested can be remotely acquired through a camera device, and the images and / or test results of the battery pack to be tested can be displayed to the user through an interactive device. The test results can also be corrected in response to the user's operation through the interactive device, which helps to realize remote testing and remote maintenance of the battery pack.

[0035] Fourthly, a battery pack appearance inspection method is provided, comprising: acquiring an image of a first battery pack to be inspected, the image showing part or all of the outer surface of the first battery pack; inputting the image into a similarity model to determine information of a second battery pack from data of multiple battery packs in a training library; determining a detection model corresponding to the second battery pack from detection models corresponding to the multiple battery packs based on the information of the second battery pack, the data of the multiple battery packs being used to train the detection models corresponding to the multiple battery packs; inputting the image into the detection model corresponding to the second battery pack to obtain a first detection result for the first battery pack; wherein the similarity between the appearance of the first battery pack and the second battery pack is greater than or equal to a similarity threshold, and the detection model corresponding to the second battery pack is used to detect a first type of defect; the first detection result includes at least one of the following: indicating whether the first type of defect exists on the outer surface of the first battery pack, and indicating the location of the first type of defect on the outer surface of the first battery pack.

[0036] In conjunction with the fourth aspect, in some implementations of the fourth aspect, the method further includes: when the first detection result indicates that the first type of defect exists on the outer surface of the first battery pack, inputting the first detection result into a defect location annotation model to obtain a first mark indicating the location of the first type of defect on the outer surface of the first battery pack; and controlling the display device to display the first mark based on the image.

[0037] In conjunction with the fourth aspect, in some implementations of the fourth aspect, the method further includes: using the first detection result to train the detection model corresponding to the second battery pack.

[0038] In conjunction with the fourth aspect, in some implementations of the fourth aspect, the method further includes: training a detection model corresponding to the second battery pack using a second marker generated in response to a user's operation on the image, the second marker indicating a second type of defect present on the outer surface of the first battery pack.

[0039] For a description of the beneficial effects of the fourth aspect, please refer to the description of the beneficial effects of the first aspect, which will not be repeated here. Attached Figure Description

[0040] Figure 1 This is a schematic block diagram of the battery pack appearance inspection device provided in the embodiments of this application;

[0041] Figure 2 This is a schematic block diagram of the battery pack appearance inspection system architecture provided in the embodiments of this application;

[0042] Figure 3 This is a schematic diagram of a migration model provided in an embodiment of this application;

[0043] Figure 4 This is a schematic block diagram of another battery pack appearance inspection system architecture provided in the embodiments of this application;

[0044] Figure 5 This is a schematic diagram illustrating the application scenario provided in the embodiments of this application;

[0045] Figure 6 This is a schematic flowchart of the battery pack appearance inspection method provided in the embodiments of this application;

[0046] Figure 7 This is another schematic block diagram of the battery pack appearance inspection device provided in the embodiments of this application. Detailed Implementation

[0047] In the description of the embodiments of this application, unless otherwise stated, " / " means "or", for example, A / B can mean A or B; "and / or" in this document describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. In this application, "at least one" means one or more, and "more" means two or more. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or multiple items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple.

[0048] The use of prefixes such as "first" and "second" in this application embodiment is solely for distinguishing different descriptive objects and does not limit the position, order, priority, quantity, or content of the described objects. The use of ordinal numbers and other prefixes to distinguish descriptive objects in this application embodiment does not constitute a limitation on the described objects. The description of the described objects is found in the claims or the context of the embodiments, and the use of such prefixes should not constitute unnecessary restrictions.

[0049] As mentioned above, under the current technological background, the deployment and use of visual inspection applications typically involves the following process: image collection, image annotation, detection model construction, detection model evaluation, deployment, and application. Generally, the first four stages involve engineers developing the model locally, i.e., offline operation. Multiple rounds of algorithm iteration are often required between detection model construction and evaluation to improve model accuracy. Once the detection model's accuracy reaches a certain level, it is deployed to a real operational environment for use and / or testing (i.e., deployment), entering the application stage. After deployment, the model undergoes iterative updates based on its actual application. At this stage, the collected images are primarily from online sources and often represent images that the detection model struggles to recognize or misidentifies. New samples are collected and transferred offline for manual image annotation and model optimization for the next version. It is evident that model optimization and iterative updates rely heavily on image collection and annotation, and image annotation often requires significant manual intervention. In reality, there are a wide variety of visual inspection scenarios, such as appearance inspection of various products, autonomous driving, and detection of people, objects, and events by monitoring equipment. Visual inspection for each of these scenarios requires repeating the above process. To achieve automated inspection of multiple scenarios or multiple products, a large amount of manpower and repetitive work is often required.

[0050] In view of this, embodiments of this application provide a battery pack appearance inspection method, apparatus and system, which can perform similar battery pack matching based on the collected images of the battery pack to be inspected, thereby realizing the rapid transfer and generalization of the detection model between different battery packs, thus increasing the applicable scenarios of the battery pack appearance inspection system.

[0051] Figure 1 A schematic block diagram of a battery pack appearance inspection device (hereinafter referred to as the inspection device) provided in an embodiment of this application is shown, as follows: Figure 1As shown, the detection device 100 includes an image acquisition unit 110 and a processing unit 120. The image acquisition unit 110 can be used to acquire an image of a first battery pack to be detected, the image showing part or all of the outer surface of the first battery pack. The processing unit 120 can be used to: input the image of the first battery pack into a similarity model; determine the information of a second battery pack from data of multiple battery packs in a training library; determine the detection model corresponding to the second battery pack from the detection models corresponding to the multiple battery packs based on the information of the second battery pack; the data of multiple battery packs is used to train the detection models corresponding to the multiple battery packs; input the image into the detection model corresponding to the second battery pack to obtain a first detection result for the first battery pack; wherein the similarity between the appearance of the first battery pack and the second battery pack is greater than or equal to a similarity threshold, and the detection model corresponding to the second battery pack is used to detect a first type of defect; the first detection result includes at least one of the following: indicating whether a first type of defect exists on the outer surface of the first battery pack, and indicating the location of the first type of defect on the outer surface of the first battery pack.

[0052] For example, the first type of defect may include, but is not limited to, surface cracks, dents, defects, scratches, etc.; or welding defects, etc. The first type of defect may vary for different battery packs. In addition, the first type of defect may also include other defects besides those mentioned above.

[0053] For example, the first battery pack and the second battery pack can be battery packs that are similar or identical in appearance but different in size; or, the first battery pack and the second battery pack can be battery packs that are partially similar in appearance.

[0054] Optionally, the device communicates with the display device, and the processing unit 120 is further configured to: when the first detection result indicates that the first type of defect exists on the outer surface of the first battery pack, input the first detection result into the defect location marking model to obtain a first mark indicating the location of the first type of defect on the outer surface of the first battery pack; and control the display device to display the first mark based on the image.

[0055] In some possible implementations, the first mark can be input into a precise positioning model so that the image displayed by the display device coincides with the first battery pack, thereby enabling the display device to display the first mark at the actual position of the first battery pack.

[0056] In some possible implementations, the display device is also used to display the first detection result, such as showing that the first battery pack has a first type of defect, or to show the user the location of the first type of defect on the outer surface of the first battery pack in the form of text.

[0057] Optionally, the processing unit 120 is further configured to: use the first detection result to train the detection model corresponding to the second battery pack.

[0058] In some possible implementations, the first marker can also be used to train the defect location annotation model.

[0059] Optionally, the processing unit 120 is further configured to: train a detection model corresponding to the second battery pack using a second marker, the second marker being generated in response to a user's operation on the image, the second marker indicating a second type of defect present on the outer surface of the first battery pack.

[0060] Understandably, the second type of defect is the defect that the detection model for the second battery pack failed to detect.

[0061] For example, the operations performed by the image acquisition unit 110 and the processing unit 120 can be performed by a single processor; or they can be performed by different processors.

[0062] In practice, the detection device 100 can be located in a cloud server or in an interactive device.

[0063] In some possible implementations, the training library does not contain data for the first battery pack.

[0064] The detection device provided in this application embodiment can perform appearance detection of similar battery packs by using the detection model corresponding to the existing battery pack. This improves the compatibility of the battery pack appearance detection device with the appearance detection of different battery packs, eliminating the need to repeat the offline image annotation process for each battery pack. This helps to reduce the time and manpower costs required for the model used by the battery pack appearance detection device to go from training to application.

[0065] To enable the transfer of the detection model from the second battery pack to the first battery pack, the detection system can have the following capabilities: Figure 2 The system architecture shown.

[0066] Figure 2 Figure (a) shows a schematic diagram of a detection system, which includes a training state and an application state. The visual image training library (hereinafter referred to as the training library) includes labeled images of one or more types of battery packs, each type of battery pack may include one or more labeled images; the multi-battery pack model (hereinafter referred to as the model) may include appearance detection models, precise localization models, defect location annotation models, transfer models, etc.

[0067] Among them, the appearance detection model (hereinafter referred to as the detection model) is based on deep learning image classification and target detection algorithms, which can realize the detection of various types of defects by recognizing key information in the image. For example, the appearance detection model may include, but is not limited to: YOLO series models (such as YOLO v2 to YOLO v5) and region-based convolutional neural network (R-CNN) series models (such as Faster R-CNN and Mask R-CNN).

[0068] The precise positioning model is based on deep learning vision algorithms and can perform multi-view calibration to achieve precise positioning in the real world and the virtual world.

[0069] The defect location annotation model is based on deep learning-based visual and multimodal algorithms. It achieves automatic annotation through learning annotation behavior and content, for example, annotating defect locations detected by an external inspection model. For instance, the defect location annotation model can be an annotation model, or it can be other models.

[0070] The transfer model is based on a deep learning transfer algorithm to transfer the detection model corresponding to the second battery pack to the first battery pack, that is, to enable the detection model corresponding to the second battery pack to detect the first battery pack.

[0071] In training mode, the detection device acquires images, detects relevant defects according to the detection model in the model, and then marks the relevant defects in the images using the defect location annotation model, obtaining marked images, which are then input into the training library. The detection device can also control the display device to display the images and the information of the battery pack indicated by the images (such as name, model, etc.) to the user, and respond to the user's preset operations to obtain the marked images, which are then input into the training library. The detection model and the defect location annotation model are trained using the marked images in the training library to improve the detection accuracy and automatic marking accuracy of the detection device.

[0072] In application mode, the detection device uses a model to detect the image to be detected (i.e., the application-mode image) and obtains a detection result. When the detection result indicates that an object in the image to be detected has a defect, the device can also mark the defect based on the model and then display the mark on the defect through a display device. Furthermore, the detection device can input the detection result and / or the defect mark into a training library to enable the model to continuously improve its learning and increase detection accuracy.

[0073] For example, in application mode, an image of the first battery pack can be input into a transfer model to determine the detection model corresponding to the second battery pack. The following combines... Figure 3This describes the steps involved in determining the detection model corresponding to the second battery pack based on the image of the first battery pack:

[0074] 1. Input the images of the first battery pack and various battery packs in the training library into the feature extraction network to obtain a low-dimensional representation of the image features. For example, taking the first battery pack as battery pack a, the image of battery pack a can be represented as (n1, W, H, C), where W represents the pixels in the width direction of the image, H represents the pixels in the height direction, C represents the number of channels of the image, and n1 represents the number of images of battery pack a, where n1 is an integer greater than or equal to 1.

[0075] Each image can be represented as:

[0076]

[0077] By inputting n1 images of battery pack a into a feature extraction network, a low-dimensional representation of the image features, a = (n1, R), can be obtained, as shown in the following matrix:

[0078]

[0079] In formula (2), the first row of the matrix is ​​a low-dimensional representation of image 1 corresponding to battery pack a, and the n1th row of the matrix is ​​a low-dimensional representation of image n1 corresponding to battery pack a. For example, R can be 128, or it can be other values.

[0080] Taking battery pack b from the various battery packs in the training library as an example, the image of battery pack b can be represented as (n2, W, H, C), where n2 represents the number of images of battery pack b, and n2 is an integer greater than or equal to 1.

[0081] By inputting n² images of battery pack b into the feature extraction network, a low-dimensional representation of the image features, b = (n², R), can be obtained, as shown in the following matrix:

[0082]

[0083] In the matrix shown in formula (3), the first row is a low-dimensional representation of image 1 corresponding to battery pack b, and the n2th row is a low-dimensional representation of image n2 corresponding to battery pack b.

[0084] For example, the feature extraction network can be a deep neural network or a CNN. More specifically, the feature extraction network can be the encoder part of a variational autoencoder (VAE), Transformer, etc.

[0085] 2. Based on the low-dimensional representations of the image features of battery pack a and battery pack b, determine the distance between the low-dimensional feature distributions of battery pack a and battery pack b. When the distance is less than the distance threshold, determine that the detection model of battery pack b should be used to detect battery pack a.

[0086] For example, taking the FID distance as an example, when FID(a,b)≤threshold, it is determined that the detection model of battery pack b is used to detect battery pack a. At this time, battery pack b is an example of the second battery pack mentioned above. Here, threshold represents the distance threshold, and FID(a,b) represents the distance between the low-dimensional feature distributions of battery pack a and battery pack b. FID(a,b) can be calculated by the following formula:

[0087]

[0088] Where, μ a μ represents the mean of the matrix a mentioned above. b Let represent the mean of matrix b, ∑a and ∑b represent the covariance matrices of matrices a and b respectively, and Tr represent the trace of the matrix.

[0089] Understandably, the smaller the FID(a,b) value, the higher the similarity in appearance between battery pack a and battery pack b.

[0090] The FID model can be understood as an example of the similarity model described above; or, the overall structure consisting of the feature extraction network and the FID model can be understood as an example of the similarity model described above. The similarity threshold can be determined by converting it based on the distance threshold mentioned above.

[0091] After determining the detection model corresponding to the second battery pack, the image of the first battery pack can be transformed using an affine transformation matrix T, i.e., T(a)≈b. Then, the detection model corresponding to the second battery pack is used to detect the transformed image T(a). Furthermore, the annotation model corresponding to the second battery pack can be used to label the transformed image; for example, A(T(a)) is the labeling result for battery pack a, which can be understood as an example of the first labeling mentioned above. Here, A() can be an annotation model, or it can be another annotation model.

[0092] In some possible implementations, the steps of determining the detection model corresponding to the second battery pack based on the image of the first battery pack, and the steps of transforming the image of the battery pack, can all be performed by the transfer model.

[0093] Figure 2(b) shows a more specific example of the detection system in both training and application states. Images of battery packs 1 through N are input to the detection device, which generates labeled images and trains each model using these labeled images. For example, the image of battery pack 1 includes at least image x1, the image of battery pack 2 includes at least image y1, and the image of battery pack N includes at least image z1. In the training state:

[0094] In one example, if the labeled image of battery pack 1 does not exist in the training library, the detection device can generate a labeled image L in response to the user's operation on image x1. x1 Then the labeled image L x1 Save to the training library; the labeled image L is saved in the training library. x1 Then, based on the labeled image L x1 Model training is performed so that when the detection device acquires other images related to battery pack 1, it can automatically generate a labeled image L. x’ If the image L is labeled x1 The marker in the image indicates the location of defect 1, so the image L after marking... x’ The markers in the diagram can indicate the location of defect 1.

[0095] In another example, if the training library stores labeled images of battery pack 2, then when the detection device acquires other images of battery pack 2, it can automatically generate labeled image L. y1 L y’ If the markers in the marked image of battery pack 2 in the training library indicate the location of defect 2, then the marked image L... y1 L y’ The markers in the diagram can indicate the location of defect 2.

[0096] In another example, if the similarity between battery pack N and battery pack 1 (and / or battery pack 2) is greater than or equal to a similarity threshold, then when the detection device acquires image z1 with battery pack N, a labeled image L can be automatically generated based on a transfer model. z* The image L after the label z* The markers in the diagram can indicate the location of defect 1 (and / or defect 2).

[0097] During application (i.e., in application mode), for the battery pack image Tbt to be inspected, when the detection model determines that the objects in image Tbt meet the preset conditions, it marks image Tbt using the defect location annotation model to obtain the marked image L. Tbt The detection device can control the display device to display the image L after marking. Tbt Alternatively, the detection device can also respond to user input, targeting the marked image L. Tbt Generate other markers to obtain the marked image LTbt’ And control the display device to display the image L after the mark. Tbt’ .

[0098] Furthermore, the labeled image L Tbt and / or labeled image L Tbt’ Input the training library to continuously train the model. The labeled image L... Tbt The marker in the image can be understood as an example of the first marker mentioned above. The image L after marking... Tbt’ The marker in can be understood as an example of the second marker mentioned above.

[0099] For example, if there is no image in the training library corresponding to the battery pack represented by image Tbt, and the similarity between battery pack M in the training library and the battery pack represented by image Tbt is greater than or equal to the similarity threshold, then the aforementioned labeled image L... Tbt It can be generated based on the detection results output by the detection model corresponding to battery pack M. Furthermore, it can use the labeled image L. Tbt The detection model corresponding to battery pack M is trained, and the specific training process is as follows:

[0100] The image Tbt is encoded in N stages. For the (i+1)th encoding stage of the N stages: based on the features output by the ith encoding stage... Figure 1 First, determine the input for the (i+1)th encoding stage; then, encode based on the input for the (i+1)th encoding stage to obtain features. Figure 2 Based on the input and features of the (i+1)th encoding stage Figure 2 Determine the feature map output at the (i+1)th encoding stage; i is an integer greater than or equal to 1 and less than or equal to N^2, and N is an integer greater than or equal to 2. Decode the image Tbt based on the encoding results obtained from N stages of encoding to obtain the label prediction result for the image Tbt; based on the label prediction result and the labeled image L... Tbt The label determines the loss function, and the parameters used in each encoding and decoding stage of the detection model are updated based on the loss function to make the label prediction result closer to the labeled image L. Tbt The mark.

[0101] For example, binary cross-entropy can be used as the loss function, or gradient harmonizing mechanism (GHM) can be used as the loss function.

[0102] It should be noted that battery packs 1 to N can be understood as examples of various battery packs in the training library mentioned above. The labeled images of battery packs 1 to N can be understood as examples of data from various battery packs in the training library mentioned above. The battery pack presented in image Tbt can be understood as an example of the first battery pack mentioned above. Battery pack M can be understood as an example of the second battery pack.

[0103] One implementation of controlling the display of the first mark based on the image of the first battery pack can be: controlling the display of the image of the first battery pack after the mark. For example, if the image of the first battery pack is image Tbt, then controlling the display of the first mark based on the image of the first battery pack can be: controlling the display of the image L after the mark. tbt .

[0104] For example, Figure 2 The detection device shown can be Figure 1 The detection device 100 shown may also be other devices with detection device functions, such as interactive devices.

[0105] In some possible implementations, the functions of the detection and display devices are implemented by an interactive device; that is, the interactive device includes both the detection and display devices. The interactive device can be a wearable device, such as an AR or VR device; alternatively, it can be an interactive system that includes a display device and a camera or voice device. The display device is used to display the image to be detected and / or the detection result, while the camera or voice device is used to detect the user's preset operations.

[0106] For example, the interactive device can respond to user actions to complete image labeling and / or confirmation of detection results. User actions may include, but are not limited to, preset gestures and preset voice commands. Preset gestures may include, but are not limited to, drawing actions such as swiping, zooming, virtual touch, and smearing. In some possible implementations, in response to user actions, image enhancement, segmentation, stitching, generation, and positional highlighting can also be performed.

[0107] For example, the images required in the training state and / or the images to be detected in the application state can be acquired by a camera device. The camera device may include, but is not limited to, cameras, video cameras, etc., and is mainly used to acquire images, videos, and other information in real-world scenes. For example, the camera device may include red, green, and blue / infrared (RGB / IR) cameras, or depth cameras, such as time-of-flight (TOF) cameras, binocular cameras, structured light cameras, etc.

[0108] Figure 4This is a schematic block diagram of the detection system provided in the embodiments of this application. Figure 4 The detection system 400 shown may include a camera device 200 and an interactive device 300. The camera device 200 is used to acquire images of the first battery pack to be detected and is located in the area where the first battery pack is located. The interactive device 300 includes a detection device 100 and is located in the area where the user is located, wherein the area where the user is located is different from the area where the first battery pack is located.

[0109] The detection system provided in this application embodiment can enable users to detect and / or maintain remote battery packs, which helps to improve detection efficiency and enhance the user's interactive experience.

[0110] To help readers better understand the detection solution provided in this application, the following is combined with... Figure 5 The application scenarios of the embodiments of this application will be introduced.

[0111] Figure 5 This diagram illustrates a scenario for inspecting the appearance of a battery pack / cell. The interaction between the camera device, the inspection device, and the battery management system (BMS) is shown below. Figure 5 As shown, a camera device captures an image of the battery pack or cell's appearance and inputs the image into a detection device. When the detection device detects a defect in the battery pack (or cell) in the image, it marks the defect. This mark can instruct maintenance personnel to maintain the battery pack (or cell). In some possible implementations, if the BMS system records information about the battery pack (or cell) in the image, the mark can also indicate the location of the defect to the BMS system. For example, the defect may include external defects in the battery pack (or cell), such as breakage, missing screws, or welding defects.

[0112] In some possible implementations, the detection device or system provided in this application embodiment can also be applied to other appearance inspection scenarios, such as appearance inspection of industrial products on industrial production lines.

[0113] Figure 6 This illustration shows a schematic flowchart of a battery pack appearance inspection method provided in an embodiment of this application. The method can be performed by… Figure 1 The detection device 100 shown is used for execution; or, it can be performed by... Figure 2 or Figure 4 The system shown is executed. Method 600 may include:

[0114] S601, acquire an image of the first battery pack to be detected, the image showing part or all of the outer surface of the first battery pack.

[0115] For example, the specific method for acquiring an image of the first battery pack can be referred to the description in the above embodiments, and will not be repeated here.

[0116] S602, the image is input into a similarity model to determine the information of the second battery pack from the data of multiple battery packs in the training library, and the detection model corresponding to the second battery pack is determined from the detection model corresponding to the multiple battery packs based on the information of the second battery pack, and the data of the multiple battery packs is used to train the detection model corresponding to the multiple battery packs.

[0117] For example, the specific method for determining the detection model corresponding to the second battery pack can be referred to the description in the above embodiments, and will not be repeated here.

[0118] For example, the detection model corresponding to the second battery pack can be the detection model described above, or it can be the part of the detection model used to detect the second battery pack.

[0119] S603, input the image into the detection model corresponding to the second battery pack to obtain a first detection result for the first battery pack; wherein, the similarity between the appearance of the first battery pack and the second battery pack is greater than or equal to a similarity threshold, and the detection model corresponding to the second battery pack is used to detect a first type of defect; the first detection result includes at least one of the following: indicating whether the first type of defect exists on the outer surface of the first battery pack, and indicating the position of the first type of defect on the outer surface of the first battery pack.

[0120] For example, method 600 may also include other steps performed by the detection device 100 described above.

[0121] The beneficial effects of the detection method provided in this application embodiment can be found in the above description of the device, and will not be repeated here.

[0122] Figure 7 This is a schematic block diagram of the battery pack appearance inspection device provided in the embodiments of this application. Figure 7 The battery pack appearance inspection device 2000 shown may include a processor 2010, a transceiver 2020, and a memory 2030. The processor 2010, transceiver 2020, and memory 2030 are connected via internal interconnection paths. The memory 2030 stores instructions, and the processor 2010 executes the instructions stored in the memory 2030 to implement the methods described in the above embodiments. Optionally, the memory 2030 may be coupled to the processor 2010 via an interface or integrated with the processor 2010.

[0123] It should be noted that the transceiver 2020 mentioned above may include, but is not limited to, transceiver devices such as input / output interfaces, to enable communication between device 2000 and other devices or communication networks.

[0124] The memory 2030 can be a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM).

[0125] Transceiver 2020 uses transceiver devices, such as but not limited to transceivers, to enable communication between device 2000 and other devices or communication networks to receive / send data / information for implementing the methods in the above embodiments.

[0126] It should be noted that the detection device 2000 and the detection device 100 can be the same device, and the detection device 2000 can be installed in the detection system 400.

[0127] In the various embodiments of this application, unless otherwise specified or in case of logical conflict, the terminology and / or descriptions between the various embodiments are consistent and can be referenced by each other. Technical features in different embodiments can be combined to form new embodiments according to their inherent logical relationships.

[0128] This application also provides a computer program product, which includes computer program code. When the computer program code is run on a computer, it causes the computer to implement the methods described in the above embodiments of this application.

[0129] This application also provides a computer-readable storage medium storing computer instructions that, when executed on a computer, cause the computer to implement the methods described in the above embodiments of this application.

[0130] This application also provides a chip, including circuitry, for performing the methods described in the above embodiments of this application.

[0131] In implementation, each step of the above method can be completed by integrated logic circuits in the processor's hardware or by instructions in software. The method disclosed in the embodiments of this application can be directly implemented by a hardware processor, or by a combination of hardware and software modules within the processor. The software modules can reside in random access memory, flash memory, read-only memory, programmable read-only memory, power-on erasable programmable memory, registers, or other mature storage media in the art. This storage medium is located in memory, and the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method. To avoid repetition, detailed descriptions are omitted here.

[0132] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0133] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0134] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0135] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0136] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A battery pack appearance inspection device, characterized in that, include: An image acquisition unit is used to acquire an image of a first battery pack to be detected, the image showing part or all of the outer surface of the first battery pack; The processing unit is configured to input the image into a similarity model to determine the information of a second battery pack from the data of multiple battery packs in the training library, and to determine the detection model corresponding to the second battery pack from the detection models corresponding to the multiple battery packs based on the information of the second battery pack. The data of the multiple battery packs is used to train the detection models corresponding to the multiple battery packs. The data of the multiple battery packs in the training library does not include the data of the first battery pack. The processing unit is further configured to: input the image into the detection model corresponding to the second battery pack to obtain a first detection result for the first battery pack; Wherein, the similarity between the appearance of the first battery pack and the second battery pack is greater than or equal to the similarity threshold, and the detection model corresponding to the second battery pack is used to detect the first type of defect; The first detection result includes at least one of the following: indicating whether the first type of defect exists on the outer surface of the first battery pack, and indicating the location of the first type of defect on the outer surface of the first battery pack.

2. The apparatus according to claim 1, characterized in that, The device communicates with the display device, and the processing unit is further configured to: When the first detection result indicates that the first type of defect exists on the outer surface of the first battery pack, the first detection result is input into the defect location annotation model to obtain a first mark indicating the location of the first type of defect on the outer surface of the first battery pack. The display device is controlled to display the first mark based on the image.

3. The apparatus according to claim 1, characterized in that, The processing unit is also used for: The detection model corresponding to the second battery pack is trained using the first detection result.

4. The apparatus according to any one of claims 1 to 3, characterized in that, The processing unit is also used for: The detection model corresponding to the second battery pack is trained using a second marker, which is generated in response to a user's operation on the image. The second marker indicates a second type of defect on the outer surface of the first battery pack.

5. An interactive device, characterized in that, The device includes a battery pack appearance inspection apparatus as described in any one of claims 1 to 4 and a display device, wherein the display device is used to display the first inspection result.

6. The interactive device according to claim 5, characterized in that, The interactive device is a virtual reality device or an augmented reality device.

7. A battery pack appearance inspection system, characterized in that, The device includes the interactive device as described in claim 5 or 6, and a camera device; wherein the camera device is used to capture the image, the camera device is located in the area where the first battery pack is located, the interactive device is located in the area where the user is located, and the user and the first battery pack are in different areas.

8. A method for inspecting the appearance of a battery pack, characterized in that, include: Acquire an image of a first battery pack to be inspected, the image showing part or all of the outer surface of the first battery pack; The image is input into a similarity model to determine the information of the second battery pack from the data of multiple battery packs in the training library. Based on the information of the second battery pack, the detection model corresponding to the second battery pack is determined from the detection models corresponding to the multiple battery packs. The data of the multiple battery packs is used to train the detection models corresponding to the multiple battery packs. The data of the multiple battery packs in the training library does not include the data of the first battery pack. The image is input into the detection model corresponding to the second battery pack to obtain a first detection result for the first battery pack; Wherein, the similarity between the appearance of the first battery pack and the second battery pack is greater than or equal to the similarity threshold, and the detection model corresponding to the second battery pack is used to detect the first type of defect; The first detection result includes at least one of the following: indicating whether the first type of defect exists on the outer surface of the first battery pack, and indicating the location of the first type of defect on the outer surface of the first battery pack.

9. The method according to claim 8, characterized in that, The method further includes: When the first detection result indicates that the first type of defect exists on the outer surface of the first battery pack, the first detection result is input into the defect location annotation model to obtain a first mark indicating the location of the first type of defect on the outer surface of the first battery pack. The control display device displays the first mark based on the image.

10. The method according to claim 8, characterized in that, The method further includes: The detection model corresponding to the second battery pack is trained using the first detection result.

11. The method according to any one of claims 8 to 10, characterized in that, The method further includes: The detection model corresponding to the second battery pack is trained using a second marker, which is generated in response to a user's operation on the image. The second marker indicates a second type of defect on the outer surface of the first battery pack.

Images