Defect detection methods, systems, apparatus, devices, storage media, and products

Abstract

The application relates to a defect detection method, system, device, equipment, storage medium and product. In response to a detection instruction of a target material, a camera device is controlled to collect a two-dimensional image and a three-dimensional image of the target material, and a defect detection result of the target material is determined according to the two-dimensional image and the three-dimensional image of the target material. In the method, the camera device is a structured light camera device integrating a two-dimensional image collection function and a three-dimensional image collection function, so that the two-dimensional image and the three-dimensional image can be collected without switching the camera device, the convenience of defect detection on the target material is improved, the two-dimensional image and the three-dimensional image are collected by the same camera device, so that the two-dimensional image and the three-dimensional image are ensured to be in the same coordinate system, the efficiency of defect detection on the target material is improved, and in addition, the two-dimensional image and the three-dimensional image are used to detect defects of the target material in two different dimensions, so that the precision of defect detection is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of welding detection, in particular to a defect detection method, system, device, equipment, storage medium and product. BACKGROUND

[0002] In a production technology enterprise, in order to guarantee the quality of products, strict quality detection must be carried out on the products. For example, the surface defects of the products are detected.

[0003] Taking the lithium battery assembly manufacturing process as an example, after the secondary injection of the battery, in order to ensure the sealing property of the battery after injection, the sealing nail welding method is needed to ensure the sealing property of the battery after the glue nail is pressed. Then, in view of the fact that the welding process is affected by many objective factors in actual application, there are many post-welding defects after welding.

[0004] In this regard, there is an urgent need for a method that can conveniently, efficiently and accurately detect post-welding defects of sealing nails. SUMMARY

[0005] Therefore, it is necessary to provide a defect detection method, system, device, equipment, storage medium and product, which can conveniently, efficiently and accurately detect post-welding defects of sealing nails.

[0006] In a first aspect, the present application provides a defect detection method, which comprises:

[0007] In response to a detection instruction of a target material, a camera device is controlled to collect a two-dimensional image and a three-dimensional image of the target material; the camera device is a structured light camera device integrating a two-dimensional image collection function and a three-dimensional image collection function;

[0008] According to the two-dimensional image and the three-dimensional image of the target material, a defect detection result of the target material is determined.

[0009] In the embodiment of the present application, in response to the detection instruction of the target material, the camera device is controlled to collect the two-dimensional image and the three-dimensional image of the target material, and the defect detection result of the target material is determined according to the two-dimensional image and the three-dimensional image of the target material. In the method, since the camera device is a structured light camera device integrating the two-dimensional image collection function and the three-dimensional image collection function, the two-dimensional image and the three-dimensional image can be collected without switching the camera device in the process of collecting the two-dimensional image and the three-dimensional image, thereby ensuring the convenience of collecting the image and improving the convenience of defect detection of the target material. Moreover, the two-dimensional image and the three-dimensional image are collected by the same camera device, so that the collected two-dimensional image and three-dimensional image are in the same coordinate system, so that the two images do not need to be preprocessed by too many complex data unification operations when the two-dimensional image and the three-dimensional image are used to detect defects, thereby improving the efficiency of defect detection of the target material. In addition, the target material is detected for defects by the two-dimensional image and the three-dimensional image of the target material in two different dimensions, so that the multi-dimensional information ensures the comprehensiveness of the defect detection of the target material, avoids the misjudgment problem caused by single image detection of the two-dimensional image or the three-dimensional image, and improves the precision of defect detection.

[0010] In one embodiment, the target material includes a plurality of detection products, and each detection product is at a different detection station. The camera device is controlled to collect the two-dimensional image and the three-dimensional image of the target material, including:

[0011] The camera device is controlled to collect the two-dimensional image and the three-dimensional image of the detection product at the current detection station.

[0012] After the two-dimensional image and the three-dimensional image of the detection product at the current detection station are collected, the camera device is controlled to move to the next detection station to collect the two-dimensional image and the three-dimensional image of the detection product at the next detection station, so as to collect the two-dimensional image and the three-dimensional image of all detection products.

[0013] In the embodiment of the present application, the camera device is controlled to collect the two-dimensional image and the three-dimensional image of the detection product at the current detection station, and after the two-dimensional image and the three-dimensional image of the detection product at the current detection station are collected, the camera device is controlled to move to the next detection station to collect the two-dimensional image and the three-dimensional image of the detection product at the next detection station, so as to collect the two-dimensional image and the three-dimensional image of all detection products. In the method, the image collection of the plurality of detection products can be realized by moving the camera device, thereby ensuring the convenience of defect detection of the target material. Moreover, after the camera device is moved, each detection product is in the field of view of the camera device when the image is collected, thereby improving the accuracy of defect detection of the target material.

[0014] In one of the embodiments, the control camera equipment collects the two-dimensional image and the three-dimensional image of the detected product on the current detection station, including:

[0015] The control camera equipment collects the two-dimensional image of the detected product on the current detection station under the condition that the two-dimensional light supplement equipment is turned on, and sends a moving instruction to the upper computer after the two-dimensional image collection is completed, the moving instruction being used to instruct the upper computer to move the two-dimensional light supplement equipment to the next detection station.

[0016] The control camera equipment collects the three-dimensional image of the detected product on the current detection station under the condition that the three-dimensional light supplement equipment is turned on.

[0017] In the embodiments, the control camera equipment collects the two-dimensional image of the detected product on the current detection station under the condition that the two-dimensional light supplement equipment is turned on, and sends a moving instruction to the upper computer after the two-dimensional image collection is completed, the moving instruction being used to instruct the upper computer to move the two-dimensional light supplement equipment to the next detection station, and then the control camera equipment collects the three-dimensional image of the detected product on the current detection station under the condition that the three-dimensional light supplement equipment is turned on. In this method, the two-dimensional light supplement equipment is used for light supplement when the two-dimensional image is collected, and the three-dimensional light supplement equipment is used for light supplement when the three-dimensional image is collected, so that the light state of the environment when the two-dimensional image and the three-dimensional image are collected is in the best state, the clarity and the information comprehensiveness of the collected two-dimensional image and the three-dimensional image of the detected product are improved, the effect of high-precision full-edge image taking of the detected product is realized, and the accuracy of subsequent defect detection is ensured.

[0018] In one of the embodiments, the detection instruction of the target material is sent by the upper computer after the target material enters the detection station and the pressing device is controlled to perform the pressing operation on the target material; the coverage area of the pressing device is greater than or equal to the upper surface area of the target material. The target material is pressed by the pressing device, and the height of all the detected products in the target material can be unified, so that all the detected products are ensured to be on the same horizontal plane, and the phenomenon of image loss of pixels caused by the camera depth of field is avoided.

[0019] In one of the embodiments, the camera equipment is a 3D structured light camera. The 3D structured light camera has higher camera precision, and has greater projection dimension by using the phase shift method structured light technology, so that more material information can be collected by the camera, the image taking speed and the image quality are greatly improved, and the 3D structured light camera can be compatible with various welding process products by using the 3D structured light, so that the compatibility of defect detection of different materials is ensured.

[0020] In one of the embodiments, the defect detection result of the target material is determined according to the two-dimensional image and the three-dimensional image of the target material, including:

[0021] spatial alignment processing is performed on the two-dimensional image and the three-dimensional image;

[0022] The two-dimensional image and the three-dimensional image after the spatial alignment processing are processed by a preset deep learning algorithm to obtain a defect detection result of the target material.

[0023] In the embodiments of the present application, spatial alignment processing is performed on the two-dimensional image and the three-dimensional image, and then the two-dimensional image and the three-dimensional image after the spatial alignment processing are processed by a preset deep learning algorithm to obtain a defect detection result of the target material. In this method, the two-dimensional image and the three-dimensional image are first spatially aligned, which improves the convenience of subsequent defect detection on the two-dimensional image and the three-dimensional image after the spatial alignment processing and reduces the computational complexity. Moreover, since the two-dimensional image and the three-dimensional image are collected by the same camera device, the two-dimensional image and the three-dimensional image do not need too many complex algorithms for spatial alignment processing, which can reduce the time for spatial alignment processing of the two-dimensional image and the three-dimensional image, improve the efficiency of spatial processing, and thus improve the efficiency of the entire defect detection process.

[0024] In one of the embodiments, the defect detection result includes a first defect detection result and a second defect detection result, the first defect is a defect without height information, and the second defect is a defect with height information.

[0025] The two-dimensional image and the three-dimensional image after the spatial alignment processing are processed by a preset deep learning algorithm to obtain a defect detection result of the target material, including:

[0026] A first detection region in the two-dimensional image and a second detection region in the three-dimensional image are obtained.

[0027] The first defect detection result is detected according to the first detection region, and the second defect detection result is detected according to the second detection region.

[0028] In the embodiments of the present application, the detection regions in the two-dimensional image and the three-dimensional image are obtained to extract the key detection regions in the images, which avoids a large amount of background redundant information and improves the accuracy of the first defect detection result and the second defect detection result.

[0029] In one of the embodiments, the second detection region in the three-dimensional image is obtained, including:

[0030] The three-dimensional image is preprocessed to obtain point cloud data of the target material.

[0031] The second detection region is determined according to the point cloud data.

[0032] In the embodiments of the present application, the second detection region is determined according to the point cloud data corresponding to the three-dimensional image of the target material, which improves the accuracy of the extraction of the second detection region.

[0033] In one of the embodiments, the detection result of the first defect detected according to the first detection area comprises:

[0034] obtaining a planar feature of the first detection area;

[0035] determining the detection result of the first defect according to the planar feature.

[0036] In the embodiments of the present application, the planar feature of the first detection area is obtained, and the defect without height information is determined based on the planar feature, thereby improving the accuracy of defect detection.

[0037] In one of the embodiments, the detection result of the second defect detected according to the second detection area comprises:

[0038] obtaining a distance between the weld height of the target material and the reference surface of the material top cover in the second detection area;

[0039] determining the detection result of the second defect according to the distance.

[0040] In the embodiments of the present application, the detection result of the defect with height information in the target material can be determined by the distance, thereby ensuring the accuracy of defect detection.

[0041] In one of the embodiments, the method further comprises:

[0042] sending the defect detection result of the target material to the upper computer; and / or, displaying the defect detection result of the target material in the front-end interface. In the present embodiment, the defect detection result of the target material is sent to the upper computer, so that the upper computer can timely and accurately obtain the defect detection result. In addition, the defect information of the target material can be more intuitively obtained by displaying the defect detection result of the target material in the front-end interface.

[0043] In a second aspect, the present application further provides a defect detection system, which comprises a support frame, a camera device arranged on the support frame, a control device in communication connection with the camera device, an upper computer in communication connection with the control device, and a detection station arranged below the support frame; the camera device is a structured light camera device integrating two-dimensional image acquisition function and three-dimensional image acquisition function;

[0044] the detection station is used for carrying the target material to be detected;

[0045] the upper computer is used for sending a detection instruction of the target material to the control device;

[0046] the control device is used for controlling the camera device to acquire two-dimensional images and three-dimensional images of the target material, and determining a defect detection result of the target material according to the two-dimensional images and the three-dimensional images of the target material.

[0047] In the embodiment of the present application, the support frame is designed in the defect detection system, the camera device is arranged on the support frame, the control device is in communication connection with the camera device, the upper computer is in communication connection with the control device, and the detection workstations are arranged below the support frame. When the target material to be detected reaches the detection workstation, the upper computer sends a detection instruction of the target material to the control device, and then the control device controls the camera device to collect two-dimensional images and three-dimensional images of the target material, and determines a defect detection result of the target material according to the two-dimensional images and the three-dimensional images of the target material. In the system, the camera device is a structured light camera device integrating two-dimensional image collection function and three-dimensional image collection function. In the process of collecting two-dimensional images and three-dimensional images, the two-dimensional images and the three-dimensional images can be collected without switching the camera device, which ensures the convenience of collecting images, thereby improving the convenience of defect detection on the target material. Moreover, the two-dimensional images and the three-dimensional images are collected by the same camera device, which ensures that the collected two-dimensional images and three-dimensional images are in the same coordinate system, so that when the two-dimensional images and the three-dimensional images are used for defect detection, there is no need for too many complex data unification operations for pre-processing of the two images, thereby improving the efficiency of defect detection on the target material. In addition, the target material is detected by two-dimensional images and three-dimensional images of different dimensions, which ensures the comprehensiveness of target material defect detection, avoids the misjudgment problem caused by single image detection of two-dimensional images or three-dimensional images, and improves the accuracy of defect detection.

[0048] In one of the embodiments, the system further comprises a lifting device; the lifting device is arranged on the support frame, and the camera device is installed on the lifting device; the positions below the support frame include a plurality of detection workstations;

[0049] The control device is configured to control the lifting device to move the camera device to the position of each detection workstation to collect two-dimensional images and three-dimensional images of the detection products in each detection workstation, respectively.

[0050] In the embodiment of the present application, the lifting device is designed on the support frame in the defect detection system, and the camera device is installed on the lifting device. The positions below the support frame include a plurality of detection workstations. The lifting device can be controlled by the control device to move the camera device to the position of each detection workstation to collect two-dimensional images and three-dimensional images of the detection products in each detection workstation, respectively. In the system, the camera device can be moved by the lifting device to realize the collection of images of a plurality of detection products, which ensures the convenience of defect detection on the target material. Moreover, after the camera device is moved, each detection product is in the field of view of the camera device when the image is collected, and the image collection is complete.

[0051] In one of the embodiments, the system further comprises a sliding assembly arranged on the support frame and below the lifting device, and a two-dimensional light supplementing device arranged on the sliding assembly.

[0052] The control device is configured to control the camera device to collect a two-dimensional image of the detection product in each detection station with the two-dimensional light supplementing device turned on, and send a moving instruction to the upper computer after the two-dimensional image of the detection product in each detection station is collected, where the moving instruction is used to instruct the upper computer to move the two-dimensional light supplementing device to the next detection station through the sliding assembly.

[0053] In the embodiments, the sliding assembly and the two-dimensional light supplementing device are arranged in the defect detection system, and the two-dimensional light supplementing device is arranged on the sliding assembly, so that the two-dimensional light supplementing device is moved to the next detection station through the sliding assembly after the camera device collects the two-dimensional image of the detection product in the current detection station, and the camera device collects the three-dimensional image of the detection product in the next detection station with the two-dimensional light supplementing device turned on, thereby avoiding the problem of unclear image caused by lack of light when the two-dimensional image is collected, improving the clarity of the collected two-dimensional image of the detection product in each detection station, and ensuring the accuracy of subsequent defect detection.

[0054] In one of the embodiments, the system further comprises a three-dimensional light supplementing device arranged at a preset distance above each detection station.

[0055] The control device is configured to control the camera device to collect a three-dimensional image of the detection product in the corresponding detection station with the three-dimensional light supplementing device turned on.

[0056] In the embodiments, the three-dimensional light supplementing device is arranged at a preset distance above each detection station in the defect detection system, so that the camera device can collect the three-dimensional image in each detection station with the three-dimensional light supplementing device turned on, thereby avoiding the problem of unclear image caused by uneven or insufficient light when the three-dimensional image is collected, ensuring the clarity of the three-dimensional image, and ensuring the accuracy of subsequent defect detection.

[0057] In one of the embodiments, the system further comprises a press fitting device in communication with the upper computer.

[0058] The upper computer is further configured to send a detection instruction to the control device after the press fitting device performs a pressing operation on the target material after the target material enters the detection station, and the coverage area of the press fitting device is greater than or equal to the upper surface area of the target material.

[0059] In the embodiment of the present application, the system also designs a pressing device. After the target material enters the detection station, the pressing device is used to press the target material, so that when the target material includes multiple detection products, the heights of the multiple detection products can be unified, and the multiple detection products are on the same horizontal plane, thereby avoiding the phenomenon of image pixel loss caused by the depth of field of the camera.

[0060] In one of the embodiments, a cavity structure is arranged at a position corresponding to each detection station on the pressing device, and the three-dimensional light supplementing device is a ring light device arranged around the inside of the cavity.

[0061] The cavity structure is used for the camera device to align the detection products on each detection station.

[0062] In the embodiment of the present application, a cavity structure is arranged at a position corresponding to each detection station on the pressing device in the system, so that the camera device can collect two-dimensional images and three-dimensional images of the target material through the cavity of the pressing device, and the three-dimensional light supplementing device is arranged on the ring light device around the inside of the cavity, so that the camera device can clearly collect the images of each detection product when collecting the images of the target material, and the accuracy of the images is improved.

[0063] In a third aspect, the present application also provides a defect detection device, which comprises:

[0064] An image collection module is configured to control a camera device to collect two-dimensional images and three-dimensional images of a target material in response to a detection instruction for the target material. The camera device is a structured light camera device integrating two-dimensional image collection functions and three-dimensional image collection functions.

[0065] A defect determination module is configured to determine a defect detection result of the target material according to the two-dimensional images and the three-dimensional images of the target material.

[0066] In a fourth aspect, the embodiment of the present application provides a computer device, which comprises a memory and a processor. The memory stores a computer program, and the processor implements the steps of any one of the methods provided in the first aspect when executing the computer program.

[0067] In a fifth aspect, the embodiment of the present application provides a computer readable storage medium, which stores a computer program. The computer program is executed by a processor to implement the steps of any one of the methods provided in the first aspect.

[0068] In a sixth aspect, the embodiment of the present application provides a computer program product, which comprises a computer program. The computer program is executed by a processor to implement the steps of any one of the methods provided in the first aspect.

[0069] The above description is only a summary of the technical solutions of the present application. In order to make the technical means of the present application more clearly understood and implemented according to the content of the specification, and in order to make the above and other purposes, characteristics and advantages of the present application more apparent and easy to understand, the following specific embodiments of the present application are described. BRIEF DESCRIPTION OF DRAWINGS

[0070] Various other advantages and benefits will become apparent to those of ordinary skill in the art, upon reading the following detailed description of the preferred embodiments. The accompanying drawings are included to provide a description of the preferred embodiments, and are not meant to limit the present application. Moreover, in the attached drawings, the same reference numerals are used to denote the same components throughout the several views. In the drawings:

[0071] Figure 1 A schematic diagram of a defect detection system architecture provided in an embodiment is shown in FIG. 1.

[0072] Figure 2 A schematic diagram of a product detection structure provided in an embodiment is shown in FIG. 2.

[0073] Figure 3 A schematic diagram of a defect detection system architecture provided in another embodiment is shown in FIG. 3.

[0074] Figure 4 A schematic diagram of a defect detection system architecture provided in another embodiment is shown in FIG. 4.

[0075] Figure 5 A schematic diagram of a defect detection system architecture provided in another embodiment is shown in FIG. 5.

[0076] Figure 6 A schematic diagram of a defect detection system architecture provided in another embodiment is shown in FIG. 6.

[0077] Figure 7 A schematic diagram of an image acquisition device provided in an embodiment is shown in FIG. 7.

[0078] Figure 8 A schematic diagram of an image acquisition device provided in another embodiment is shown in FIG. 8.

[0079] Figure 9 A schematic diagram of a defect detection method provided in an embodiment is shown in FIG. 9.

[0080] Figure 10 A schematic diagram of a defect detection method provided in another embodiment is shown in FIG. 10.

[0081] Figure 11 A schematic diagram of a defect detection method provided in another embodiment is shown in FIG. 11.

[0082] Figure 12 A schematic diagram of a defect detection method provided in another embodiment is shown in FIG. 12.

[0083] Figure 13 Flowchart of the defect detection method in another embodiment;

[0084] Figure 14 Flowchart of the defect detection method in another embodiment;

[0085] Figure 15 Flowchart of the defect detection method in another embodiment;

[0086] Figure 16 Flowchart of the defect detection method in another embodiment;

[0087] Figure 17 Flowchart of the defect detection method in another embodiment;

[0088] Figure 18 Flowchart of the defect detection method in another embodiment;

[0089] Figure 19 Flowchart of the defect detection method in another embodiment;

[0090] Figure 20 Structural block diagram of the defect detection device in one embodiment;

[0091] Figure 21 Internal structural diagram of the computer device in one embodiment.

[0092] The reference numerals in the detailed description of the embodiments are as follows:

[0093] 100: defect detection system; 101: support frame;

[0094] 102: camera device; 103: control device;

[0095] 104: host computer; 105: detection station;

[0096] 106: target material; 301: lifting device;

[0097] 302: detection product; 401: two-dimensional light supplementing device;

[0098] 402: sliding assembly; 501: three-dimensional light supplementing device;

[0099] 601: press-fitting device; 701: camera moving shaft;

[0100] 702: two-dimensional light supplementing device moving shaft; 703: two-dimensional light supplementing device;

[0101] 704: camera device; 705: three-dimensional light supplementing device;

[0102] 706: product detection; 707: press fitting device;

[0103] 708: moving direction of camera device; 801: gantry of camera device;

[0104] 802: camera device; 803: projector;

[0105] 804: shadowless light; 805: ring light;

[0106] 806: support of shadowless light. DETAILED DESCRIPTION

[0107] The embodiments of the technical solution of the present application will be described in detail below with reference to the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present application, and therefore only serve as examples, and cannot limit the protection scope of the present application.

[0108] In this paper, the "embodiment" means that the specific features, structures or characteristics described in combination with the embodiment can be contained in at least one embodiment of the present application. The phrase appears at various places in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment to other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as understood by those skilled in the art to which the present application belongs; the terms used herein are only for the purpose of describing specific embodiments and are not intended to limit the present application; the terms "include" and any variations thereof in the specification and claims of the present application and the above description of drawings are intended to cover non-exclusive inclusion.

[0109] In the description of the embodiments of the present application, the technical terms "first", "second", etc. are only used to distinguish different objects, and cannot be understood as indicating or implying relative importance or implicitly indicating the number, specific order or primary and secondary relationship of the indicated technical features. In the description of the embodiments of the present application, the term "and / or" is only a description of the association relationship of the associated objects, which means that there can be three relationships, for example, A and / or B, which can represent the three cases of A existing alone, A and B existing together, and B existing alone. In the description of the embodiments of the present application, the term "multiple" refers to more than two (including two). In the description of the embodiments of the present application, the technical terms "upper", "lower", "horizontal", "inner", etc. indicate the orientation or position relationship shown in the drawings, which is only for the convenience of describing the embodiments of the present application and simplifying the description, and does not indicate or imply that the indicated device or element must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the embodiments of the present application. In the description of the embodiments of the present application, unless otherwise explicitly specified and limited, the technical terms "connected", "connected", etc. should be understood in a broad sense, for example, it can be fixedly connected, or it can be detachably connected, or it can be integrated; it can be mechanically connected, or it can be electrically connected; it can be directly connected, or it can be indirectly connected through an intermediate medium; it can be the internal connection of two elements or the interaction relationship between two elements. For those skilled in the art, the specific meaning of the above terms in the embodiments of the present application can be understood according to the specific circumstances.

[0110] In the power battery system, in order to ensure the sealing performance of the battery after the secondary injection of the battery, the battery needs to be sealed by welding after being pressed and glued. However, after the battery is sealed by welding after being pressed and glued, there may be defects such as poor welding height and broken welding.

[0111] In order to detect the defects after the sealing nail welding, the applicant found that the combination of a two-dimensional camera and a three-dimensional line scanning camera can be used for detecting the defects after the sealing nail welding. However, the combination of a two-dimensional camera and a three-dimensional line scanning camera has some disadvantages, for example, the outer circle of the two-dimensional image collected by the combination of a two-dimensional camera and a three-dimensional line scanning camera is seriously blackened, and the three-dimensional image collected is seriously missing pixels, which leads to a high algorithm misjudgment rate and low detection accuracy. For another example, the two-dimensional camera and the three-dimensional line scanning camera are different cameras, which leads to a high requirement for the algorithm of the spatial alignment of the two-dimensional image and the three-dimensional image, and low defect detection efficiency. For another example, the position of the two-dimensional camera or the three-dimensional line scanning camera needs to be moved when collecting the two-dimensional image and the three-dimensional image, which makes the defect detection not convenient.

[0112] Based on the above considerations, in order to solve the problem of inconvenience, low efficiency and low precision in the process of detecting defects after sealing nail welding, the applicant proposes a defect detection method through the use of a 3D structured light camera to collect two-dimensional images and three-dimensional images of the target material, and to detect defects in the two-dimensional images and three-dimensional images.

[0113] In such a defect detection design, since the 3D structured light camera can collect two-dimensional images and three-dimensional images at the same time, it does not need to move the 3D structured light camera, improving the convenience in the defect detection process. In addition, the image collected by the 3D structured light camera has high precision and does not have the problem of blackening of the outer circle and loss of pixels, ensuring the precision of defect detection. In addition, the 3D structured light camera has been calibrated internally, so that the collected two-dimensional images and three-dimensional images are in the same calibration position, so that the spatial alignment of the two-dimensional images and three-dimensional images has low computational complexity, solving the problem of algorithm processing time timeout caused by the spatial alignment of two-dimensional / three-dimensional images, and improving the defect detection efficiency.

[0114] It should be understood that the technical effects that can be achieved by the defect detection method provided in the embodiments of the present application are not limited to this, but can also achieve other technical effects, for example, solving the problem that the incoming material of the battery cell cannot guarantee that the battery cell is at the same height, resulting in the phenomenon of loss of camera image pixels due to the camera depth problem, thereby causing detection errors, etc. The technical effects that can be achieved by the embodiments of the present application are described in detail below.

[0115] It should be noted that the defect detection scheme provided by the present application is applicable to all related fields of welding, and is compatible with step nail welding and other new process welding, which is not limited in the embodiments of the present application.

[0116] The defect detection method provided in the embodiments of the present application is applied to the defect detection system provided in the embodiments of the present application, and the defect detection system is used to output a defect detection result when a defect is detected in the target material. Wherein, the defect detection system can be realized by software, hardware and their combinations. Therefore, when the process of the defect detection method is described, the defect detection system provided in the embodiments of the present application is described first.

[0117] As Figure 1As shown, the defect detection system 100 comprises a support frame 101, a camera device 102 arranged on the support frame 101, a control device 103 in communication connection with the camera device, an upper computer 104 in communication connection with the control device 103, and a detection station 105 arranged below the support frame; the camera device 102 is a structured light camera device integrating two-dimensional image acquisition function and three-dimensional image acquisition function; the detection station 105 is used for carrying a target material 106 to be detected; the upper computer 104 is used for sending a detection instruction of the target material 106 to the control device 103; the control device 103 is used for controlling the camera device 102 to acquire two-dimensional images and three-dimensional images of the target material, and determining a defect detection result of the target material 106 according to the two-dimensional images and the three-dimensional images of the target material 106.

[0118] In this embodiment, the support frame is used for supporting the camera device, after the control device receives the detection instruction of the target material, the control device controls the camera device to acquire two-dimensional images and three-dimensional images of the target material placed on the detection station, and after the camera device completes acquisition of the two-dimensional images and the three-dimensional images of the target material, the camera device sends the two-dimensional images and the three-dimensional images to the control device, and the control device determines the defect detection result of the target material according to the received two-dimensional images and three-dimensional images of the target material.

[0119] The control device can be, but is not limited to, various personal computers, notebook computers, tablet computers, etc.; the control device and the camera device can be connected through a USB port, and the upper computer and the control device can be connected through a cable.

[0120] The target material is a material to be detected, and the target material can include one detection product or multiple detection products. Accordingly, if the detection product is multiple, the detection station can also be one or multiple, that is, one detection station corresponds to multiple target materials, or one detection station corresponds to one target material.

[0121] Taking the scenario of a power battery system after secondary liquid injection and welding of a battery cell as an example, in order to ensure the sealing performance of the battery after liquid injection, the battery after secondary liquid injection needs to be sealed and welded after glue pressing and nail pressing. In order to verify the sealing performance of the battery, the battery can be subjected to defect detection after sealing and welding of the nail. Therefore, the target material can be a battery cell after sealing and welding of the nail. It should be noted that the target material can also be a battery cell after sealing and welding of the nail after multiple liquid injections.

[0122] So, in the process of battery secondary liquid injection post-sealing nail welding defect detection using the defect detection system provided by the embodiment of the application, the four cells can be welded side by side in the sealing nail welding machine, and then the sealing nail welded cells are placed in the fixture clamp of the turntable. Taking the upper computer as a programmable logic controller (PLC) as an example, the PLC controls the sealing nail welded cells to flow to the detection station with the clamp. During the flow process and after reaching the detection station, the target material can be placed in the fixture clamp of the turntable to ensure the stability of the target material.

[0123] The detection instruction represents that the target material needs to be detected. The detection instruction can include two-dimensional code information of the target material, and the two-dimensional code information can include the number and attribute information of the target material. The two-dimensional code information can uniquely identify the target material. The detection instruction can be sent by the upper computer to the control device. The upper computer can control the target material to reach the preset detection position. When the upper computer controls the target material to reach the detection position, a detection instruction is generated according to the target material, and then the detection instruction is sent to the control device to enable the control device to detect the target material.

[0124] For example, continuing the above example, after the cells flow into the detection station, the PLC sends a signal that the cells have reached the detection station to the control device. When the control device receives the signal, it means that the detection instruction of the target material has been received. At this time, the control device will control the camera device to collect the two-dimensional image and the three-dimensional image of the target material.

[0125] The two-dimensional image represents two-dimensional plane image information in the pixel scale. The three-dimensional image collected by the 3D structured light camera is a depth image. Each pixel of the depth image reflects the distance from the target material to the 3D structured light camera. For example, the format of the three-dimensional image can be tif format.

[0126] The camera device is a structured light camera device integrating two-dimensional image acquisition function and three-dimensional image acquisition function. Optionally, the camera device can be a 3D structured light camera.

[0127] The 3D structured light camera mainly includes an infrared camera and a laser dot array projector. The laser dot array projector is used to project a structured light pattern (speckle pattern) to the target scene, and the infrared camera collects the infrared structured light image of the target. Since the 3D structured light camera uses multiple gratings for encoding, the encoding accuracy can be as fine as one pixel or even sub-pixel in principle, and the point cloud quality and accuracy are higher than those of other principles. Moreover, the 3D structured light camera collects the three-dimensional image of the detection product at the current detection station according to the phase shift method, and compared with the three-dimensional line scanning laser, the phase shift method in the 3D structured light camera has a larger projection dimension, which ensures that the 3D structured light camera can obtain more material information, greatly improving the image taking speed and image quality.

[0128] Further, the internal part of the 3D structured light camera has been calibrated, avoiding the re-calibration process and reducing the process flow. Moreover, the 3D structured light camera is compatible with various welding process products, such as step nails (welding height is higher than the welding height of ordinary nails), and has higher compatibility.

[0129] Moreover, the image taking module in the 3D structured light has more convenient and fast data management and configuration management, and the parameters are easy to change.

[0130] After the camera device collects the two-dimensional image and the three-dimensional image of the target material, the two-dimensional image and the three-dimensional image of the target material can be sent to the control device. The control device detects the two-dimensional image and the three-dimensional image of the target material to determine the defect detection result of the target material.

[0131] Taking the scenario of battery secondary liquid injection in a power battery system as an example, the general requirements for the sealing nail welding include: no un-welded, melted nail, broken welding sealing nail is allowed; the welding height is within 0.1 mm of the specified height; defects such as pinhole, burst point, and offset welding need to be within the specified range.

[0132] Therefore, the defect detection result of the battery cell can be the defect category of the target material after the sealing nail welding, such as un-welded, melted nail, broken welding, pinhole, burst point, and offset welding. As shown in FIG. 1, Figure 2 (a) of FIG. 1 shows a schematic diagram of un-welded battery cell, Figure 2 (b) of FIG. 1 shows a schematic diagram of successfully welded battery cell, and Figure 2 (c) of FIG. 1 shows a schematic diagram of broken welding battery cell. Figure 2

[0133] ​In the embodiment of the present application, the support frame is designed in the defect detection system, the camera equipment is arranged on the support frame, the control equipment is in communication connection with the camera equipment, the upper computer is in communication connection with the control equipment, and the detection station is arranged below the support frame. When the target material to be detected reaches the detection station, the upper computer sends a detection instruction of the target material to the control equipment, and then the control equipment controls the camera equipment to collect two-dimensional images and three-dimensional images of the target material, and determines a defect detection result of the target material according to the two-dimensional images and the three-dimensional images of the target material. In the system, the camera equipment is a structured light camera equipment integrating two-dimensional image collection function and three-dimensional image collection function. In the process of collecting two-dimensional images and three-dimensional images, the two-dimensional images and the three-dimensional images can be collected without switching the camera equipment, which ensures the convenience of collecting images, thereby improving the convenience of defect detection of the target material. Moreover, the two-dimensional images and the three-dimensional images are collected by the same camera equipment, which ensures that the collected two-dimensional images and three-dimensional images are in the same coordinate system, so that when the two-dimensional images and the three-dimensional images are used for defect detection, there is no need for too many complex data unification operations for pre-processing of the two images, thereby improving the efficiency of defect detection of the target material. In addition, the target material is detected by two-dimensional images and three-dimensional images of different dimensions, which ensures the comprehensiveness of target material defect detection, avoids the misjudgment problem caused by single image detection of two-dimensional images or three-dimensional images, and improves the accuracy of defect detection.

[0134] In the following, it is assumed that the target material includes a plurality of detection products, the defect detection system includes a plurality of detection stations, and one detection station corresponds to one detection product. In one embodiment, as shown in Figure 3 The system 100 further includes a lifting device 301. The lifting device 301 is arranged on the support frame, and the camera equipment is installed on the lifting device 301. The support frame 101 includes a plurality of detection stations 105 below. The control equipment 103 is used to control the lifting device 301 to move the camera equipment 102 to the position of each detection station 105, so as to collect two-dimensional images and three-dimensional images of the detection products 302 in each detection station 105.

[0135] It should be noted that, Figure 2 The detection station in the embodiment is four, and the corresponding detection product is four. In actual application, the number of detection stations can be set according to requirements. Moreover, Figure 3 The detection product in the embodiment is a cylinder, but in actual application, the shape of the detection product is not limited.

[0136] In actual application, the target material can refer to a material including a plurality of detection products. In this case, each detection product is in a different detection station.

[0137] In the scenario of multiple detection stations, the camera device collects two-dimensional images and three-dimensional images of each detection product, which can be collected in sequence according to the order of the detection stations. For example, if there are four detection stations and each detection station contains four detection products, the control device can control the lifting device to control the camera device to collect two-dimensional images and three-dimensional images of the detection products in the first detection station first, and then collect two-dimensional images and three-dimensional images of the detection products in the second detection station, and so on, until all the two-dimensional images and three-dimensional images of the detection products in all the detection stations are collected.

[0138] Specifically, the control device collects two-dimensional images and three-dimensional images of the detection products in each detection station in sequence. When the control device receives the detection instruction sent by the upper computer, it sends a collection instruction to the camera device to control the camera device to collect two-dimensional images and three-dimensional images of the detection products in the first station. At this time, the camera device is at the position corresponding to the first detection station. After the camera device collects the detection products in the first detection station, it sends a collection completion instruction to the control device. The control device controls the lifting device to move the camera device to the position of the second detection station to continue collecting two-dimensional images and three-dimensional images of the detection products in the second detection station. Until all the two-dimensional images and three-dimensional images of the detection products in all the detection stations are collected, the control device controls the lifting device to move the camera device to the position of the first detection station.

[0139] The lifting device can move up and down and left and right to control the camera device to move to the position of each detection station, so that the camera device can accurately collect each detection product.

[0140] Alternatively, multiple detection products can be placed in one tooling fixture of the turntable, or one detection product can correspond to one tooling fixture.

[0141] Continuing with the example of detecting the battery after secondary liquid injection and welding, the control device can control the lifting device to move the camera device to the position of each battery to collect two-dimensional images and three-dimensional images of each battery. For example, if there are four batteries, the control device can control the lifting device to move the camera device to the position of the first battery to collect two-dimensional images and three-dimensional images of the first battery, and then move the camera device to the position of the second battery to collect two-dimensional images and three-dimensional images of the second battery, and so on, until all the two-dimensional images and three-dimensional images of the batteries are collected.

[0142] Optionally, after the two-dimensional image and the three-dimensional image of the last battery cell are collected, the control device can control the lifting device to move the camera device to the position of the first battery cell, i.e., the first detection station.

[0143] In the embodiment of the present application, the lifting device is designed on the support frame in the defect detection system, and the camera device is installed on the lifting device. The support frame below includes a plurality of detection stations. The camera device can be moved to the position of each detection station by the control device to collect the two-dimensional image and the three-dimensional image of the detection product in each detection station. In the system, the camera device can be moved by the lifting device to collect the images of a plurality of detection products, ensuring the convenience of the target material for defect detection. Moreover, after the camera device is moved, each detection product is in the field of view of the camera device when the image is collected, and the incomplete image collection does not occur.

[0144] In one embodiment, as shown in FIG. 1, the system 100 further includes a sliding assembly 401 and a two-dimensional light supplement device 402. The sliding assembly 401 is arranged on the support frame 201 below the lifting device 301, and the two-dimensional light supplement device 402 is arranged on the sliding assembly 401. The control device 103 is used to control the camera device 102 to collect the two-dimensional image of the detection product 302 in each detection station 105 with the two-dimensional light supplement device 402 turned on. After the two-dimensional image of the detection product 302 in each detection station 105 is collected, the control device 103 sends a moving instruction to the upper computer 104. The moving instruction is used to instruct the upper computer 104 to move the two-dimensional light supplement device 402 to the next detection station 105 through the sliding assembly 401. Figure 4 The camera device collects the two-dimensional image of the detection product in each detection station with the two-dimensional light supplement device turned on. The camera device collects the two-dimensional image of the detection product in each detection station in turn. For example, there are four detection stations, and there are four detection products in each detection station. The two-dimensional light supplement device is moved to the corresponding detection station through the sliding assembly. The control device controls the camera device to collect the two-dimensional image of the detection product in the first detection station with the two-dimensional light supplement device turned on. After the two-dimensional image of the detection product in the first detection station is collected, the camera device sends a moving instruction to the upper computer. The upper computer controls the two-dimensional light supplement device to move to the second detection station. When the detection product in the second detection station is completed, the upper computer controls the two-dimensional light supplement device to move to the third detection station. In this way, the two-dimensional image and the three-dimensional image of the detection product in all detection stations are collected in turn.

[0145]

[0146] ​The two-dimensional supplementary lighting device can move on the sliding component to ensure that it can be moved to each detection station; the host computer and the two-dimensional supplementary lighting device can be connected by a cable; the two-dimensional supplementary lighting device and the control device can be connected by a cable.

[0147] Alternatively, the two-dimensional supplemental lighting device can be a shadowless light source.

[0148] In this embodiment, a sliding component and a two-dimensional supplementary lighting device are designed in the defect detection system. The two-dimensional supplementary lighting device is set on the sliding component, so that after the camera device acquires a two-dimensional image of the product being inspected at the current inspection station, the two-dimensional supplementary lighting device is moved to the next inspection station by the sliding component. This allows the camera device to acquire images of the next inspection station while the two-dimensional supplementary lighting device is on, avoiding the problem of unclear images caused by insufficient lighting when acquiring two-dimensional images. This improves the clarity of the two-dimensional images of the products being inspected at each inspection station and ensures the accuracy of subsequent defect detection.

[0149] In one embodiment, such as Figure 5 As shown, the system 100 also includes a three-dimensional supplementary lighting device 501, which is set at a preset distance above each inspection station 105; and a control device 103, which controls the camera device 102 to acquire three-dimensional images of the inspection product 302 on the corresponding inspection station 105 when the three-dimensional supplementary lighting device 501 is turned on.

[0150] The control equipment controls the camera equipment to acquire three-dimensional images of the products being inspected at the corresponding inspection station when the three-dimensional supplementary lighting equipment is turned on. The three-dimensional supplementary lighting equipment can be turned on when the control equipment receives the inspection instruction for the target material and sends an activation command to the three-dimensional supplementary lighting equipment, or when the host computer sends an activation command to the three-dimensional supplementary lighting equipment after the camera equipment has acquired two-dimensional images, so that the three-dimensional supplementary lighting equipment can be turned on.

[0151] Based on the above embodiment, the camera equipment collects two-dimensional images of the detection products in each detection station when the two-dimensional light supplementing equipment is turned on, and collects three-dimensional images of the detection products in the corresponding detection station when the three-dimensional light supplementing equipment is turned on. For example, there are four detection stations, and there are four detection products in each detection station. The two-dimensional light supplementing equipment moves to the corresponding detection station through the sliding assembly. The control equipment controls the camera equipment to collect the two-dimensional images of the detection products in the first detection station when the two-dimensional light supplementing equipment is turned on. After the two-dimensional images of the detection products in the first detection station are collected, the camera equipment sends a moving instruction to the upper computer, the upper computer controls the two-dimensional light supplementing equipment to move to the second detection station, and the camera equipment collects the three-dimensional images of the detection products in the first detection station when the three-dimensional light supplementing equipment is turned on. After the camera equipment collects the two-dimensional images and the three-dimensional images of the detection products in the first detection station, the control equipment controls the camera equipment to move to the second detection station through the lifting device. Similarly, the camera equipment collects the two-dimensional images and the three-dimensional images of the detection products in the second detection station, and collects the two-dimensional images and the three-dimensional images of the detection products in all detection stations one by one.

[0152] The three-dimensional light supplementing equipment and the control equipment can be connected through a cable.

[0153] In the defect detection system, the three-dimensional light supplementing equipment is arranged at a distance above each detection station in the embodiment of the application. When the camera equipment collects each detection station, the camera equipment can collect the three-dimensional images under the condition that the three-dimensional light supplementing equipment is turned on, which avoids the problem of unclear images caused by uneven or insufficient light when collecting three-dimensional images, ensures the clarity of the three-dimensional images, and ensures the accuracy of subsequent defect detection.

[0154] In one embodiment, as shown in Figure 6 The system 100 further includes a press fitting device 601, which is in communication connection with the upper computer 303. The upper computer 104 is further configured to detect that the target material 106 enters the detection station, and send a detection instruction to the control equipment 103 after the press fitting device 601 performs a pressing operation on the target material 106. The coverage area of the press fitting device 601 is greater than or equal to the upper surface area of the target material 106.

[0155] When the upper computer detects that the target material enters the detection station, the upper computer controls the press fitting device to perform a pressing operation on the target material. After the pressing operation, the upper computer sends a detection instruction to the control equipment. The control equipment and the upper computer can be connected through a network cable.

[0156] The down-pressing operation is used to unify the height of all the detected products in the target material, so that all the detected products are in the same plane, and the phenomenon of image missing pixels caused by the camera depth of field is greatly avoided.

[0157] The pressing device can be a cover plate, and the coverage area of the pressing device is greater than or equal to the upper surface area of the target material, so as to ensure that the pressing device can accurately press the target material, and avoid the situation that the height of the detected products in the target material is not uniform due to the coverage area of the pressing device being smaller than the target material.

[0158] In the embodiment of the present application, the system further designs a pressing device. After the target material enters the detection station, the pressing device is used to press the target material, so that the height of the multiple detected products can be unified when the target material includes multiple detected products, and the multiple detected products are in the same plane, thereby avoiding the phenomenon of image missing pixels caused by the camera depth of field.

[0159] In one embodiment, continuing to refer to Figure 6 , the pressing device 601 is provided with a cavity structure at a position corresponding to each detection station 105, and the three-dimensional light supplementing device 501 is a ring light device arranged around the inside of the cavity; the cavity structure is used for the camera device 102 to align the detected product 201 on each detection station 105.

[0160] The pressing device is provided with a cavity structure corresponding to the detection station, and the three-dimensional light supplementing device is arranged around the inside of the cavity, which is a ring light device.

[0161] One cavity structure corresponds to one detection station, so that the camera device can collect the detected product through the cavity, and one three-dimensional light supplementing device is arranged inside one cavity, thereby avoiding the problem that the three-dimensional image collected by the camera device is unclear due to light problems.

[0162] In the embodiment of the present application, the pressing device in the system is provided with a cavity structure at a position corresponding to each detection station, so that the camera device can collect two-dimensional images and three-dimensional images of the target material through the cavity of the pressing device, and the three-dimensional light supplementing device is arranged on the ring light device around the inside of the cavity, so that the camera device can clearly collect the image of each detected product when collecting the image of the target material, thereby improving the accuracy of the image.

[0163] As shown in Figure 7 and Figure 8 , Fig. 1 is a motion diagram in the image collection process, Figure 7 Fig. 2 is a hardware structure diagram of collecting images, Figure 8 Figure 7 ​701 represents a camera movement axis, 702 represents a two-dimensional light supplement device movement axis, 703 represents a two-dimensional light supplement device, 704 represents a camera device, four arrows respectively represent four three-dimensional light sources, 705 represents a three-dimensional light supplement device, 706 represents a detected product, 707 represents a press-fitting device, and 708 represents a camera device movement direction.

[0164] Figure 8 801 in FIG. 8 represents a camera gantry, 802 represents a camera device, 803 represents a projector (two-dimensional light supplement device), 804 represents shadowless light, 805 represents ring light (three-dimensional light supplement device), and 806 represents a shadowless light support. Figure 8 In the application, the sealing pin detection accuracy is 0.006 mm, a single 3D structured light camera is used, the camera is vertically installed and can be adjusted up and down and left and right, can meet the detection field of view of the object distance of 116±12.5 mm, the camera can be adjusted through a lifting device, according to the compatibility of the detected product, a formula is configured, the camera acquisition distance can be adjusted by one key, the camera imaging is clear; and the light source installation angle range is increased and the camera can move up and down and left and right, the adaptability is stronger, the adjustment is easier, so more products can be compatible.

[0165] Based on the same inventive concept, the embodiments of the present application also provide a defect detection method for implementing the defect detection system described above. The implementation scheme for solving the problem provided by the method is similar to the implementation scheme described in the system above. Therefore, the specific limitations in one or more defect detection method embodiments provided below can refer to the limitations of the defect detection system described above, and will not be repeated here.

[0166] In one embodiment, a defect detection method is provided for application to Figure 1 For example, the defect detection system in FIG. 8, the embodiment relates to the specific process that the detection device controls the camera device to collect the two-dimensional image and the three-dimensional image of the target material in response to the detection instruction of the target material, and determines the defect detection result of the target material according to the two-dimensional image and the three-dimensional image of the target material, as shown in FIG. 8, the embodiment includes the following steps: Figure 9

[0167] S901, in response to the detection instruction of the target material, controlling the camera device to collect the two-dimensional image and the three-dimensional image of the target material; the camera device is a structured light camera device integrating two-dimensional image collection function and three-dimensional image collection function.

[0168] When the control device receives the detection instruction of the target material, the control device controls the camera device to collect the two-dimensional image and the three-dimensional image of the target material.

[0169] ​Specifically, when the control device receives the detection instruction of the target material, the control device generates a collection instruction for the camera device and sends the collection instruction to the camera device. After the camera device receives the collection instruction, the camera device collects the two-dimensional image and the three-dimensional image of the target material.

[0170] Optionally, the camera device can collect the two-dimensional image and the three-dimensional image of the target material in the following manners: first, the two-dimensional image is collected, and then the three-dimensional image is collected after the collection of the two-dimensional image is completed; second, the three-dimensional image is collected first, and then the two-dimensional image is collected after the collection of the three-dimensional image is completed; or third, the two-dimensional image and the three-dimensional image are collected simultaneously.

[0171] Meanwhile, in order to avoid the camera device from being polluted by the external environment, for example, dust on the lens of the camera device, optionally, in the embodiment of the present application, a protective cover is arranged outside the camera device, which can prevent the camera device from being polluted by the external environment, and the protective cover can also prevent the emitted laser from causing harm to the human eye.

[0172] The protective cover can be sleeved in front of the lens of the camera device, and the protective cover can be made of metal, hard plastic, soft rubber or other materials, and the protective cover can be in the form of a cylinder, a petal, a square or other shapes.

[0173] S902, determining a defect detection result of the target material according to the two-dimensional image and the three-dimensional image of the target material.

[0174] After the camera device collects the two-dimensional image and the three-dimensional image of the target material, the camera device can send the two-dimensional image and the three-dimensional image of the target material to the control device, and the control device detects the two-dimensional image and the three-dimensional image of the target material to determine the defect detection result of the target material.

[0175] Specifically, the two-dimensional image and the three-dimensional image can be detected simultaneously to obtain a two-dimensional detection result corresponding to the two-dimensional image and a three-dimensional detection result corresponding to the three-dimensional image, and the defect detection result of the target material can be determined according to the two-dimensional detection result and the three-dimensional detection result. Alternatively, the two-dimensional detection result and the three-dimensional detection result can be directly determined as the defect detection result of the target material.

[0176] In one embodiment, the two-dimensional image and the three-dimensional image of the target material can be detected by a preset neural network model. Specifically, the two-dimensional image and the three-dimensional image are taken as inputs of the neural network model, and the neural network model directly outputs the detection result of the target material through analysis.

[0177] In another embodiment, the two-dimensional image and the three-dimensional image can be detected by a preset two-dimensional detection model and a three-dimensional detection model respectively. Specifically, the two-dimensional image is taken as an input of the two-dimensional detection model, the two-dimensional detection model analyzes the two-dimensional image, and a two-dimensional detection result corresponding to the two-dimensional image is determined; the three-dimensional image is taken as an input of the three-dimensional detection model, the three-dimensional detection model analyzes the three-dimensional image, and a three-dimensional detection result corresponding to the three-dimensional image is determined.

[0178] In the above defect detection method, in response to a detection instruction of the target material, the camera device is controlled to collect a two-dimensional image and a three-dimensional image of the target material, and a defect detection result of the target material is determined according to the two-dimensional image and the three-dimensional image of the target material. In this method, because the camera device is a structured light camera device integrating a two-dimensional image collection function and a three-dimensional image collection function, the two-dimensional image and the three-dimensional image can be collected without switching the camera device, which ensures the convenience of collecting images, thereby improving the convenience of defect detection of the target material. Moreover, the two-dimensional image and the three-dimensional image are collected by the same camera device, which ensures that the collected two-dimensional image and three-dimensional image are in the same coordinate system, so that the two images do not need to be preprocessed by too many complex data unification operations when the two images are used for defect detection, thereby improving the efficiency of defect detection of the target material. In addition, the two-dimensional image and the three-dimensional image of the target material are used for defect detection of the target material, which ensures the comprehensiveness of the target material defect detection and avoids the misjudgment problem caused by single image detection of the two-dimensional image or the three-dimensional image, thereby improving the accuracy of defect detection.

[0179] In actual application, the target material can refer to a material including a plurality of detection products, in which case the detection products are in different detection stations. Based on this, how to control the camera device to collect two-dimensional images and three-dimensional images of the detection products in the target material in different detection stations is described in detail below. Figure 10 As shown in FIG. 10, the process of controlling the camera device to collect the two-dimensional image and the three-dimensional image of the target material includes the following steps:

[0180] S1001, the camera device is controlled to collect a two-dimensional image and a three-dimensional image of a detection product in a current detection station.

[0181] S1002, after the two-dimensional image and the three-dimensional image of the detection product in the current detection station are collected, the camera device is controlled to move to a next detection station to collect a two-dimensional image and a three-dimensional image of a detection product in the next detection station, so as to collect two-dimensional images and three-dimensional images of all detection products.

[0182] Continuing with the example of detecting products that are secondary injection after welding, multiple battery cells can be welded simultaneously side by side using a sealing nail welding machine. After welding, the multiple battery cells after sealing nail welding are placed in the fixture clamps of the turntable. The host computer controls the movement of the multiple battery cells after welding to the detection stations, with one battery cell corresponding to one detection station. One fixture clamp can correspond to one battery cell, and one fixture clamp can also correspond to multiple battery cells.

[0183] When the host computer moves the multiple battery cells to their respective detection stations, it sends a detection instruction to the control device. The control device controls the camera device to collect two-dimensional images and three-dimensional images of the battery cells according to the detection instruction. Taking the example of four battery cells, the camera device can sequentially collect the detection products in the detection stations. Specifically, taking the first detection station as the current detection station, the camera device first collects the two-dimensional images and three-dimensional images of the battery cells in the first detection station. After the two-dimensional images and three-dimensional images of the battery cells in the first detection station are collected, the control device controls the camera device to move to the second detection station, and the camera device collects the two-dimensional images and three-dimensional images of the battery cells in the second detection station. This process continues until the camera device collects the two-dimensional images and three-dimensional images of the battery cells in all detection stations. The control device and the host computer can communicate through a network.

[0184] When the camera device collects the two-dimensional images and three-dimensional images of the detection products in the current detection station, it sends a signal to the control device indicating that the image collection is complete. The control device generates a control instruction based on the signal and sends it to the camera device, instructing the camera device to move to the next detection station via the lifting device and collect the two-dimensional images and three-dimensional images of the detection products in the next detection station. This process continues until the two-dimensional images and three-dimensional images of the detection products in all detection stations are collected.

[0185] It should be noted that the application uses a single camera device. Considering the size variation of the detection products, the camera device is installed on a lifting device that can move up and down and left and right.

[0186] Optionally, the order in which the camera device collects the detection products in each detection station can be pre-set, and can be sequential collection of the detection products in the detection stations or a detection order determined according to certain rules.

[0187] In one embodiment, the camera device is a 3D structured light camera. The 3D structured light camera has higher camera precision. It uses the phase shift method structured light technology to have a larger projection dimension, ensuring that the camera can capture more material information, greatly improving the image capture speed and image quality. The 3D structured light camera is compatible with multiple welding process products using 3D structured light, ensuring the compatibility of different material defect detection.

[0188] In the defect detection method, the camera equipment is controlled to collect two-dimensional images and three-dimensional images of the detection product at the current detection station, and after the two-dimensional images and three-dimensional images of the detection product at the current detection station are collected, the camera equipment is controlled to move to the next detection station to collect two-dimensional images and three-dimensional images of the detection product at the next detection station, so as to collect two-dimensional images and three-dimensional images of all detection products. In this method, the camera equipment can be moved to collect images of multiple detection products, ensuring the convenience of defect detection of the target material. Moreover, after the camera equipment is moved, each detection product is in the field of view of the camera equipment when the image is collected, which also improves the accuracy of defect detection of the target material.

[0189] The following will describe how to control the camera equipment to collect two-dimensional images and three-dimensional images of the detection product at the current detection station through an embodiment. In an embodiment, as shown in FIG. 1, the camera equipment is controlled to collect two-dimensional images and three-dimensional images of the detection product at the current detection station, including the following steps. Figure 11

[0190] S1101, the camera equipment is controlled to collect two-dimensional images of the detection product at the current detection station under the condition that the two-dimensional light supplement equipment is turned on, and after the two-dimensional images are collected, a moving instruction is sent to the upper computer, the moving instruction being used to instruct the upper computer to move the two-dimensional light supplement equipment to the next detection station.

[0191] S1102, the camera equipment is controlled to collect three-dimensional images of the detection product at the current detection station under the condition that the three-dimensional light supplement equipment is turned on.

[0192] After the control device receives the detection instruction of the target material, the control device will first send a trigger signal to the two-dimensional light supplement equipment and the three-dimensional light supplement equipment to turn on the two-dimensional light supplement equipment and the three-dimensional light supplement equipment. The control device controls the camera equipment to collect two-dimensional images of the detection product at the current detection station under the condition that the two-dimensional light supplement equipment is turned on.

[0193] Alternatively, the trigger signal received by the two-dimensional light supplement equipment and the three-dimensional light supplement equipment can also be sent by the upper computer. Specifically, when the upper computer sends a detection instruction to the control device, the upper computer can also send a trigger signal to the two-dimensional light supplement equipment and the three-dimensional light supplement equipment to control the two-dimensional light supplement equipment and the three-dimensional light supplement equipment to turn on.

[0194] After the camera equipment collects two-dimensional images of the detection product at the current detection station, a moving instruction is sent to the upper computer to instruct the upper computer to move the two-dimensional light supplement equipment to the next detection station. Specifically, after the upper computer receives the moving instruction, the upper computer will send a moving signal to the two-dimensional light supplement equipment according to the moving instruction, so that the two-dimensional light supplement equipment moves to the next detection station according to the moving signal.​

[0195] Then, the control device controls the camera device to collect the three-dimensional image of the detected product on the current detection station under the condition that the three-dimensional light supplement device is turned on.

[0196] The two-dimensional light supplement device can include one, and the two-dimensional light supplement device is installed in the sliding assembly to move the two-dimensional light supplement device in different directions, so that the camera device collects the two-dimensional image of each detection station through the two-dimensional light supplement device.

[0197] The three-dimensional light supplement device can include multiple, and one detection station corresponds to one three-dimensional light supplement device; the three-dimensional light supplement device can be a ring light.

[0198] In the above defect detection method, the camera device is controlled to collect the two-dimensional image of the detected product on the current detection station under the condition that the two-dimensional light supplement device is turned on, and after the two-dimensional image is collected, a moving instruction is sent to the upper computer to instruct the upper computer to move the two-dimensional light supplement device to the next detection station, and then the camera device is controlled to collect the three-dimensional image of the detected product on the current detection station under the condition that the three-dimensional light supplement device is turned on. In this method, the two-dimensional light supplement device is used for light supplement when collecting the two-dimensional image, and the three-dimensional light supplement device is used for light supplement when collecting the three-dimensional image, so that the light state of the environment when collecting the two-dimensional image and the three-dimensional image is in the best state, improving the clarity and information comprehensiveness of the collected two-dimensional image and three-dimensional image of the detected product, realizing the effect of high-precision full-edge image acquisition of the detected product, and ensuring the accuracy of subsequent defect detection.

[0199] Based on the detection instruction of the response target material in the above embodiment, the sending condition process of the detection instruction will be described through an embodiment. In one embodiment, the detection instruction of the target material is sent after the upper computer detects that the target material enters the detection station and controls the press-fit device to perform a pressing operation on the target material. The coverage area of the press-fit device is greater than or equal to the upper surface area of the target material.

[0200] When the upper computer detects that the target material enters the detection station, the press-fit device is controlled to perform a pressing operation on the target material. After the pressing operation is completed, the upper computer generates a detection instruction of the target material, and then sends the detection instruction to the detection device.

[0201] After the press-fit device completes the pressing operation on the target material, a pressing success signal can be fed back to the upper computer. The upper computer generates a detection instruction in response to the pressing success signal, and sends the detection instruction to the control device, so that the control device performs defect detection on the target material.

[0202] In the above defect detection method, the target material is pressed down by the pressing device. For the target material, especially the material including multiple detection products, the height of all detection products in the target material can be unified, so that all detection products are ensured to be in the same plane, and the phenomenon of image pixel loss caused by the camera depth of field is avoided.

[0203] In the above embodiments, how to obtain the two-dimensional image and the three-dimensional image of the target material is described. Next, how to perform defect detection analysis based on the two-dimensional image and the three-dimensional image of the target material is described through an embodiment. In an embodiment, as shown in FIG. 13, a defect detection result of the target material is determined based on the two-dimensional image and the three-dimensional image of the target material, including the following steps. Figure 12

[0204] S1201, performing spatial alignment processing on the two-dimensional image and the three-dimensional image.

[0205] Before performing defect detection on the two-dimensional image and the three-dimensional image, in order to reduce the time calculation complexity and improve the calculation efficiency, the two-dimensional image and the three-dimensional image are usually subjected to spatial alignment processing, that is, the two-dimensional image and the three-dimensional image are unified in the same coordinate system.

[0206] Taking a 3D structured light camera as an example, before collecting an image, the 3D structured light camera will calibrate the interior of the camera to obtain an intrinsic matrix, a rotation matrix, a translation matrix, and a homography matrix of the 3D structured light camera. Therefore, the homography matrix of the 3D structured light camera can be directly used to perform spatial alignment of the two-dimensional image and the three-dimensional image.

[0207] In another embodiment, the way of performing spatial alignment of the two-dimensional image and the three-dimensional image can also be to use the intrinsic matrix, the rotation matrix, and the translation matrix of the camera to achieve spatial alignment of the two-dimensional image and the three-dimensional image. For example, taking alignment of the two-dimensional image to the three-dimensional image as an example, formula (1) is used to achieve spatial alignment of the two-dimensional image to the three-dimensional image.

[0208]

[0209] wherein the coordinates of the two-dimensional image are (u, v), the coordinates corresponding to the image point in the coordinate system with the two-dimensional image as the origin are (x, y, z), K, R, and T represent the intrinsic matrix, the rotation matrix, and the translation matrix of the camera respectively, and s = z represents the scale.

[0210] ​In another embodiment, the spatial alignment of the two-dimensional image and the three-dimensional image can also be performed by using a preset spatial alignment model, inputting the two-dimensional image and the three-dimensional image into the spatial alignment model, and analyzing the two-dimensional image and the three-dimensional image by using the spatial alignment model to obtain the two-dimensional image and the three-dimensional image after spatial alignment.

[0211] The defect detection method in the prior art is a combination of a two-dimensional camera and a three-dimensional line scanning camera. Since the two-dimensional camera and the three-dimensional line scanning camera are two cameras, the two-dimensional image and the three-dimensional image are not in the same spatial position. Therefore, when the two-dimensional image and the three-dimensional image in the prior art are aligned, the built-in parameters of the two-dimensional camera and the three-dimensional line scanning camera need to be considered, and the alignment algorithm has a high requirement and high complexity. In the present application, the two-dimensional image and the three-dimensional image collected by the 3D structured light camera are in the same spatial position. Therefore, when the two-dimensional image and the three-dimensional image are aligned, only the calibration parameters of the 3D structured light camera need to be considered. Therefore, a simple alignment algorithm can be used to align the two-dimensional image and the three-dimensional image.

[0212] In S1202, the two-dimensional image and the three-dimensional image after spatial alignment are processed by using a preset deep learning algorithm to obtain a defect detection result of the target material.

[0213] The deep learning algorithm can be used to construct a deep learning model. The two-dimensional image and the three-dimensional image can be directly input into the deep learning model to obtain the defect detection result of the target material.

[0214] It should be noted that the deep learning algorithm in the present embodiment can include but is not limited to convolutional neural network, recurrent neural network, generative adversarial network, and deep reinforcement learning.

[0215] In the above defect detection method, the two-dimensional image and the three-dimensional image are spatially aligned, and then the two-dimensional image and the three-dimensional image after spatial alignment are processed by using a preset deep learning algorithm to obtain a defect detection result of the target material. In this method, the two-dimensional image and the three-dimensional image are first spatially aligned, which improves the convenience of subsequent defect detection of the two-dimensional image and the three-dimensional image after spatial alignment and reduces the computational complexity. Since the two-dimensional image and the three-dimensional image are collected by the same camera device, the two-dimensional image and the three-dimensional image do not need to be spatially aligned by using a complex algorithm. Therefore, the time for spatial alignment of the two-dimensional image and the three-dimensional image can be reduced, the spatial processing efficiency is improved, and the efficiency of the entire defect detection process is improved.

[0216] With the scenario of battery secondary liquid injection in a power battery system as an example, the welded battery cell can have defects such as incomplete welding, broken welding, high badness, and burst hole. According to these defects, the defects of the battery cell can be divided into defects containing height information and defects not containing height information. Therefore, the defects of the target material can be further divided into defects containing height information and defects not containing height information, and the defects not containing height information are determined as the first defects, and the defects containing height information are determined as the second defects. The defect detection result includes the detection result of the first defects and the detection result of the second defects. The following will describe how to determine the detection result of the first defects and the detection result of the second defects through an embodiment. In an embodiment, as shown in Figure 13 The defect detection result of the target material is obtained by processing the spatially aligned two-dimensional image and three-dimensional image through a preset deep learning algorithm, including the following steps:

[0217] S1301, obtaining a first detection region in the two-dimensional image and a second detection region in the three-dimensional image.

[0218] The collected two-dimensional image and three-dimensional image include a plurality of pixel points, but not all pixel points are equally important. Some pixel points are not valuable for the detection result, and some pixel points will affect the detection result. Therefore, it is necessary to extract the key region in the image and discard the region without value and with influence on the detection result. On the one hand, it can reduce the data amount and improve the processing speed, and on the other hand, removing the redundant region is beneficial to obtain more accurate detection result.

[0219] In order to improve the detection efficiency and accuracy, the region of interest (ROI) in the image can be extracted before the image is detected. Extracting the region of interest is to select an image region from the image, and this region is the focus of image detection. The region can be a square, a circle, an ellipse, an irregular polygon, etc.

[0220] Therefore, before the two-dimensional image and the three-dimensional image are detected, the first detection region of the two-dimensional image and the second detection region of the three-dimensional image can be extracted. The first detection region and the second detection region can be the region of interest in the two-dimensional image and the three-dimensional image.

[0221] The first detection region of the two-dimensional image can be obtained through a region detection model. Specifically, the two-dimensional image is input into the region detection model, and finally the first detection region of the two-dimensional image is output.

[0222] In an embodiment, as shown in Figure 14 The specific process of obtaining the second detection region in the three-dimensional image can include:

[0223] S1401, pre-process the three-dimensional image to obtain point cloud data of the target material.

[0224] The manner of obtaining the point cloud data of the target material according to the three-dimensional image can be that the point cloud data is obtained by coordinate conversion calculation of the three-dimensional image. Specifically, the conversion from the three-dimensional image to the point cloud data can be performed according to formula (2).

[0225]

[0226] wherein x, y, z are point cloud coordinates, x', y' are image coordinates, and D is a depth value.

[0227] The point cloud data often contains a large number of hash points and isolated points. Therefore, after obtaining the point cloud data of the target material, a point cloud filtering method can be used to perform point cloud filtering processing on the point cloud data, that is, to filter out the noise in the point cloud data.

[0228] S1402, determining a second detection area according to the point cloud data.

[0229] The second detection area of the point cloud data can be determined by using a feature extraction method; for example, the geometric features of the point cloud data are extracted, and the second detection area of the point cloud data is extracted by using the geometric feature distribution and the particle filtering framework; the second detection area can also be obtained by using a three-dimensional point cloud key point extraction algorithm.

[0230] In this embodiment, the point cloud data of the target material is obtained by pre-processing the three-dimensional image, and the second detection area is determined according to the point cloud data. In this method, the second detection area is determined by the point cloud data corresponding to the three-dimensional image of the target material, which improves the accuracy of the extraction of the second detection area.

[0231] S1302, detecting a first defect detection result according to the first detection area.

[0232] The first detection area is obtained according to a two-dimensional image, and the two-dimensional image is a two-dimensional planar image. Therefore, the first detection area can be used to detect defects that do not contain height information; for example, the first defect can be an un-welded, broken weld, or a non-sealing pin defect; therefore, the first detection result can be directly determined by the visual appearance of the first detection area.

[0233] In one embodiment, as shown in FIG. 13B, detecting a first defect detection result according to the first detection area includes the following steps: Figure 15

[0234] S1501, obtaining a planar feature of the first detection area.

[0235] The image planar feature mainly includes color features, texture features, and shape features.​

[0236] The color feature is usually expressed by a color histogram, and the color histogram is not affected by image rotation and translation, and is not affected by image scale change by means of normalization, and is particularly suitable for describing images that are difficult to automatically segment and images that do not need to consider the spatial position of objects.

[0237] The texture feature is a global feature that describes the surface properties of the scene corresponding to the image or image region.

[0238] The shape feature has two types of representation methods, one is a contour feature, and the other is a region feature. The contour feature of the image is mainly directed to the outer boundary of the object, and the region feature of the image relates to the entire shape region.

[0239] Therefore, one or more of the color histogram, the texture feature, and the contour feature of the first detection region can be extracted as the planar feature of the first detection region.

[0240] S1502, determining the detection result of the first defect according to the planar feature.

[0241] Based on the obtained planar feature, the detection result of the first defect can be determined, and specifically, the detection result of the first defect can be determined by using an observation method.

[0242] Therefore, please continue to refer to Figure 2 If the contour feature of the first detection region is obtained, the detection result of the first defect corresponding to the first detection region can be directly determined as that the detected product corresponding to the first detection region has a broken weld. Figure 2 If the contour feature of the first detection region is obtained, the detection result of the first defect corresponding to the first detection region can be directly determined as that the detected product corresponding to the first detection region has a broken weld.

[0243] In addition, the planar feature in the above embodiment can also be input into a detection algorithm, and the detection result of the first defect is determined in the detection algorithm.

[0244] In the above defect detection method, the planar feature of the first detection region is obtained, and the detection result of the first defect is determined according to the planar feature. In this method, the planar feature of the first detection region is obtained, and the defect judgment without height information is performed based on the planar feature, thereby improving the accuracy of defect detection.

[0245] S1303, determining the detection result of the second defect according to the second detection region.

[0246] The second detection area is obtained according to a three-dimensional image, and the three-dimensional image is a depth image. Therefore, the second detection area can be used to detect defects with height information. Therefore, the second defect can be a poor welding height, a step detection, a burst hole, and other defects with height information.

[0247] The second detection area can be detected according to a deep learning algorithm to determine the detection result of the second defect. The basic model in the deep learning algorithm includes but is not limited to a multi-layer perception model, a deep neural network model, and a recurrent neural network model. The representatives thereof are a deep belief network (DBN), a convolution neural network (CNN), and a recurrent neural network (RNN).

[0248] In one embodiment, a specific way of obtaining the detection result of the second defect is also provided. As shown in Figure 16 The detection result of the second defect is detected according to the second detection area, which includes the following steps:

[0249] S1601, obtaining the distance between the welding height of the target material and the material top cover reference surface in the second detection area.

[0250] Because the second detection area can reflect the height information of the detected product, the distance between the welding height of the target material and the material top cover reference surface in the second detection area can be obtained, and the defect information of the detected product corresponding to the second detection area can be determined according to the distance.

[0251] The distance can be obtained by inputting the second detection area into a distance obtaining network and analyzing the second detection area by the distance obtaining network to obtain the distance between the welding height of the target material and the material top cover reference surface in the second detection area.

[0252] S1602, determining the second defect detection result according to the distance.

[0253] In one embodiment, the detection result of the second defect is determined according to the distance between the welding height of the target material and the material top cover reference surface. That is, the defect category of the detected product is determined according to the distance, and the detection result of the second defect is obtained. For example, if the distance is within a first preset range, it can be determined that the detection result of the second defect is a poor welding height. If the distance is within a second preset range, it can be determined that the detection result of the second defect is a burst hole, and so on.

[0254] In another embodiment, if the distance between the weld bead height of the target material and the top cover reference surface of the material is less than a preset threshold, it is determined that the corresponding detected product is qualified, otherwise, it is determined that the corresponding detected product is unqualified.

[0255] The unqualified product can be further identified and detected by a deep learning algorithm to determine the defect type of the detected product.

[0256] In this embodiment, the distance between the weld bead height of the target material and the top cover reference surface of the material is obtained in the second detection area, and the second defect detection result is determined according to the distance. In this method, the detection result of the defect containing height information in the target material can be determined by the distance, ensuring the accuracy of defect detection.

[0257] In the above defect detection method, the first detection area in the two-dimensional image and the second detection area in the three-dimensional image are obtained, the detection result of the first defect is detected according to the first detection area, and the detection result of the second defect is detected according to the second detection area. In this method, the key detection area in the image is extracted by obtaining the detection area in the two-dimensional image and the three-dimensional image, avoiding a large amount of background redundant information, and improving the accuracy of the first defect detection result and the second defect detection result.

[0258] After obtaining the defect detection result of the target material, the defect detection result can be further displayed. In one embodiment, the embodiment comprises: sending the defect detection result of the target material to the upper computer; and / or displaying the defect detection result of the target material in the front-end interface.

[0259] After obtaining the defect detection result of the target material, the control device can also display the defect detection result of the target material in the front-end interface of the visual software of the control device, so that the monitoring personnel can intuitively determine the defect information of the target material.

[0260] Optionally, after the control device determines the defect detection result of the target material, the control device can send the defect detection result of the target material to the upper computer, and the upper computer can determine whether to release the target material according to the defect detection result, so that the target material flows to the next processing process stage.

[0261] In the above defect detection method, sending the defect detection result of the target material to the upper computer can enable the upper computer to timely and accurately obtain the defect detection result. Displaying the defect detection result of the target material in the front-end interface can make the acquisition of the defect information of the target material more intuitive.

[0262] In one embodiment, a defect detection method is also provided to detect the product as the sealed nail welded battery cell, the battery cell as four, the camera equipment as the 3D structured light camera, the two-dimensional light supplement equipment as the two-dimensional light source, the three-dimensional light supplement equipment as the three-dimensional light source, the upper computer as the PLC, and the control equipment as the personal computer (PC) are taken as examples for illustration.

[0263] As shown in Figure 17 , the detection flow chart of the battery cell is shown in Figure 17 . After the four battery cells are welded side by side in the sealing nail welding machine, the sealed nail welded battery cell is placed in the fixture of the turntable, the PLC controls the sealed nail welded battery cell to flow to the detection station with the fixture, after the battery cell flows into the starting position of the detection station, the PLC controls the cover plate to be pressed down to ensure that the four battery cell top covers are at the same height, then the PLC communicates with the PC through the network, the PLC sends the signal of the battery cell reaching the detection station to the PC, the PC controls the two-dimensional light source and the three-dimensional light source to be bright, then the PC controls the 3D structured light camera to take a two-dimensional mode photograph at the first detection station, when the two-dimensional mode photograph is completed, the 3D structured light camera sends a two-dimensional image taking completion signal to the PLC, after the PLC receives the two-dimensional image taking completion signal, the PLC sends a control instruction to the two-dimensional light source to make the two-dimensional light source move to the second detection station, the PC controls the 3D structured light camera to take a three-dimensional mode image, after the three-dimensional mode image is completed, the 3D structured light camera sends a three-dimensional mode image taking completion signal to the PLC, then the PLC controls the 3D structured light camera to move to the second detection station.

[0264] Similarly, the second detection station, the third detection station and the fourth detection station are taken a two-dimensional image and a three-dimensional image based on the above-mentioned way of taking a two-dimensional image and a three-dimensional image of the battery cell at the first detection station, until all the battery cells are imaged, the PLC controls the two-dimensional light source and the 3D structured light camera to return to the original position.

[0265] The 3D structured light camera sends the collected two-dimensional image and three-dimensional image to the PC, the vision software system in the PC acquires the two-dimensional image and three-dimensional image, and calls the artificial intelligence (AI) algorithm to pre-process the two-dimensional image and three-dimensional image, determine the detection area of the two-dimensional image and three-dimensional image, calculate the flatness and step in the detection area of the two-dimensional image and three-dimensional image respectively, then obtain the detection result according to the obtained flatness and step, and upload the detection result to the PLC through the transmission control protocol (TCP), and then the PLC feeds back the detection result to the manufacturing execution system (MES).

[0266] It should be noted that if there are multiple battery cells, then two-dimensional and three-dimensional images of each battery cell are obtained. Then, the two-dimensional and three-dimensional images of each battery cell are detected separately, and the detection results correspond to the detection results of each battery cell. Figure 17 The sensing head in the image is a 3D structured light camera.

[0267] AI algorithms include, but are not limited to, decision trees, random forests, logistic regression, Naive Bayes, K-nearest neighbors, K-means, neural networks, Markov algorithms, etc.

[0268] Specifically, to improve detection efficiency and detection accuracy, deep learning and point cloud detection algorithms are used for defect judgment. First, the 3D and 2D images acquired by the 3D structured light camera are spatially aligned. The spatially aligned 3D and 2D images are then fed into the AI ​​algorithm for defect identification and detection. The 3D image is converted into point cloud data by the AI ​​algorithm. Then, the ROI is located in the 3D image and the 2D image based on the point cloud data, resulting in the 3D detection area and the 2D detection area corresponding to the 3D image and the 2D image, respectively. Finally, defect detection and judgment are performed on the 3D and 2D detection areas.

[0269] like Figure 18 As shown, Figure 18 The process of using artificial intelligence algorithms includes an image acquisition module, an image preprocessing module, and an algorithm detection and judgment module. The preprocessing module includes labeling the training images.

[0270] In one embodiment, such as Figure 19 As shown, taking the tested product as a battery cell after sealing and welding, and a 3D structured light camera as the imaging equipment, this embodiment includes the following steps:

[0271] S1901, the host computer controls the battery cell to flow to the inspection station along with the fixture. After the battery cell flows into the starting position of the inspection station, the host computer controls the cover plate to press down.

[0272] S1902, the PLC sends the signal of the battery cell arriving at the inspection station to the control equipment, and the control equipment controls the two-dimensional light source and the three-dimensional light source to turn on.

[0273] S1903, the control equipment controls the 3D structured light camera to take two-dimensional pictures at the first inspection station.

[0274] S1904, the 3D structured light camera sends a two-dimensional image acquisition completion signal to the control device. After receiving the two-dimensional image acquisition completion signal, the control device sends a control command to the two-dimensional light source so that the two-dimensional light source moves to the next detection station.

[0275] S1905, the control device controls the 3D structured light camera to take a three-dimensional mode image, and after the three-dimensional mode image is taken, the 3D structured light camera sends a three-dimensional mode image taking completion signal to the control device, and then the control device controls the 3D structured light camera to move to the next detection station.

[0276] S1906, based on the above-mentioned first detection station, the two-dimensional image and the three-dimensional image of the battery cell are taken, and the two-dimensional image and the three-dimensional image of the next detection station are taken until all the battery cells are taken, and then the two-dimensional light source, the three-dimensional light source and the 3D structured light camera return to the original position.

[0277] S1907, the 3D structured light camera sends the collected two-dimensional image and three-dimensional image to the control device, and the control device calls an AI algorithm to detect defects in the two-dimensional image and the three-dimensional image to obtain a detection result.

[0278] Specifically, the 3D structured light camera sends the collected two-dimensional image and three-dimensional image to the control device, and the vision software system in the control device acquires the two-dimensional image and the three-dimensional image, and calls an AI algorithm to preprocess the two-dimensional image and the three-dimensional image, determines the detection area of the two-dimensional image and the three-dimensional image, calculates the flatness and the step in the detection area of the two-dimensional image and the three-dimensional image, and then obtains the detection result according to the obtained flatness and step, and uploads the detection result to the PLC and the upper computer through the TCP protocol, and then the upper computer feeds back the detection result to the manufacturing execution system.

[0279] The specific limitations of the defect detection method provided in this embodiment can be referred to the step limitations of the various embodiments of the defect detection method in the above text, which will not be repeated here.

[0280] It should be understood that although each step in the flowchart attached in the above-mentioned embodiments is displayed in sequence according to the arrow, these steps are not necessarily executed in the order indicated by the arrow. Unless otherwise stated herein, the execution of these steps has no strict order limitation, and these steps can be executed in other orders. Moreover, at least part of the steps in the above-mentioned embodiments can include multiple steps or multiple stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily sequential, but can be executed alternately or alternately with at least part of other steps or steps or stages in other steps.

[0281] Based on the same inventive concept, the embodiments of the present application also provide a defect detection device for implementing the defect detection method described above. The device provides a solution to the implementation scheme as described in the above method, and therefore the specific limitations in one or more defect detection device embodiments provided below can refer to the limitations of the defect detection method described above, which will not be repeated here.

[0282] In one embodiment, as shown in Figure 20 The embodiments of the present application also provide a defect detection device 2000, which comprises an image acquisition module 2001 and a defect determination module 2002, wherein:

[0283] The image acquisition module 2001 is configured to control the camera device to acquire the two-dimensional image and the three-dimensional image of the target material in response to a detection instruction of the target material; the camera device is a structured light camera device integrating two-dimensional image acquisition function and three-dimensional image acquisition function;

[0284] The defect determination module 2002 is configured to determine the defect detection result of the target material according to the two-dimensional image and the three-dimensional image of the target material.

[0285] In one embodiment, the image acquisition module 2001 comprises:

[0286] The first image acquisition unit is configured to control the camera device to acquire the two-dimensional image and the three-dimensional image of the detection product on the current detection station;

[0287] The second image acquisition unit is configured to control the camera device to move to the next detection station to acquire the two-dimensional image and the three-dimensional image of the detection product on the next detection station after the acquisition of the two-dimensional image and the three-dimensional image of the detection product on the current detection station is completed, so as to acquire the two-dimensional image and the three-dimensional image of all detection products.

[0288] In one embodiment, the first image acquisition unit comprises:

[0289] The two-dimensional image acquisition subunit is configured to control the camera device to acquire the two-dimensional image of the detection product on the current detection station under the condition that the two-dimensional light supplement device is turned on, and send a moving instruction to the upper computer after the acquisition of the two-dimensional image is completed, the moving instruction being used to instruct the upper computer to move the two-dimensional light supplement device to the next detection station;

[0290] The three-dimensional image acquisition subunit is configured to control the camera device to acquire the three-dimensional image of the detection product on the current detection station under the condition that the three-dimensional light supplement device is turned on.

[0291] In one embodiment, the detection instruction of the target material is sent by the upper computer after detecting that the target material enters the detection station, and the pressing device is controlled to perform the pressing operation on the target material.

[0292] In one embodiment, the camera device is a 3D structured light camera.

[0293] In one embodiment, the defect determination module 1602 comprises:

[0294] An alignment unit is configured to perform spatial alignment processing on the two-dimensional image and the three-dimensional image.

[0295] A detection unit is configured to process the two-dimensional image and the three-dimensional image after the spatial alignment processing by using a preset deep learning algorithm, to obtain a defect detection result of the target material.

[0296] In one embodiment, the detection unit comprises:

[0297] A detection area acquisition subunit is configured to acquire a first detection area in the two-dimensional image and a second detection area in the three-dimensional image.

[0298] A detection subunit is configured to detect a first defect detection result according to the first detection area, and detect a second defect detection result according to the second detection area.

[0299] In one embodiment, the detection area acquisition subunit comprises:

[0300] A conversion subunit is configured to pre-process the three-dimensional image to obtain point cloud data of the target material.

[0301] A detection area determination subunit is configured to determine the second detection area according to the point cloud data.

[0302] In one embodiment, the detection subunit comprises:

[0303] A feature acquisition subunit is configured to acquire a planar feature of the first detection area.

[0304] A first detection result determination subunit is configured to determine the first defect detection result according to the planar feature.

[0305] In one embodiment, the detection subunit comprises:

[0306] A distance acquisition subunit is configured to acquire a distance between a weld height of the target material and a material top cover reference surface in the second detection area.

[0307] A second detection result determination subunit is configured to determine the second defect detection result according to the distance.

[0308] In one embodiment, the device 2000 further includes:

[0309] The sending module is used to send the defect detection results of the target material to the host computer; and / or to display the defect detection results of the target material in the front-end interface.

[0310] Specific limitations regarding the defect detection device can be found in the limitations of each step in the defect detection method described above, and will not be repeated here. Each module in the aforementioned defect detection device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the target device in hardware form, or stored in the target device's memory in software form, so that the target device can invoke and execute the operations corresponding to each module.

[0311] In one embodiment, a computer device is provided, such as Figure 21 As shown, the computer device includes a processor, memory, communication interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When the computer program is executed by the processor, it implements a defect detection method. The display screen can be an LCD screen or an e-ink screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad mounted on the computer device casing, or an external keyboard, touchpad, or mouse.

[0312] Those skilled in the art will understand that Figure 21 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0313] In one embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.

[0314] The implementation principles and technical effects of each step in this embodiment are similar to those of the defect detection method described above, and will not be repeated here.

[0315] In an embodiment, a computer readable storage medium is provided, having stored thereon a computer program which, when executed by a processor, implements the steps of any of the above method embodiments.

[0316] The steps implemented by the computer program executed by the processor in this embodiment have similar implementation principles and technical effects to those of the above defect detection method, and will not be described here in detail.

[0317] In an embodiment, a computer program product is provided, comprising a computer program which, when executed by a processor, implements the steps of any of the above method embodiments.

[0318] The steps implemented by the computer program executed by the processor in this embodiment have similar implementation principles and technical effects to those of the above defect detection method, and will not be described here in detail.

[0319] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer readable storage medium, and when executed, can include the processes of the above-mentioned embodiments of the method. Any reference to memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical storage, high-density embedded non-volatile memory, resistive memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration but not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The database involved in the embodiments provided in the present application can include at least one of a relational database and a non-relational database. The non-relational database can include a distributed database based on a block chain, etc., without being limited thereto. The processor involved in the embodiments provided in the present application can be a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., without being limited thereto.

[0320] Any combination of the technical features of the above embodiments can be made. In order to make the description simple, all possible combinations of the technical features in the above embodiments are not described, however, as long as the combination of the technical features does not exist, it should be considered as the scope of the present application.

[0321] The above embodiments only express several implementation manners of the present application, and the description is more specific and detailed, but it should not be understood as a limitation on the patent scope of the present application. It should be noted that, for those skilled in the art, without departing from the concept of the present application, a number of modifications and improvements can be made, which are all within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the appended claims.

Claims

1. A defect detection method characterized by, The method comprises: in response to a detection instruction of a target material, controlling a camera device to collect a two-dimensional image and a three-dimensional image of the target material; the camera device is a structured light camera device integrating a two-dimensional image collection function and a three-dimensional image collection function; determining a defect detection result of the target material according to the two-dimensional image and the three-dimensional image of the target material; wherein the target material comprises a plurality of detection products, and each detection product is at a different detection station; the control of the camera device to collect the two-dimensional image and the three-dimensional image of the target material comprises: controlling the camera device to collect a two-dimensional image and a three-dimensional image of a detection product at a current detection station; after the two-dimensional image and the three-dimensional image of the detection product at the current detection station are collected, controlling the camera device to move to a next detection station to collect a two-dimensional image and a three-dimensional image of a detection product at the next detection station, so as to collect two-dimensional images and three-dimensional images of all detection products; the control of the camera device to collect a two-dimensional image and a three-dimensional image of a detection product at a current detection station comprises: controlling the camera device to collect a two-dimensional image of the detection product at the current detection station under the condition that a two-dimensional light supplement device is turned on, and sending a moving instruction to an upper computer after the two-dimensional image is collected, the moving instruction being used to instruct the upper computer to move the two-dimensional light supplement device to the next detection station; controlling the camera device to collect a three-dimensional image of the detection product at the current detection station under the condition that a three-dimensional light supplement device is turned on; the determination of a defect detection result of the target material according to the two-dimensional image and the three-dimensional image of the target material comprises: performing spatial alignment processing on the two-dimensional image and the three-dimensional image; processing the two-dimensional image and the three-dimensional image processed by spatial alignment processing by using a preset deep learning algorithm to obtain the defect detection result of the target material.

2. The method of claim 1, wherein, The detection instruction of the target material is sent by an upper computer after the target material enters a detection station and a pressing device is controlled to perform a pressing operation on the target material; the coverage area of the pressing device is greater than or equal to the upper surface area of the target material.

3. The method of claim 1, wherein, The camera device is a 3D structured light camera.

4. The method of claim 1, wherein, The defect detection result comprises a detection result of a first defect and a detection result of a second defect; the first defect is a defect without height information, and the second defect is a defect with height information; the processing of the two-dimensional image and the three-dimensional image processed by spatial alignment processing by using a preset deep learning algorithm to obtain the defect detection result of the target material comprises: obtaining a first detection area in the two-dimensional image and a second detection area in the three-dimensional image; detecting the detection result of the first defect according to the first detection area and detecting the detection result of the second defect according to the second detection area.

5. The method of claim 4, wherein, the obtaining of the second detection area in the three-dimensional image comprises: preprocessing the three-dimensional image to obtain point cloud data of the target material; determining the second detection area according to the point cloud data.

6. The method of claim 4, wherein, The detection result of the first defect detected according to the first detection area includes: Obtaining a planar feature of the first detection area; According to the planar feature, the detection result of the first defect is determined.

7. The method of claim 4, wherein, The detection result of the second defect detected according to the second detection area includes: Obtaining the distance between the weld height of the target material and the material top cover reference surface in the second detection area; According to the distance, the second defect detection result is determined.

8. The method of claim 1, wherein, The method further includes: Sending the defect detection result of the target material to the upper computer; and / or, Displaying the defect detection result of the target material in the front-end interface.

9. A defect detection system, characterized by, The system includes a support frame, a camera device arranged on the support frame, a control device in communication with the camera device, an upper computer in communication with the control device, and a detection station arranged below the support frame; the camera device is a structured light camera device integrating two-dimensional image acquisition function and three-dimensional image acquisition function; the camera device acquires two-dimensional images and three-dimensional images in the same coordinate system; The detection station is used to carry the target material to be detected; The upper computer is used to send a detection instruction of the target material to the control device; The control device is used to control the camera device to acquire two-dimensional images and three-dimensional images of the target material, and determine a defect detection result of the target material according to the two-dimensional images and the three-dimensional images of the target material; The system further includes a lifting device; the lifting device is arranged on the support frame, and the camera device is installed on the lifting device; the support frame below includes a plurality of detection stations; The control device is used to control the lifting device to move the camera device to the position of each detection station to acquire two-dimensional images and three-dimensional images of the detection products in each detection station respectively; The system further includes a sliding assembly and a two-dimensional light supplement device; the sliding assembly is arranged on the support frame and below the lifting device, and the two-dimensional light supplement device is arranged on the sliding assembly; The control device is used to control the camera device to acquire two-dimensional images of the detection products in each detection station when the two-dimensional light supplement device is turned on, and send a moving instruction to the upper computer after the acquisition of the two-dimensional images of the detection products in each detection station is completed, the moving instruction being used to instruct the upper computer to move the two-dimensional light supplement device to the next detection station through the sliding assembly.

10. The system of claim 9, wherein, The system further includes a three-dimensional light supplement device; the three-dimensional light supplement device is arranged at a preset distance above each detection station; The control device is used to control the camera device to acquire three-dimensional images of the detection products in the corresponding detection station when the three-dimensional light supplement device is turned on.

11. The system of claim 10, wherein, The system further includes a press-fitting device; the press-fitting device is in communication with the upper computer. The host computer is further configured to detect that the target material enters the detection station, and send the detection instruction to the control device after the pressing device presses the target material.

12. The system of claim 11, wherein, The pressing device is provided with a cavity structure at a position corresponding to each detection station, and the three-dimensional light supplementing device is a ring light device arranged around the inside of the cavity. The cavity structure is used for the camera device to align the detection products on each detection station.

13. A defect detection apparatus characterized by comprising: The device comprises: An image acquisition module configured to control a camera device to acquire a two-dimensional image and a three-dimensional image of a target material in response to a detection instruction for the target material, the camera device being a structured light camera device integrating a two-dimensional image acquisition function and a three-dimensional image acquisition function; A defect determination module configured to determine a defect detection result of the target material based on the two-dimensional image and the three-dimensional image of the target material; The image acquisition module comprises: A first image acquisition unit configured to control the camera device to acquire a two-dimensional image and a three-dimensional image of a detection product on a current detection station; A second image acquisition unit configured to control the camera device to move to a next detection station to acquire a two-dimensional image and a three-dimensional image of a detection product on the next detection station after the acquisition of the two-dimensional image and the three-dimensional image of the detection product on the current detection station is completed, so as to acquire two-dimensional images and three-dimensional images of all detection products; The first image acquisition unit comprises: A two-dimensional image acquisition subunit configured to control the camera device to acquire a two-dimensional image of a detection product on the current detection station with a two-dimensional light supplementing device turned on, and send a moving instruction to a host computer after the acquisition of the two-dimensional image is completed, the moving instruction being used to instruct the host computer to move the two-dimensional light supplementing device to the next detection station; A three-dimensional image acquisition subunit configured to control the camera device to acquire a three-dimensional image of the detection product on the current detection station with a three-dimensional light supplementing device turned on; The defect determination module comprises: An alignment unit configured to perform spatial alignment processing on the two-dimensional image and the three-dimensional image; A detection unit configured to process the two-dimensional image and the three-dimensional image subjected to the spatial alignment processing through a preset deep learning algorithm to obtain a defect detection result of the target material.

14. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that, The processor executes the computer program to implement the steps of the method of any one of claims 1 to 8.

15. A computer readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the method of any one of claims 1 to 8.

16. A computer program product comprising a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the method of any one of claims 1 to 8.

Images