Square shell material frame detection method based on machine vision

Abstract

The present application relates to a method for detecting a square shell material frame based on machine vision, the method comprising: obtaining three-dimensional point cloud data and a color two-dimensional image of the square shell material frame, generating a color three-dimensional model based on the three-dimensional point cloud data and the color two-dimensional image, segmenting the color three-dimensional model to obtain at least two positioning blocks, obtaining an edge line of the square shell material frame based on the positioning blocks, determining the size of the square shell material frame based on the edge line, determining a reference plane on the upper surface of each positioning block, and judging whether there is a defect on the surface of the square shell material frame corresponding to each positioning block based on the reference plane. Through the present application, the problem of low efficiency in material frame detection is solved. Based on a machine vision solution, the three-dimensional data and the color two-dimensional image of the material frame are obtained, the length, width, and depth information of the material frame are calculated, and whether the positioning blocks of the material frame are damaged is detected, thereby improving the efficiency of material frame detection through automated detection.

Description

Technical Field

[0001] The present application relates to the field of product detection, and in particular to a method for detecting a square shell material frame based on machine vision. Background Art

[0002] Material frames are used to place and transport products, such as battery material frames, on the production line. Before placing batteries in the frame, the frame's stoppers, bottom pads, and other structures must be inspected to ensure accurate stop dimensions, structural anomalies, and missing pads are present to avoid damage to the batteries. In related technologies, material frames still require manual inspection, which is inefficient and poses safety risks.

[0003] Currently, no effective solution has been proposed to the problem of low efficiency of material frame detection in related technologies. Summary of the Invention

[0004] The embodiments of the present application provide a method, system, electronic device, and storage medium for detecting a square shell material frame based on machine vision, so as to at least solve the problem of low efficiency of material frame detection in the related art.

[0005] In a first aspect, an embodiment of the present application provides a method for detecting a square shell frame based on machine vision, the method comprising:

[0006] Acquire three-dimensional point cloud data and a color two-dimensional image of the square shell material frame, and generate a color three-dimensional model based on the three-dimensional point cloud data and the color two-dimensional image;

[0007] Segmenting the color three-dimensional model to obtain at least two positioning blocks;

[0008] Obtaining an edge line of the square shell material frame based on the positioning block, and determining a size of the square shell material frame according to the edge line;

[0009] A reference plane on the upper surface of each positioning block is determined, and based on the reference plane, it is determined whether there is a defect on the surface of the square shell material frame corresponding to each positioning block.

[0010] In some embodiments, obtaining three-dimensional point cloud data and a color two-dimensional image of the square shell material frame, and generating a color three-dimensional model based on the three-dimensional point cloud data and the color two-dimensional image includes:

[0011] Acquire the three-dimensional point cloud data of the square shell material frame by using a 3D vision sensor, and acquire the color two-dimensional image of the square shell material frame by using a 2D area array camera;

[0012] Performing distortion correction on the color two-dimensional image according to internal parameters and distortion parameters of the 2D area array camera to obtain a color corrected image;

[0013] Based on the external parameters of the 3D vision sensor, the three-dimensional point cloud data is converted into the camera coordinate system of the 2D area array camera, and a color three-dimensional model is generated according to the converted three-dimensional point cloud data and the color corrected image.

[0014] In some embodiments, before converting the three-dimensional point cloud data into the camera coordinate system of the 2D area array camera, the method further includes:

[0015] The number of neighboring points within a preset radius threshold of each point in the three-dimensional point cloud data is counted, and points whose number of neighboring points is lower than the preset threshold are removed.

[0016] In some embodiments, the square shell material frame is a battery material frame; the positioning block includes a first positioning block and a second positioning block of the battery; obtaining an edge line of the square shell material frame based on the positioning block, and determining the size of the square shell material frame according to the edge line includes:

[0017] Scanning the first positioning block and the second positioning block respectively to obtain the edge line of the square shell frame;

[0018] The intersection points of the edge lines are determined, and the length and width of the square shell frame are determined based on the distance between the intersection points.

[0019] In some embodiments, scanning the first positioning block and the second positioning block respectively to obtain the edge line of the square shell frame includes:

[0020] Scanning the first positioning block row by row to obtain the transverse edge of the first positioning block and fitting the edge, and determining the first edge line and the third edge line of the square shell frame according to the fitting result;

[0021] Scanning the second positioning block row by row to obtain the transverse edge of the second positioning block and fitting the same, and determining the second edge line and the fourth edge line of the square shell material frame according to the fitting results;

[0022] Scanning the first positioning blocks in columns to obtain vertical edges of the first positioning blocks and performing fitting, and determining a fifth edge line of the square shell frame according to the fitting results;

[0023] The second positioning blocks are scanned in columns to obtain vertical edges of the second positioning blocks and perform fitting, and the sixth edge line of the square shell frame is determined according to the fitting results.

[0024] In some embodiments, determining the reference plane of the upper surface of each positioning block includes:

[0025] Performing plane extraction on the positioning block to determine the initial plane and plane parameters;

[0026] Obtaining the distance between each point in the positioning block and the initial plane, and marking the points whose distance is less than a preset distance threshold as points in the initial plane;

[0027] Based on an iterative algorithm, with the initial plane as the initial value and the maximum number of points in the plane as the goal, the plane parameters are modified to determine the reference plane.

[0028] In some embodiments, the square shell material frame is a battery material frame; the method further includes:

[0029] Determining the position of the bayonet in the positioning block according to the position of the intersection;

[0030] Based on the bayonet position, generating a gasket area at the bottom of the battery material frame, and obtaining depth information of the gasket area from the three-dimensional color model;

[0031] Extracting the plane where the gasket is located based on a pre-built gasket detection model and the depth information to obtain a gasket plane;

[0032] The depth of the battery material frame is determined according to the reference plane of the gasket plane and the upper surface of the positioning block.

[0033] In a second aspect, an embodiment of the present application provides a square shell material frame detection system based on machine vision, the system comprising:

[0034] A data acquisition module is used to acquire three-dimensional point cloud data and a color two-dimensional image of the square shell material frame, and generate a color three-dimensional model based on the three-dimensional point cloud data and the color two-dimensional image;

[0035] a segmentation module, configured to segment the color three-dimensional model to obtain at least two positioning blocks;

[0036] a size detection module, configured to obtain an edge line of the square shell material frame based on the positioning block, and determine a size of the square shell material frame according to the edge line;

[0037] The defect detection module is used to determine the reference plane of the upper surface of each positioning block, and judge whether there is a defect on the surface of the square shell material frame corresponding to each positioning block based on the reference plane.

[0038] In a third aspect, an embodiment of the present application provides a computer device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, the machine vision-based square shell material frame detection method as described in the first aspect above is implemented.

[0039] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the square shell material frame detection method based on machine vision as described in the first aspect above.

[0040] Compared to related technologies, the machine vision-based square shell material frame detection method provided in the embodiments of the present application obtains 3D point cloud data and a color 2D image of the square shell material frame, generates a color 3D model based on the 3D point cloud data and the color 2D image, segments the color 3D model to obtain at least two positioning blocks, obtains the edge line of the square shell material frame based on the positioning blocks, determines the size of the square shell material frame based on the edge line, determines the reference plane of the upper surface of each positioning block, and determines whether the surface of the square shell material frame corresponding to each positioning block has defects based on the reference plane, thereby solving the problem of low material frame detection efficiency. Based on the machine vision solution, the 3D data and color 2D image of the material frame are obtained, the length, width, and depth information of the material frame is calculated, and the material frame positioning block is detected for damage, thereby improving the material frame detection efficiency through automated detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0042] Figure 1 is a flow chart of a method for detecting a square shell material frame based on machine vision according to an embodiment of the present application;

[0043] Figure 2 is a schematic diagram of a battery material frame according to an embodiment of the present application;

[0044] Figure 3 This is a schematic diagram of a material frame size measurement according to an embodiment of the present application;

[0045] Figure 4 is a schematic diagram of positioning block defect detection according to an embodiment of the present application;

[0046] Figure 5 is a schematic diagram of bottom gasket missing detection according to an embodiment of the present application;

[0047] Figure 6 This is a flow chart of measuring a square shell battery material frame according to an embodiment of the present application;

[0048] Figure 7 1 is a structural block diagram of a square shell material frame detection system based on machine vision according to an embodiment of the present application;

[0049] Figure 8 Schematic diagram of the internal structure of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0050] In order to make the purpose, technical solutions and advantages of this application more clearly understood, the present application is described and illustrated below in conjunction with the accompanying drawings and examples. It should be understood that the specific embodiments described herein are merely used to explain this application and are not intended to limit this application. Based on the embodiments provided in this application, all other embodiments obtained by those of ordinary skill in the art without making any creative efforts are within the scope of protection of this application.

[0051] Obviously, the drawings described below are merely examples or embodiments of the present application. Those skilled in the art can, without inventive effort, apply the present application to other similar scenarios based on these drawings. Furthermore, it is also understood that, although the effort involved in such a development process may be complex and lengthy, for those skilled in the art related to the content disclosed in this application, changes in design, manufacturing, or production based on the technical content disclosed in this application are merely conventional technical means and should not be construed as an insufficiency of the content disclosed in this application.

[0052] References to "embodiments" in this application mean that a particular feature, structure, or characteristic described in connection with the embodiment may be included in at least one embodiment of the application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it refer to independent or alternative embodiments that are mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described in this application may be combined with other embodiments unless there is a conflict.

[0053] Unless otherwise defined, technical or scientific terms used herein shall have the ordinary meaning as understood by persons of ordinary skill in the art to which this application belongs. The terms "a," "an," "an," "the," and similar expressions used herein do not denote quantitative limitations and may refer to either the singular or the plural. The terms "comprise," "include," "have," and any variations thereof, used herein, are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or apparatus comprising a series of steps or modules (units) is not limited to the listed steps or units but may also include steps or units not listed, or may include other steps or units inherent to the process, method, product, or apparatus. The terms "connected," "connected," "coupled," and similar expressions used herein are not limited to physical or mechanical connections but may include electrical connections, whether direct or indirect. As used herein, "plurality" means two or more. "And / or" describes an association between associated objects, indicating that three possible relationships exist. For example, "A and / or B" may mean: A exists alone; A and B exist simultaneously; or B exists alone. The character " / " generally indicates that the objects before and after are in an "or" relationship. The terms "first", "second", "third", etc. involved in this application are only used to distinguish similar objects and do not represent a specific order for the objects.

[0054] This embodiment provides a method for detecting a square shell frame based on machine vision. Figure 1 FIG. 1 is a flow chart of a method for detecting a square shell material frame based on machine vision according to an embodiment of the present application. Figure 1 As shown, the process includes the following steps:

[0055] Step S101 , obtaining three-dimensional point cloud data and a color two-dimensional image of a square shell material frame, and generating a color three-dimensional model based on the three-dimensional point cloud data and the color two-dimensional image.

[0056] In some embodiments, step S101 specifically includes:

[0057] Step S1011 , obtaining three-dimensional point cloud data of the square shell material frame through a 3D vision sensor, and obtaining a color two-dimensional image of the square shell material frame through a 2D area array camera.

[0058] Optionally, a 3D vision sensor is used to acquire 3D point cloud data of the prismatic battery housing, and an external RGB color 2D area array camera is used to acquire a color 2D image of the housing. Camera calibration is performed to calculate the intrinsic parameters, distortion parameters, and extrinsic parameters of the 3D vision sensor and external 2D area array camera.

[0059] Step S1012 : performing distortion correction on the color two-dimensional image according to the internal parameters and distortion parameters of the 2D area array camera to obtain a color corrected image.

[0060] Step S1013 : Based on the external parameters of the 3D vision sensor, the 3D point cloud data is converted into the camera coordinate system of the 2D area array camera, and a color 3D model is generated according to the converted 3D point cloud data and the color corrected image.

[0061] The 3D point cloud data of the material frame is converted to the world coordinate system based on the external parameters of the 3D vision sensor, and then converted to the camera coordinate system of the 2D area array camera. The converted 3D point cloud data is matched with the pixels of the color 2D image to obtain a depth image with color information, namely the color 3D model.

[0062] In some embodiments, before converting the 3D point cloud data into the camera coordinate system of the 2D area array camera, the method further includes:

[0063] Count the number of neighboring points within a preset radius threshold for each point in the 3D point cloud data, and remove points whose number of neighboring points is lower than the preset threshold.

[0064] Remove outliers or noise points whose number of neighborhood points is lower than the threshold and retain the main part of the frame.

[0065] After the color 3D model is constructed, the 3D space range is set according to the estimated material frame size, and thresholds are set in the x, y, and z directions respectively to remove invalid data outside the material frame range and filter out interfering background to reduce the amount of data that needs to be processed and improve processing efficiency.

[0066] Step S102: segment the color 3D model to obtain at least two positioning blocks.

[0067] Step S103 , obtaining edge lines of the square shell material frame based on the positioning block, and determining the size of the square shell material frame according to the edge lines.

[0068] In some embodiments, the square shell material frame is a battery material frame; the positioning block includes a first positioning block and a second positioning block of the battery; step S103 specifically includes:

[0069] Step S1031 : Scan the first positioning block and the second positioning block respectively to obtain the edge line of the square shell frame.

[0070] Figure 2 This is a schematic diagram of a battery material frame according to an embodiment of the present application.

[0071] In some embodiments, step S1031 specifically includes:

[0072] Step S201 : Scan the first positioning block row by row to obtain the transverse edge of the first positioning block and perform fitting, and determine the first edge line and the third edge line of the square shell frame according to the fitting result.

[0073] Step S202 : Scan the second positioning block row by row to obtain the transverse edge of the second positioning block and perform fitting, and determine the second edge line and the fourth edge line of the square shell frame according to the fitting results.

[0074] Step S203 , scanning the first positioning block by column, obtaining the vertical edge of the first positioning block and fitting it, and determining the fifth edge line of the square shell frame according to the fitting result.

[0075] Step S204 , scanning the second positioning block by column, obtaining the vertical edge of the second positioning block and fitting it, and determining the sixth edge line of the square shell frame according to the fitting result.

[0076] Figure 3 This is a schematic diagram of a material frame size measurement according to an embodiment of the present application. Figure 3 As shown, after the positioning block parts are segmented, the upper and lower horizontal edges of the left positioning block (first positioning block) are obtained by row scanning in scanning area 2, and straight lines l1 and l3 are obtained by fitting; the upper and lower horizontal edges of the right positioning block (second positioning block) are obtained by row scanning in scanning area 3, and straight lines l2 and l4 are obtained by fitting; the vertical edges of the left and right positioning blocks are obtained by column scanning in scanning area 1, and straight lines l5 and l6 are obtained by fitting.

[0077] Step S1032: determining the intersection points of the edge lines, and determining the length and width of the square shell frame based on the distance between the intersection points.

[0078] by Figure 3 For example, calculate the coordinates of intersection 1 and intersection 3 of the two groups of straight lines l1 and l5, and l3 and l5 respectively, calculate the coordinates of intersection 2 and intersection 4 of the two groups of straight lines l2 and l6, and l4 and l6 respectively, and calculate the length and width of a battery position based on the coordinates of the four intersection points.

[0079] Continue to refer to Figure 1 After determining the size of the square shell material frame, execute step S104.

[0080] Step S104 , determining a reference plane on the upper surface of each positioning block, and judging whether there is a defect on the surface of the square shell material frame corresponding to each positioning block based on the reference plane.

[0081] In some embodiments, determining the reference plane of the upper surface of each positioning block in step S104 includes:

[0082] Step S1041: extract the plane of the positioning block to determine the initial plane and plane parameters.

[0083] Step S1042 : obtaining the distance between each point in the positioning block and the initial plane, and marking the points whose distance is less than a preset distance threshold as points in the initial plane.

[0084] Step S1043 , based on an iterative algorithm, with the initial plane as the initial value and the maximum number of points in the plane as the goal, modify the plane parameters and determine the reference plane.

[0085] Perform plane extraction on a single positioning block. Optionally, randomly select three points in the positioning block's 3D data. Given a known spatial plane equation of ax+by+cz+d=0, calculate the initial plane parameters based on this equation.

[0086] Modify the plane parameters, calculate the distances from the remaining points to the plane corresponding to the current plane parameters, mark points with distances less than the preset threshold as plane interior points, and count the number of inliers under the current plane parameters. If the number of inliers exceeds the previous maximum number of inliers, iterate the plane equation based on all current inliers until the current plane contains the maximum number of inliers or the preset number of iterations is exceeded. The final result is the parameters corresponding to the reference plane.

[0087] When the positioning block is defective or damaged, it usually causes the local surface of the positioning block to be uneven. After calculating the reference plane of the upper surface, the surface condition of the positioning block can be detected, and the damaged part can be extracted based on the reference plane to prevent the defective material frame from causing damage to the battery. Figure 4 This is a schematic diagram of positioning block defect detection according to an embodiment of the present application.

[0088] In some embodiments, the square shell material frame is a battery material frame; and the method further comprises:

[0089] Step S301: Determine the position of the bayonet in the positioning block according to the position of the intersection.

[0090] Step S302 : Based on the position of the bayonet, a gasket area at the bottom of the battery material frame is generated, and depth information of the gasket area is obtained from the three-dimensional color model.

[0091] Step S303 : extracting the plane where the gasket is located based on the pre-built gasket detection model and depth information to obtain the gasket plane.

[0092] Step S304 , determining the depth of the battery material frame according to the reference plane of the gasket plane and the upper surface of the positioning block.

[0093] The four intersection points of each battery cell position determine the location of the latches in the left and right positioning blocks, establishing the bottom gasket inspection area. Optionally, some material frames use brightly colored gasket materials, leveraging this feature to inspect color information in the color 2D image, improving inspection accuracy and efficiency.

[0094] Optionally, the gasket plane is determined using a gasket detection model. Images of the gasket area under actual working conditions are collected to construct a deep learning dataset and train a gasket detection model with high accuracy and good robustness to achieve stable detection results.

[0095] During inspection, based on the gasket detection model, the depth information of the corresponding area is used to extract the gasket plane, calculate the height difference between the gasket plane and the upper surface of the positioning block, and output the depth information of the corresponding battery material frame. It also determines whether any positioning block gaskets are missing. Figure 5 This is a schematic diagram of bottom gasket missing detection according to an embodiment of the present application.

[0096] By using the above method, three-dimensional point cloud data and a color two-dimensional image of a square shell material frame are obtained, a color three-dimensional model is generated based on the three-dimensional point cloud data and the color two-dimensional image, the color three-dimensional model is segmented to obtain at least two positioning blocks, edge lines of the square shell material frame are obtained based on the positioning blocks, the size of the square shell material frame is determined based on the edge lines, a reference plane on the upper surface of each positioning block is determined, and based on the reference plane, it is determined whether the surface of the square shell material frame corresponding to each positioning block has defects, thereby solving the problem of low material frame detection efficiency. Figure 6 This is a flow chart for measuring a square shell battery material frame according to an embodiment of the present application.

[0097] Based on the machine vision solution, the three-dimensional data and color two-dimensional image of the material frame are obtained, the length, width and depth information of the material frame are calculated, and the material frame positioning block is detected to see if it is damaged and the gasket is in place, thereby improving the detection efficiency and accuracy of the material frame.

[0098] It should be noted that the steps shown in the above process or the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0099] This embodiment also provides a machine vision-based square shell material frame detection system, which is used to implement the above-mentioned embodiments and preferred embodiments. Details that have already been described will not be repeated here. As used below, the terms "module," "unit," "subunit," etc. may refer to a combination of software and / or hardware that implements a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, implementation using hardware, or a combination of software and hardware, is also possible and contemplated.

[0100] Figure 7 FIG. 1 is a block diagram of a machine vision-based square shell material frame detection system according to an embodiment of the present application. Figure 7 As shown, the system includes:

[0101] The data acquisition module 71 is used to acquire the 3D point cloud data and the color 2D image of the square shell material frame, and generate a color 3D model based on the 3D point cloud data and the color 2D image.

[0102] The segmentation module 72 is used to segment the color 3D model to obtain at least two positioning blocks.

[0103] The size detection module 73 is used to obtain the edge line of the square shell material frame based on the positioning block, and determine the size of the square shell material frame according to the edge line.

[0104] The defect detection module 74 is used to determine the reference plane of the upper surface of each positioning block, and judge whether there is a defect on the surface of the square shell material frame corresponding to each positioning block based on the reference plane.

[0105] In some embodiments, the data acquisition module 71 includes:

[0106] The acquisition module is used to obtain the three-dimensional point cloud data of the square shell material frame through the 3D vision sensor and obtain the color two-dimensional image of the square shell material frame through the 2D area array camera.

[0107] The correction module is used to perform distortion correction on the color two-dimensional image according to the internal parameters and distortion parameters of the 2D area array camera to obtain a color corrected image.

[0108] The model generation module is used to convert the three-dimensional point cloud data into the camera coordinate system of the 2D area array camera based on the external parameters of the 3D vision sensor, and generate a color three-dimensional model based on the converted three-dimensional point cloud data and the color corrected image.

[0109] In some embodiments, the data acquisition module 71 further includes:

[0110] The preprocessing module is used to count the number of neighboring points of each point in the three-dimensional point cloud data within a preset radius threshold, and remove points whose number of neighboring points is lower than the preset threshold.

[0111] In some embodiments, the square shell material frame is a battery material frame; the positioning block includes a first positioning block and a second positioning block of the battery; and the size detection module 73 includes:

[0112] The scanning module is used to scan the first positioning block and the second positioning block respectively to obtain the edge line of the square shell frame.

[0113] The measurement module is used to determine the intersection points of the edge lines and determine the length and width of the square shell frame based on the distance between the intersection points.

[0114] In some embodiments, the scanning module includes:

[0115] The first scanning module is used to scan the first positioning block row by row, obtain the transverse edge of the first positioning block and perform fitting, and determine the first edge line and the third edge line of the square shell material frame according to the fitting result.

[0116] The second scanning module is used to scan the second positioning block row by row, obtain the transverse edge of the second positioning block and perform fitting, and determine the second edge line and the fourth edge line of the square shell material frame according to the fitting results.

[0117] The third scanning module is used to scan the first positioning block in columns, obtain the vertical edge of the first positioning block and fit it, and determine the fifth edge line of the square shell frame according to the fitting result.

[0118] The fourth scanning module is used to scan the second positioning block in columns, obtain the vertical edge of the second positioning block and fit it, and determine the sixth edge line of the square shell frame according to the fitting result.

[0119] In some embodiments, the defect detection module 74 includes:

[0120] The initial value module is used to extract the plane of the positioning block and determine the initial plane and plane parameters.

[0121] The statistical module is used to obtain the distance between each point in the positioning block and the initial plane, and mark the points with a distance less than a preset threshold as points in the initial plane.

[0122] The iteration module is used to modify the plane parameters and determine the reference plane based on an iterative algorithm, with the initial plane as the initial value and the maximum number of points in the plane as the goal.

[0123] In some embodiments, the square shell material frame is a battery material frame; the system further includes:

[0124] The bayonet detection module is used to determine the bayonet position in the positioning block according to the position of the intersection.

[0125] The depth information acquisition module is used to generate the gasket area at the bottom of the battery material frame based on the bayonet position and obtain the depth information of the gasket area from the three-dimensional color model.

[0126] The plane extraction module is used to extract the plane where the gasket is located based on the pre-built gasket detection model and depth information to obtain the gasket plane.

[0127] The depth determination module is used to determine the depth of the battery material frame based on the reference plane of the gasket plane and the upper surface of the positioning block.

[0128] Through the above system, the data acquisition module 71 acquires the 3D point cloud data and color 2D image of the square shell material frame, and generates a color 3D model based on the 3D point cloud data and color 2D image. The segmentation module 72 segments the color 3D model to obtain at least two positioning blocks. The size detection module 73 obtains the edge lines of the square shell material frame based on the positioning blocks and determines the size of the square shell material frame based on the edge lines. The defect detection module 74 determines the reference plane on the upper surface of each positioning block and determines whether there are defects on the surface of the square shell material frame corresponding to each positioning block based on the reference plane, thus solving the problem of low material frame detection efficiency. Based on the machine vision solution, the 3D data and color 2D image of the material frame are acquired, the length, width, and depth information of the material frame is calculated, and the positioning blocks of the material frame are detected for damage. The automated detection improves the efficiency of material frame detection.

[0129] It should be noted that the above modules can be functional modules or program modules, and can be implemented through software or hardware. For modules implemented through hardware, the above modules can be located in the same processor; or the above modules can be located in different processors in any combination.

[0130] This embodiment further provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0131] Optionally, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0132] Optionally, in this embodiment, the processor may be configured to execute the following steps through a computer program:

[0133] S1, obtaining three-dimensional point cloud data and a color two-dimensional image of the square shell material frame, and generating a color three-dimensional model based on the three-dimensional point cloud data and the color two-dimensional image.

[0134] S2, segmenting the color 3D model to obtain at least two positioning blocks.

[0135] S3, obtaining an edge line of the square shell material frame based on the positioning block, and determining a size of the square shell material frame according to the edge line.

[0136] S4, determining a reference plane on the upper surface of each positioning block, and judging whether there is a defect on the surface of the square shell material frame corresponding to each positioning block based on the reference plane.

[0137] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementation modes, and this embodiment will not be repeated here.

[0138] In one embodiment, Figure 8 is a schematic diagram of the internal structure of an electronic device according to an embodiment of the present application, such as Figure 8 As shown, an electronic device is provided, which may be a server, and its internal structure diagram may be as shown in FIG. Figure 8 As shown. The electronic device includes a processor, a memory, a network interface and a database connected via a system bus. The processor of the electronic device is used to provide computing and control capabilities. The memory of the electronic device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The database of the electronic device is used to store data. The network interface of the electronic device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, a square shell material frame detection method based on machine vision is implemented.

[0139] Those skilled in the art will understand that Figure 8 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the electronic device to which the solution of the present application is applied. The specific electronic device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0140] Those skilled in the art will understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0141] Those skilled in the art should understand that the various technical features of the above-described embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the various technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0142] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art could make various modifications and improvements without departing from the spirit of the present application, all of which fall within the scope of protection of the present application. Therefore, the scope of protection of the present patent application shall be determined by the appended claims.

Claims

1. A method for detecting a square shell frame based on machine vision, characterized in that: The method comprises: Acquiring three-dimensional point cloud data and a color two-dimensional image of the square shell material frame, and generating a color three-dimensional model based on the three-dimensional point cloud data and the color two-dimensional image, including: Acquire three-dimensional point cloud data of the square shell material frame through a 3D vision sensor, and acquire a color two-dimensional image of the square shell material frame through a 2D area array camera; perform distortion correction on the color two-dimensional image according to the internal parameters and distortion parameters of the 2D area array camera to obtain a color-corrected image; convert the three-dimensional point cloud data into the camera coordinate system of the 2D area array camera based on the external parameters of the 3D vision sensor, and generate a color three-dimensional model according to the converted three-dimensional point cloud data and the color-corrected image; Segmenting the color three-dimensional model to obtain at least two positioning blocks, wherein the positioning blocks include a first positioning block and a second positioning block; Obtaining edge lines of the square shell material frame based on the positioning blocks, and determining the size of the square shell material frame according to the edge lines, including: scanning the first positioning blocks and the second positioning blocks in rows and columns respectively to obtain edge lines of the square shell material frame; determining intersection points of the edge lines, and determining the length and width of the square shell material frame based on the distance between the intersection points; Determine the reference plane of the upper surface of each positioning block, and judge whether there is a defect on the surface of the square shell material frame corresponding to each positioning block based on the reference plane, including: when the square shell material frame is a battery material frame, Determining the position of the bayonet in the positioning block according to the position of the intersection; Based on the bayonet position, generating a gasket area at the bottom of the battery material frame, and obtaining depth information of the gasket area from the color three-dimensional model; Extracting the plane where the gasket is located based on a pre-built gasket detection model and the depth information to obtain a gasket plane; Determining the depth of the battery material frame according to the reference plane of the gasket plane and the upper surface of the positioning block; It is determined based on the reference plane whether the square shell material frame has any damage and whether the gasket at the bottom of the battery material frame is missing.

2. The method according to claim 1, characterized in that Before converting the three-dimensional point cloud data into the camera coordinate system of the 2D area array camera, the method further includes: The number of neighboring points within a preset radius threshold of each point in the three-dimensional point cloud data is counted, and points whose number of neighboring points is lower than the preset threshold are removed.

3. The method according to claim 2, characterized in that Scanning the first positioning block and the second positioning block in rows and columns respectively to obtain the edge line of the square shell frame includes: Scanning the first positioning block row by row to obtain the transverse edge of the first positioning block and fitting the edge, and determining the first edge line and the third edge line of the square shell frame according to the fitting result; Scanning the second positioning block row by row to obtain the transverse edge of the second positioning block and fitting the same, and determining the second edge line and the fourth edge line of the square shell material frame according to the fitting results; Scanning the first positioning blocks in columns to obtain vertical edges of the first positioning blocks and performing fitting, and determining a fifth edge line of the square shell frame according to the fitting results; The second positioning blocks are scanned in columns to obtain vertical edges of the second positioning blocks and perform fitting, and the sixth edge line of the square shell frame is determined according to the fitting results.

4. The method according to claim 1, wherein Determining the reference plane of the upper surface of each positioning block includes: Performing plane extraction on the positioning block to determine the initial plane and plane parameters; Obtaining the distance between each point in the positioning block and the initial plane, and marking the points whose distance is less than a preset distance threshold as points in the initial plane; Based on an iterative algorithm, with the initial plane as the initial value and the maximum number of points in the plane as the goal, the plane parameters are modified to determine the reference plane.

5. A square shell material frame detection system based on machine vision, characterized in that: The system comprises: The data acquisition module is used to acquire the three-dimensional point cloud data and the color two-dimensional image of the square shell material frame, and generate a color three-dimensional model based on the three-dimensional point cloud data and the color two-dimensional image, including: Acquire three-dimensional point cloud data of the square shell material frame through a 3D vision sensor, and acquire a color two-dimensional image of the square shell material frame through a 2D area array camera; perform distortion correction on the color two-dimensional image according to the internal parameters and distortion parameters of the 2D area array camera to obtain a color-corrected image; convert the three-dimensional point cloud data into the camera coordinate system of the 2D area array camera based on the external parameters of the 3D vision sensor, and generate a color three-dimensional model according to the converted three-dimensional point cloud data and the color-corrected image; a segmentation module, configured to segment the color three-dimensional model to obtain at least two positioning blocks, wherein the positioning blocks include a first positioning block and a second positioning block; a size detection module, configured to obtain edge lines of the square shell material frame based on the positioning blocks, and determine the size of the square shell material frame according to the edge lines, including: scanning the first positioning blocks and the second positioning blocks in rows and columns respectively to obtain edge lines of the square shell material frame; determining the intersection points of the edge lines, and determining the length and width of the square shell material frame based on the distance between the intersection points; The defect detection module is used to determine the reference plane of the upper surface of each positioning block, and judge whether there is a defect on the surface of the square shell material frame corresponding to each positioning block based on the reference plane, including: when the square shell material frame is a battery material frame, Determining the position of the bayonet in the positioning block according to the position of the intersection; Based on the bayonet position, generating a gasket area at the bottom of the battery material frame, and obtaining depth information of the gasket area from the color three-dimensional model; Extracting the plane where the gasket is located based on a pre-built gasket detection model and the depth information to obtain a gasket plane; Determining the depth of the battery material frame according to the reference plane of the gasket plane and the upper surface of the positioning block; It is determined based on the reference plane whether the square shell material frame has any damage and whether the gasket at the bottom of the battery material frame is missing.

6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the square shell material frame detection method based on machine vision according to any one of claims 1 to 4 is implemented.

7. A storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the square shell material frame detection method based on machine vision as described in any one of claims 1 to 4 is implemented.

Images