Square shell material frame detection method based on machine vision

Abstract

The invention relates to a square shell material frame detection method based on machine vision, and the method comprises the steps: obtaining three-dimensional point cloud data and a color two-dimensional image of a square shell material frame, generating a color three-dimensional model according to the three-dimensional point cloud data and the color two-dimensional image, segmenting the color three-dimensional model to obtain at least two positioning blocks, and obtaining edge lines of the square shell material frame based on the positioning blocks, determining the size of the square shell material frame according to the edge lines, determining a reference plane of the upper surface of each positioning block, and judging whether the surface of the square shell material frame corresponding to each positioning block has defects based on the reference plane. According to the invention, the problem of low material frame detection efficiency is solved. The three-dimensional data and the color two-dimensional image of the material frame are obtained based on a machine vision scheme, the length, width and depth information of the material frame is calculated, whether the material frame positioning block is damaged or not is detected, and the detection efficiency of the material frame is improved through automatic detection.

Description

Technical Field

[0001] The present application relates to the field of product detection, and in particular to a square shell material frame detection method based on machine vision. Background Art

[0002] The material frame is used to place and transport products in the production line, such as battery material frames. Before placing the battery in the material frame, it is necessary to first check the limit blocks, bottom pads and other structures of the material frame, including whether the limit size is accurate, whether the structure is abnormal, and whether the pads are missing, to avoid damage to the battery. In related technologies, the material frame still needs to be inspected manually, which is inefficient and poses safety risks.

[0003] Currently, no effective solution has been proposed for the problem of low efficiency of material frame detection in related technologies. Summary of the invention

[0004] The embodiments of the present application provide a method, system, electronic device and storage medium for detecting a square shell material frame based on machine vision, so as to at least solve the problem of low efficiency of material frame detection in the related art.

[0005] In a first aspect, an embodiment of the present application provides a method for detecting a square shell material frame based on machine vision, the method comprising: Acquire the three-dimensional point cloud data and the color two-dimensional image of the square shell material frame, and generate a color three-dimensional model according to the three-dimensional point cloud data and the color two-dimensional image; Segmenting the color three-dimensional model to obtain at least two positioning blocks; Obtaining an edge line of the square shell material frame based on the positioning block, and determining a size of the square shell material frame according to the edge line; A reference plane on the upper surface of each positioning block is determined, and based on the reference plane, it is determined whether there are defects on the surface of the square shell material frame corresponding to each positioning block.

[0006] In some embodiments, the acquiring of the three-dimensional point cloud data and the color two-dimensional image of the square shell material frame, and generating a color three-dimensional model according to the three-dimensional point cloud data and the color two-dimensional image comprises: Acquire the three-dimensional point cloud data of the square shell material frame by using a 3D visual sensor, and acquire the color two-dimensional image of the square shell material frame by using a 2D array camera; According to the internal parameters and distortion parameters of the 2D area array camera, distortion correction is performed on the color two-dimensional image to obtain a color corrected image; Based on the external parameters of the 3D vision sensor, the three-dimensional point cloud data is converted to the camera coordinate system of the 2D area array camera, and a color three-dimensional model is generated according to the converted three-dimensional point cloud data and the color corrected image.

[0007] In some embodiments, before converting the three-dimensional point cloud data to the camera coordinate system of the 2D area array camera, the method further includes: The number of neighboring points within a preset radius threshold of each point in the three-dimensional point cloud data is counted, and points whose number of neighboring points is lower than the preset number threshold are removed.

[0008] In some embodiments, the square shell material frame is a battery material frame; the positioning block includes a first positioning block and a second positioning block of the battery; obtaining the edge line of the square shell material frame based on the positioning block, and determining the size of the square shell material frame according to the edge line includes: Scanning the first positioning block and the second positioning block respectively to obtain the edge line of the square shell material frame; The intersection points of the edge lines are determined, and the length and width of the square shell material frame are determined based on the distance between the intersection points.

[0009] In some embodiments, scanning the first positioning block and the second positioning block respectively to obtain the edge line of the square shell material frame includes: Scan the first positioning block row by row to obtain the transverse edge of the first positioning block and fit it, and determine the first edge line and the third edge line of the square shell material frame according to the fitting result; Scan the second positioning block row by row to obtain the transverse edge of the second positioning block and fit it, and determine the second edge line and the fourth edge line of the square shell material frame according to the fitting result; Scan the first positioning block by column, obtain the vertical edge of the first positioning block and fit it, and determine the fifth edge line of the square shell material frame according to the fitting result; The second positioning blocks are scanned in columns to obtain and fit the vertical edges of the second positioning blocks, and the sixth edge line of the square shell material frame is determined according to the fitting result.

[0010] In some embodiments, determining the reference plane of the upper surface of each positioning block includes: Performing plane extraction on the positioning block to determine the initial plane and plane parameters; Obtaining the distance between each point in the positioning block and the initial plane, and marking the points whose distance is less than a preset distance threshold as points in the initial plane; Based on an iterative algorithm, the initial plane is used as an initial value, the maximum number of points in the plane is used as a goal, the plane parameters are modified, and the reference plane is determined.

[0011] In some embodiments, the square shell material frame is a battery material frame; the method further comprises: Determining the position of the bayonet in the positioning block according to the position of the intersection; Based on the bayonet position, a gasket area at the bottom of the battery material frame is generated, and depth information of the gasket area is obtained from the three-dimensional color model; Based on the pre-built gasket detection model and the depth information, the plane where the gasket is located is extracted to obtain the gasket plane; The depth of the battery material frame is determined according to the reference plane of the gasket plane and the upper surface of the positioning block.

[0012] In a second aspect, an embodiment of the present application provides a square shell material frame detection system based on machine vision, the system comprising: A data acquisition module, used to acquire the three-dimensional point cloud data and the color two-dimensional image of the square shell material frame, and generate a color three-dimensional model according to the three-dimensional point cloud data and the color two-dimensional image; A segmentation module, used for segmenting the color three-dimensional model to obtain at least two positioning blocks; A size detection module, used for obtaining an edge line of the square shell material frame based on the positioning block, and determining a size of the square shell material frame according to the edge line; The defect detection module is used to determine the reference plane on the upper surface of each positioning block, and judge whether there are defects on the surface of the square shell material frame corresponding to each positioning block based on the reference plane.

[0013] In a third aspect, an embodiment of the present application provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the square shell material frame detection method based on machine vision as described in the first aspect above is implemented.

[0014] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the square shell material frame detection method based on machine vision as described in the first aspect above.

[0015] Compared with the related art, the square shell material frame detection method based on machine vision provided in the embodiment of the present application obtains the three-dimensional point cloud data and the color two-dimensional image of the square shell material frame, generates a color three-dimensional model according to the three-dimensional point cloud data and the color two-dimensional image, divides the color three-dimensional model to obtain at least two positioning blocks, obtains the edge line of the square shell material frame based on the positioning blocks, determines the size of the square shell material frame according to the edge line, determines the reference plane on the upper surface of each positioning block, and determines whether the surface of the square shell material frame corresponding to each positioning block has defects based on the reference plane, thereby solving the problem of low material frame detection efficiency. The three-dimensional data and the color two-dimensional image of the material frame are obtained based on the machine vision solution, the length, width and depth information of the material frame are calculated, and the material frame positioning block is detected to see if it is damaged, thereby improving the detection efficiency of the material frame through automated detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings: Figure 1 is a flow chart of a square shell material frame detection method based on machine vision according to an embodiment of the present application; Figure 2 is a schematic diagram of a battery material frame according to an embodiment of the present application; Figure 3 is a schematic diagram of material frame size measurement according to an embodiment of the present application; Figure 4 is a schematic diagram of positioning block defect detection according to an embodiment of the present application; Figure 5 is a schematic diagram of bottom gasket missing detection according to an embodiment of the present application; Figure 6 This is a square shell battery material frame measurement flow chart according to an embodiment of the present application; Figure 7 is a structural block diagram of a square shell material frame detection system based on machine vision according to an embodiment of the present application; Figure 8 It is a schematic diagram of the internal structure of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0017] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application is described and illustrated below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. Based on the embodiments provided in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application.

[0018] Obviously, the drawings described below are only some examples or embodiments of the present application. For ordinary technicians in this field, the present application can also be applied to other similar scenarios based on these drawings without creative work. In addition, it can also be understood that although the efforts made in this development process may be complicated and lengthy, for ordinary technicians in this field related to the content disclosed in this application, some changes in design, manufacturing or production based on the technical content disclosed in this application are just conventional technical means, and should not be understood as insufficient content disclosed in this application.

[0019] Reference to "embodiments" in this application means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those of ordinary skill in the art that the embodiments described in this application may be combined with other embodiments without conflict.

[0020] Unless otherwise defined, the technical terms or scientific terms involved in this application should be understood by people with ordinary skills in the technical field to which this application belongs. The words "one", "a", "a", "the" and the like involved in this application do not indicate a quantity limitation, and may indicate the singular or plural. The terms "include", "comprise", "have" and any of their variations involved in this application are intended to cover non-exclusive inclusions; for example, a process, method, system, product or device that includes a series of steps or modules (units) is not limited to the listed steps or units, but may also include steps or units that are not listed, or may also include other steps or units inherent to these processes, methods, products or devices. The words "connect", "connected", "coupled" and the like involved in this application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. The "multiple" involved in this application refers to two or more. "And / or" describes the association relationship of associated objects, indicating that there may be three relationships, for example, "A and / or B" can mean: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the objects before and after are in an "or" relationship. The terms "first", "second", "third", etc. involved in this application are only used to distinguish similar objects and do not represent a specific ordering of the objects.

[0021] This embodiment provides a square shell material frame detection method based on machine vision. Figure 1 is a flow chart of a method for detecting a square shell material frame based on machine vision according to an embodiment of the present application, such as Figure 1 As shown, the process includes the following steps: Step S101, obtaining the three-dimensional point cloud data and the color two-dimensional image of the square shell material frame, and generating a color three-dimensional model according to the three-dimensional point cloud data and the color two-dimensional image.

[0022] In some embodiments, step S101 specifically includes: Step S1011, obtaining three-dimensional point cloud data of the square shell material frame through a 3D vision sensor, and obtaining a color two-dimensional image of the square shell material frame through a 2D area array camera.

[0023] Optionally, a 3D vision sensor is used to obtain the 3D point cloud data of the square shell battery material frame, and an external RGB color 2D array camera is used to obtain a color 2D image of the material frame. The internal parameters, distortion parameters and external parameters of the 3D vision sensor and the external 2D array camera are calculated by camera calibration.

[0024] Step S1012, performing distortion correction on the color two-dimensional image according to the internal parameters and distortion parameters of the 2D area array camera to obtain a color corrected image.

[0025] Step S1013, based on the external parameters of the 3D vision sensor, the three-dimensional point cloud data is converted to the camera coordinate system of the 2D area array camera, and a color three-dimensional model is generated according to the converted three-dimensional point cloud data and the color corrected image.

[0026] According to the external parameters of the 3D vision sensor, the 3D point cloud data of the material frame is converted to the world coordinate system, and then converted to the camera coordinate system of the 2D array camera. The 3D point cloud data after coordinate conversion is matched with the color 2D image pixels to obtain a depth image with color information, that is, a color 3D model.

[0027] In some of the embodiments, before converting the 3D point cloud data into the camera coordinate system of the 2D area array camera, the method further includes: Count the number of neighboring points within a preset radius threshold for each point in the 3D point cloud data, and remove points whose number of neighboring points is lower than the preset threshold.

[0028] Remove outliers or noise points whose number of neighborhood points is lower than the threshold value and retain the main part of the frame.

[0029] After the color 3D model is built, the 3D space range is set according to the estimated frame size, and thresholds are set in the x, y, and z directions respectively to remove invalid data outside the frame range and filter out interfering background to reduce the amount of data to be processed and improve processing efficiency.

[0030] Step S102, segmenting the color 3D model to obtain at least two positioning blocks.

[0031] Step S103, obtaining the edge line of the square shell material frame based on the positioning block, and determining the size of the square shell material frame according to the edge line.

[0032] In some embodiments, the square shell material frame is a battery material frame; the positioning block includes a first positioning block and a second positioning block of the battery; step S103 specifically includes: Step S1031 , scanning the first positioning block and the second positioning block respectively to obtain the edge line of the square shell material frame.

[0033] Figure 2It is a schematic diagram of a battery material frame according to an embodiment of the present application.

[0034] In some embodiments, step S1031 specifically includes: Step S201 , scanning the first positioning block row by row, obtaining the transverse edge of the first positioning block and fitting it, and determining the first edge line and the third edge line of the square shell material frame according to the fitting result.

[0035] Step S202 , scanning the second positioning block row by row, obtaining the lateral edge of the second positioning block and fitting it, and determining the second edge line and the fourth edge line of the square shell material frame according to the fitting result.

[0036] Step S203 , scanning the first positioning block by column, obtaining the vertical edge of the first positioning block and fitting it, and determining the fifth edge line of the square shell material frame according to the fitting result.

[0037] Step S204, scanning the second positioning block by column, obtaining and fitting the vertical edge of the second positioning block, and determining the sixth edge line of the square shell material frame according to the fitting result.

[0038] Figure 3 is a schematic diagram of measuring the size of a material frame according to an embodiment of the present application, such as Figure 3 As shown, after segmenting to obtain the positioning block part, the upper and lower lateral edges of the left positioning block (first positioning block) are obtained by line scanning in the scanning area 2, and the straight line l is obtained by fitting. 1 and l 3 ; Scan the scanning area 3 by line to obtain the upper and lower lateral edges of the right positioning block (the second positioning block), and fit the straight line l 2 and l 4 ; Scan the left and right vertical edges of the positioning blocks by column in scanning area 1, and fit the straight line l 5 and l 6 .

[0039] Step S1032, determining the intersection points of the edge lines, and determining the length and width of the square shell frame based on the distance between the intersection points.

[0040] by Figure 3 For example, calculate l 1 and l 5 , l 3 and l 5 The corresponding coordinates of the intersection points 1 and 3 of the two sets of straight lines are calculated respectively. 2 and l 6 , l 4 and l 6 The coordinates of the intersection points 2 and 4 of the two sets of straight lines correspond to each other, and the length and width of a battery position are calculated based on the coordinates of the four intersection points.

[0041] Continue to refer to Figure 1After determining the size of the square shell material frame, execute step S104.

[0042] Step S104, determining the reference plane on the upper surface of each positioning block, and judging whether there are defects on the surface of the square shell material frame corresponding to each positioning block based on the reference plane.

[0043] In some embodiments, determining the reference plane on the upper surface of each positioning block in step S104 includes: Step S1041, extracting a plane from the positioning block to determine an initial plane and plane parameters.

[0044] Step S1042, obtaining the distance between each point in the positioning block and the initial plane, and marking the points that are less than a preset distance threshold as points in the initial plane.

[0045] Step S1043, based on an iterative algorithm, with the initial plane as the initial value and the maximum number of points in the plane as the goal, modify the plane parameters and determine the reference plane.

[0046] Perform plane extraction on a single positioning block. Optionally, randomly select three points in the three-dimensional data of the positioning block, and the spatial plane equation is known to be: ax+by+cz+d=0, and calculate the initial plane parameters based on the equation.

[0047] Modify the plane parameters, calculate the distance from the remaining points to the plane corresponding to the current plane parameters, mark the points less than the preset distance threshold as plane internal points, and count the number of internal points under the current plane parameters. When the number of internal points is greater than the previous maximum number of internal points, iterate the plane equation based on all current internal points until the current plane contains the largest number of internal points or exceeds the preset number of iterations, then stop iterating, and the final result is the parameter corresponding to the reference plane.

[0048] When the positioning block is defective or damaged, it usually causes the local surface of the positioning block to be uneven. After calculating the reference plane of the upper surface, the surface state of the positioning block can be detected, and the damaged part can be extracted based on the reference plane to prevent the defective material frame from causing damage to the battery. Figure 4 This is a schematic diagram of positioning block defect detection according to an embodiment of the present application.

[0049] In some embodiments, the square shell material frame is a battery material frame; the method further comprises: Step S301, determining the position of the bayonet in the positioning block according to the position of the intersection.

[0050] Step S302, based on the position of the bayonet, generate the gasket area at the bottom of the battery material frame, and obtain the depth information of the gasket area from the three-dimensional color model.

[0051] Step S303: extracting the plane where the gasket is located based on the pre-built gasket detection model and depth information to obtain the gasket plane.

[0052] Step S304, determining the depth of the battery material frame according to the reference plane of the gasket plane and the upper surface of the positioning block.

[0053] The positions of the bayonet in the left and right positioning blocks are determined based on the four intersection points of each battery position, and the bottom gasket detection area is established. Optionally, some material frames use gasket materials with eye-catching colors, and this feature is used to check the color information in the color 2D image to improve the accuracy and efficiency of the detection.

[0054] Optionally, the gasket plane is determined by a gasket detection model. Gasket area images under actual working conditions are collected, a deep learning data set is constructed, and a gasket detection model with high accuracy and good robustness is trained to achieve a stable detection effect.

[0055] During the inspection, based on the gasket detection model, the depth information of the corresponding area is used to extract the gasket plane, calculate the height difference between the gasket plane and the upper surface of the positioning block, and output the depth information of the corresponding battery material frame. And determine whether the gasket of the positioning block is missing. Figure 5 This is a schematic diagram of bottom gasket missing detection according to an embodiment of the present application.

[0056] Through the above method, three-dimensional point cloud data and a color two-dimensional image of a square shell material frame are obtained, a color three-dimensional model is generated according to the three-dimensional point cloud data and the color two-dimensional image, the color three-dimensional model is segmented to obtain at least two positioning blocks, the edge line of the square shell material frame is obtained based on the positioning blocks, the size of the square shell material frame is determined according to the edge line, the reference plane on the upper surface of each positioning block is determined, and it is judged based on the reference plane whether there is a defect on the surface of the square shell material frame corresponding to each positioning block, thereby solving the problem of low efficiency of material frame detection. Figure 6 This is a square shell battery material frame measurement flow chart according to an embodiment of the present application.

[0057] Based on the machine vision solution, the three-dimensional data and color two-dimensional image of the material frame are obtained, the length, width and depth information of the material frame are calculated, and the material frame positioning block is detected to see if it is damaged and the gasket is in place, thereby improving the detection efficiency and accuracy of the material frame.

[0058] It should be noted that the steps shown in the above process or the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0059] This embodiment also provides a square shell material frame detection system based on machine vision, which is used to implement the above embodiments and preferred implementations, and will not be repeated here. As used below, the terms "module", "unit", "subunit", etc. can implement a combination of software and / or hardware for a predetermined function. Although the device described in the following embodiments is preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0060] Figure 7 is a structural block diagram of a square shell material frame detection system based on machine vision according to an embodiment of the present application, such as Figure 7 As shown, the system includes: The data acquisition module 71 is used to acquire the three-dimensional point cloud data and the color two-dimensional image of the square shell material frame, and generate a color three-dimensional model according to the three-dimensional point cloud data and the color two-dimensional image.

[0061] The segmentation module 72 is used to segment the color three-dimensional model to obtain at least two positioning blocks.

[0062] The size detection module 73 is used to obtain the edge line of the square shell material frame based on the positioning block, and determine the size of the square shell material frame according to the edge line.

[0063] The defect detection module 74 is used to determine the reference plane on the upper surface of each positioning block, and judge whether there are defects on the surface of the square shell material frame corresponding to each positioning block based on the reference plane.

[0064] In some embodiments, the data acquisition module 71 includes: The acquisition module is used to obtain the three-dimensional point cloud data of the square shell material frame through the 3D visual sensor, and to obtain the color two-dimensional image of the square shell material frame through the 2D array camera.

[0065] The correction module is used to perform distortion correction on the color two-dimensional image according to the internal parameters and distortion parameters of the 2D array camera to obtain a color corrected image.

[0066] The model generation module is used to convert the three-dimensional point cloud data into the camera coordinate system of the 2D array camera based on the external parameters of the 3D vision sensor, and generate a color three-dimensional model according to the converted three-dimensional point cloud data and the color corrected image.

[0067] In some embodiments, the data acquisition module 71 further includes: The preprocessing module is used to count the number of neighboring points of each point in the three-dimensional point cloud data within a preset radius threshold, and remove points whose number of neighboring points is lower than the preset number threshold.

[0068] In some embodiments, the square shell material frame is a battery material frame; the positioning block includes a first positioning block and a second positioning block of the battery; the size detection module 73 includes: The scanning module is used to scan the first positioning block and the second positioning block respectively to obtain the edge line of the square shell material frame.

[0069] The measuring module is used to determine the intersection points of the edge lines and determine the length and width of the square shell frame based on the distance between the intersection points.

[0070] In some embodiments, the scanning module includes: The first scanning module is used to scan the first positioning block row by row, obtain the transverse edge of the first positioning block and fit it, and determine the first edge line and the third edge line of the square shell material frame according to the fitting result.

[0071] The second scanning module is used to scan the second positioning block row by row, obtain the transverse edge of the second positioning block and fit it, and determine the second edge line and the fourth edge line of the square shell material frame according to the fitting result.

[0072] The third scanning module is used to scan the first positioning block in columns, obtain the vertical edge of the first positioning block and fit it, and determine the fifth edge line of the square shell material frame according to the fitting result.

[0073] The fourth scanning module is used to scan the second positioning block in columns, obtain the vertical edge of the second positioning block and fit it, and determine the sixth edge line of the square shell material frame according to the fitting result.

[0074] In some embodiments, the defect detection module 74 includes: The initial value module is used to extract the plane of the positioning block and determine the initial plane and plane parameters.

[0075] The statistical module is used to obtain the distance between each point in the positioning block and the initial plane, and mark the points that are less than a preset distance threshold as points in the initial plane.

[0076] The iteration module is used to modify the plane parameters and determine the reference plane based on an iterative algorithm, taking the initial plane as the initial value and the maximum number of points in the plane as the goal.

[0077] In some embodiments, the square shell material frame is a battery material frame; the system further comprises: The bayonet detection module is used to determine the bayonet position in the positioning block according to the position of the intersection.

[0078] The depth information acquisition module is used to generate the gasket area at the bottom of the battery material frame based on the bayonet position and obtain the depth information of the gasket area from the three-dimensional color model.

[0079] The plane extraction module is used to extract the plane where the gasket is located based on the pre-built gasket detection model and depth information to obtain the gasket plane.

[0080] The depth determination module is used to determine the depth of the battery material frame according to the reference plane of the gasket plane and the upper surface of the positioning block.

[0081] Through the above system, the data acquisition module 71 acquires the three-dimensional point cloud data and color two-dimensional image of the square shell material frame, generates a color three-dimensional model based on the three-dimensional point cloud data and the color two-dimensional image, and the segmentation module 72 segments the color three-dimensional model to obtain at least two positioning blocks. The size detection module 73 obtains the edge line of the square shell material frame based on the positioning block, and determines the size of the square shell material frame based on the edge line. The defect detection module 74 determines the reference plane on the upper surface of each positioning block, and determines whether there is a defect on the surface of the square shell material frame corresponding to each positioning block based on the reference plane, thereby solving the problem of low efficiency of material frame detection. Based on the machine vision solution, the three-dimensional data and color two-dimensional image of the material frame are acquired, the length, width, and depth information of the material frame are calculated, and the material frame positioning block is detected to see if it is damaged, thereby improving the detection efficiency of the material frame through automated detection.

[0082] It should be noted that the above modules can be functional modules or program modules, and can be implemented by software or hardware. For modules implemented by hardware, the above modules can be located in the same processor; or the above modules can be located in different processors in any combination.

[0083] This embodiment further provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0084] Optionally, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0085] Optionally, in this embodiment, the processor may be configured to perform the following steps through a computer program: S1, obtaining the three-dimensional point cloud data and the color two-dimensional image of the square shell material frame, and generating a color three-dimensional model according to the three-dimensional point cloud data and the color two-dimensional image.

[0086] S2, segmenting the color 3D model to obtain at least two positioning blocks.

[0087] S3, obtaining the edge line of the square shell material frame based on the positioning block, and determining the size of the square shell material frame according to the edge line.

[0088] S4, determining the reference plane on the upper surface of each positioning block, and judging whether there is a defect on the surface of the square shell material frame corresponding to each positioning block based on the reference plane.

[0089] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementation modes, and this embodiment will not be described in detail here.

[0090] In one embodiment, Figure 8 is a schematic diagram of the internal structure of an electronic device according to an embodiment of the present application, such as Figure 8 As shown, an electronic device is provided, which may be a server, and its internal structure diagram may be as shown in Figure 8 As shown. The electronic device includes a processor, a memory, a network interface and a database connected through a system bus. Among them, the processor of the electronic device is used to provide computing and control capabilities. The memory of the electronic device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the electronic device is used to store data. The network interface of the electronic device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a square shell material frame detection method based on machine vision is implemented.

[0091] Those skilled in the art will understand that Figure 8 The structure shown in the figure is merely a block diagram of a partial structure related to the scheme of the present application, and does not constitute a limitation on the electronic device to which the scheme of the present application is applied. The specific electronic device may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.

[0092] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0093] Those skilled in the art should understand that the technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0094] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A method for detecting a square shell frame based on machine vision, characterized in that: The method comprises: Acquire the three-dimensional point cloud data and the color two-dimensional image of the square shell material frame, and generate a color three-dimensional model according to the three-dimensional point cloud data and the color two-dimensional image; Segmenting the color three-dimensional model to obtain at least two positioning blocks; Obtaining an edge line of the square shell material frame based on the positioning block, and determining a size of the square shell material frame according to the edge line; A reference plane on the upper surface of each positioning block is determined, and based on the reference plane, it is determined whether there are defects on the surface of the square shell material frame corresponding to each positioning block.

2. The method according to claim 1, characterized in that: The step of acquiring the three-dimensional point cloud data and the color two-dimensional image of the square shell material frame and generating a color three-dimensional model according to the three-dimensional point cloud data and the color two-dimensional image comprises: Acquire the three-dimensional point cloud data of the square shell material frame by using a 3D visual sensor, and acquire the color two-dimensional image of the square shell material frame by using a 2D array camera; According to the internal parameters and distortion parameters of the 2D area array camera, distortion correction is performed on the color two-dimensional image to obtain a color corrected image; Based on the external parameters of the 3D vision sensor, the three-dimensional point cloud data is converted to the camera coordinate system of the 2D area array camera, and a color three-dimensional model is generated according to the converted three-dimensional point cloud data and the color corrected image.

3. The method according to claim 1, characterized in that Before converting the three-dimensional point cloud data into the camera coordinate system of the 2D area array camera, the method further includes: The number of neighboring points within a preset radius threshold of each point in the three-dimensional point cloud data is counted, and points whose number of neighboring points is lower than the preset number threshold are removed.

4. The method according to claim 1, characterized in that: The square shell material frame is a battery material frame; the positioning block includes a first positioning block and a second positioning block of the battery; the edge line of the square shell material frame is obtained based on the positioning block, and the size of the square shell material frame is determined according to the edge line. Scanning the first positioning block and the second positioning block respectively to obtain the edge line of the square shell material frame; The intersection points of the edge lines are determined, and the length and width of the square shell material frame are determined based on the distance between the intersection points.

5. The method according to claim 4, characterized in that The step of respectively scanning the first positioning block and the second positioning block to obtain the edge line of the square shell material frame comprises: Scan the first positioning block row by row to obtain the transverse edge of the first positioning block and fit it, and determine the first edge line and the third edge line of the square shell material frame according to the fitting result; Scan the second positioning block row by row to obtain the transverse edge of the second positioning block and fit it, and determine the second edge line and the fourth edge line of the square shell material frame according to the fitting result; Scan the first positioning block by column, obtain the vertical edge of the first positioning block and fit it, and determine the fifth edge line of the square shell material frame according to the fitting result; The second positioning blocks are scanned in columns to obtain and fit the vertical edges of the second positioning blocks, and the sixth edge line of the square shell material frame is determined according to the fitting result.

6. The method according to claim 1, characterized in that Determining the reference plane on the upper surface of each positioning block includes: Performing plane extraction on the positioning block to determine the initial plane and plane parameters; Obtaining the distance between each point in the positioning block and the initial plane, and marking the points whose distance is less than a preset distance threshold as points in the initial plane; Based on an iterative algorithm, the initial plane is used as an initial value, the maximum number of points in the plane is used as a goal, the plane parameters are modified, and the reference plane is determined.

7. The method according to claim 6, characterized in that The square shell material frame is a battery material frame; the method further comprises: Determining the position of the bayonet in the positioning block according to the position of the intersection; Based on the bayonet position, a gasket area at the bottom of the battery material frame is generated, and depth information of the gasket area is obtained from the three-dimensional color model; Based on the pre-built gasket detection model and the depth information, the plane where the gasket is located is extracted to obtain the gasket plane; The depth of the battery material frame is determined according to the reference plane of the gasket plane and the upper surface of the positioning block.

8. A square shell material frame detection system based on machine vision, characterized in that: The system comprises: A data acquisition module, used to acquire the three-dimensional point cloud data and the color two-dimensional image of the square shell material frame, and generate a color three-dimensional model according to the three-dimensional point cloud data and the color two-dimensional image; A segmentation module, used for segmenting the color three-dimensional model to obtain at least two positioning blocks; A size detection module, used for obtaining an edge line of the square shell material frame based on the positioning block, and determining a size of the square shell material frame according to the edge line; The defect detection module is used to determine the reference plane on the upper surface of each positioning block, and judge whether there are defects on the surface of the square shell material frame corresponding to each positioning block based on the reference plane.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the square shell material frame detection method based on machine vision as described in any one of claims 1 to 7 is implemented.

10. A storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, a square shell material frame detection method based on machine vision as described in any one of claims 1 to 7 is implemented.

Images