Automatic detection system for automobile frame production defects based on visual recognition

Abstract

The present invention discloses an automatic detection system for automobile frame production defects based on visual recognition, which relates to the technical field related to image detection. The system includes: a three-dimensional modeling module for performing three-dimensional modeling of the target automobile seat frame; a camera distribution analysis module for determining the distribution characteristics of the camera module; a posture fitting module for performing skeleton transmission posture fitting based on a standard skeleton three-dimensional model to construct multiple automobile frame transmission postures; a coverage area fitting module for performing image coverage area fitting to construct a target transmission posture combination and a target skeleton coverage area combination; and a cross-type detection module for performing cross-type detection of multiple postures. The system solves the technical problems existing in the prior art of difficulty in fully capturing skeleton defect information, incomplete defect detection coverage due to the variability of skeleton transmission postures, and poor defect detection efficiency and accuracy, thereby achieving the technical effect of improving the accuracy and efficiency of automobile frame production defect detection.

Description

Technical Field

[0001] The present application relates to the technical field related to image detection, and in particular to an automatic detection system for automobile frame production defects based on visual recognition. Background Art

[0002] The automotive seat frame is a key load-bearing structural component of the vehicle interior, and its quality impacts seat safety, comfort, and user experience. Traditional production defect detection methods based on manual or simple mechanical inspection are inefficient, highly subjective, and susceptible to fatigue. These methods struggle to ensure consistent and accurate test results, and cannot meet the high-efficiency inspection requirements of large-scale assembly line production. They can also damage the frame and have limited detection capabilities for complex shapes and minor defects. Visual recognition technology is rapidly developing in the industrial inspection field. However, automotive seat frames are complex, with numerous irregular curved surfaces and hidden areas. Single-view visual inspection struggles to fully capture surface defects. Furthermore, in actual production lines, the frame's transport posture varies greatly, making it difficult to ensure that the camera module fully covers the frame surface for efficient and accurate defect detection.

[0003] Therefore, in the current related technologies, there are technical problems such as difficulty in fully capturing skeleton defect information and variable skeleton transmission postures resulting in incomplete defect detection coverage, leading to poor defect detection efficiency and accuracy. Summary of the Invention

[0004] This application solves the technical problems in the prior art, such as the difficulty in fully capturing skeleton defect information, the variable skeleton transmission posture resulting in incomplete defect detection coverage, and poor defect detection efficiency and accuracy, by providing an automatic detection system for automobile skeleton production defects based on visual recognition. This achieves the technical effect of improving the accuracy and efficiency of automobile skeleton production defect detection.

[0005] The present application provides an automatic detection system for automobile frame production defects based on visual recognition, and the system includes: a three-dimensional modeling module, which is used to perform three-dimensional modeling of the target automobile seat frame to be inspected and generate a standard skeleton three-dimensional model; a camera distribution analysis module, which is used to determine the camera module distribution characteristics of the camera modules in the defect detection platform relative to the skeleton conveyor belt; a posture fitting module, which is used to perform skeleton transmission posture fitting based on the standard skeleton three-dimensional model and construct multiple automobile skeleton transmission postures that meet preset stability requirements; a coverage area fitting module, which is used to perform image coverage area fitting for the multiple automobile skeleton transmission postures based on the camera module distribution characteristics and construct a target transmission posture combination that meets the skeleton coverage requirements and a corresponding target skeleton coverage area combination; a cross-detection module, which is used to perform cross-detection of multiple postures of the target automobile seat frame based on the target transmission posture combination and the target skeleton coverage area combination to generate a target defect detection result.

[0006] In a possible implementation, the automatic detection system for automobile frame production defects based on visual recognition also performs the following processing: collecting structural material characteristics of each position of the target automobile seat frame and constructing skeleton weight distribution information; performing contact surface fitting with the skeleton conveyor belt based on the standard skeleton three-dimensional model to determine the contact support posture set; extracting the first contact support posture in the contact support posture set, and performing adaptation judgment between the center of gravity and the support point in combination with the skeleton weight distribution information to generate a first adaptation judgment result; if the first adaptation judgment result is adaptation, adding the first contact support posture to the multiple automobile frame transmission postures.

[0007] In a possible implementation, the automatic detection system for automobile frame production defects based on visual recognition also performs the following processing: constructing a center of gravity recognition channel, which trains a convolutional neural network model based on historical contact posture data, historical weight distribution data and corresponding center of gravity identification information; using the center of gravity recognition channel to analyze the contact support posture and the skeleton weight distribution information, and output a first center of gravity distribution position; based on the first center of gravity distribution position, predicting the matching support point, judging whether the first prediction result is compatible with the support point in the first contact support posture, and generating the first adaptation judgment result.

[0008] In a possible implementation, the automatic detection system for automobile skeleton production defects based on visual recognition also performs the following processing: extracting skeleton edge fulcrums in various directions from the standard skeleton three-dimensional model; performing contact surface fitting in various directions based on the skeleton edge fulcrums in various directions to generate a fitted contact surface set, wherein the fitted contact surface is constructed by N fulcrums, and N is an integer greater than or equal to 3; based on the standard skeleton three-dimensional model, performing three-dimensional model rotation with the contact surfaces in the fitted contact surface set to generate the contact support posture set.

[0009] In a possible implementation, the automatic detection system for automobile skeleton production defects based on visual recognition also performs the following processing: collecting the camera parameters of the camera module, performing digital twin modeling in combination with the distribution characteristics of the camera module, and constructing a twin camera model; using the twin camera model to perform camera simulation on the multiple automobile skeleton transmission postures, and determining multiple image coverage areas based on the simulation results; performing combination optimization based on the multiple image coverage areas, determining the optimal combination that covers the entire surface of the automobile skeleton and has the least number of combinations, and generating the target transmission posture combination and the target skeleton coverage area combination.

[0010] In a possible implementation, the automatic detection system for automobile frame production defects based on visual recognition also performs the following processing: extracting any area from the multiple image coverage areas, and determining the missing area of ​​any area relative to the complete surface of the automobile frame; performing missing matching with the missing area in other areas except the any area, and generating a missing matching result; generating a set of area combination results with the any area and the missing matching result; and screening the area combination result that covers the complete surface of the automobile frame and has the least number of combinations in the area combination result set as the optimal combination.

[0011] In a possible implementation, the visual recognition-based automatic detection system for automobile skeleton production defects also performs the following processing: performing missing matching on the missing area in other areas other than any of the areas, including combined matching of more than one other area and individual matching of one other area.

[0012] In a possible implementation, the automatic detection system for automobile frame production defects based on visual recognition also performs the following processing: extracting multiple target postures from the target transmission posture combination, and multiple skeleton coverage areas from the target skeleton coverage area combination; collecting the total production volume and sampling inspection indicators of the production batch corresponding to the target automobile seat frame, and calculating the number of sampling inspections of multiple target postures based on the defect occurrence probability of the multiple skeleton coverage areas to generate a first cross-type detection scheme; based on the first cross-type detection scheme, performing image acquisition according to the multiple target postures to generate multiple image acquisition data sets; performing defect detection and marking based on the multiple image acquisition data sets to generate the target defect detection results.

[0013] In a possible implementation, the automatic detection system for automobile skeleton production defects based on visual recognition also performs the following processing: determining the preset image acquisition environment light information, and obtaining multiple defect-free target sample images in combination with the multiple target postures; constructing multiple calibrated pixel distribution features of the multiple skeleton coverage areas based on the multiple target sample images; performing pixel distribution difference analysis on the multiple image acquisition data sets using the multiple calibrated pixel distribution features, determining samples whose pixel distribution differences are greater than or equal to a preset difference threshold, marking them as defects, and generating the target defect detection results.

[0014] In a possible implementation, the automatic automobile frame production defect detection system based on visual recognition also performs the following processing: based on the multiple target postures, the camera module position and posture change position are fitted on the skeleton conveyor belt to generate a second cross-type detection scheme; based on the second cross-type detection scheme, the same target automobile seat frame is subjected to continuous transformation detection of multiple postures to generate corresponding detection images, and defect detection is performed using the multiple calibrated pixel distribution features.

[0015] The automatic detection system for automobile skeleton production defects based on visual recognition proposed in this application includes a three-dimensional modeling module for performing three-dimensional modeling of the target automobile seat skeleton; a camera distribution analysis module for determining the distribution characteristics of the camera module relative to the skeleton conveyor belt; a posture fitting module for performing skeleton transmission posture fitting based on the standard skeleton three-dimensional model to construct multiple automobile skeleton transmission postures; a coverage area fitting module for performing image coverage area fitting to construct target transmission posture combinations and target skeleton coverage area combinations; and a cross-detection module for performing cross-detection of multiple postures to generate target defect detection results. This solves the technical problems existing in the prior art, such as the difficulty in fully capturing skeleton defect information, the incomplete defect detection coverage caused by the variability of skeleton transmission postures, and the poor defect detection efficiency and accuracy, thereby achieving the technical effect of improving the accuracy and efficiency of automobile skeleton production defect detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] To more clearly illustrate the technical solutions of the embodiments of the present disclosure, the accompanying drawings of the embodiments of the present disclosure are briefly introduced below. Flowcharts are used in this application to illustrate the operations performed by the systems according to the embodiments of the present application. It should be understood that the preceding or following operations are not necessarily performed in precise order. Instead, various steps may be processed in reverse order or simultaneously as needed. Furthermore, other operations may be added to these processes, or one or more operations may be removed from these processes.

[0017] Figure 1 Schematic diagram of the structure of the automatic detection system for automobile frame production defects based on visual recognition provided in an embodiment of the present application.

[0018] Figure 2 A schematic diagram of the execution process of the posture fitting module in the automatic detection system for automobile skeleton production defects based on visual recognition provided in an embodiment of the present application.

[0019] Explanation of the reference numerals: three-dimensional modeling module 10 , camera distribution analysis module 20 , posture fitting module 30 , coverage area fitting module 40 , cross detection module 50 . DETAILED DESCRIPTION

[0020] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below.

[0021] In order to make the purpose, technical solutions and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limiting this application. All other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0022] In the following description, reference is made to “some embodiments”, which describes a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict, and the terms “first\second” involved are merely used to distinguish similar objects and do not represent a specific ordering of the objects. The terms “including” and “having” and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, system, product, or server that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or modules that are not clearly listed or that are inherent to these processes, products, or devices. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this application belongs. The terms used herein are for the purpose of describing the embodiments of this application only.

[0023] The embodiment of the present application provides an automatic detection system for automobile frame production defects based on visual recognition, such as Figure 1 As shown, the system includes:

[0024] The three-dimensional modeling module 10 is used to perform three-dimensional modeling on the target automobile seat frame to be inspected and generate a standard frame three-dimensional model.

[0025] Preferably, the task performed by the 3D modeling module is to use 3D modeling technology (such as computer-aided design software CAD, 3D scanning modeling software, etc.) to construct a 3D digital model of the target automobile seat frame that needs to be inspected for defects. Specifically, the target automobile seat frame is scanned in all directions by a 3D scanning device (such as a laser scanner, a structured light scanner, etc.) to obtain point cloud data of its surface, including the 3D coordinate information of each point on the surface of the automobile seat frame, and the precise size, shape and other data of the frame are obtained in combination with the design drawings; then the collected point cloud data is processed to remove noise points, fill in missing data, etc. to improve the quality and accuracy of the data, and then the processed point cloud data is surface fitted to construct a surface model of the automobile seat frame. The generated surface model is then optimized, mapped with the precise size of the frame and detailed features (such as holes, protrusions, grooves, etc.) are added to make it closer to the actual automobile seat frame shape, and finally a 3D digital model of the target automobile seat frame is generated, that is, a standard frame 3D model, which is used for comparison and analysis with the frame model or image obtained during the actual inspection process to determine whether there are defects and the location and type of the defects.

[0026] The camera distribution analysis module 20 is used to determine the distribution characteristics of the camera modules in the defect detection platform relative to the skeleton conveyor belt.

[0027] Preferably, the coordinate position of each camera in space is accurately measured by using a positioning device (such as a laser rangefinder, etc.), including the height of the camera from the conveyor belt surface, and the specific position in the length and width directions of the conveyor belt, that is, the position in the horizontal direction (such as the X-axis and Y-axis) and the height in the vertical direction (Z-axis). At the same time, the installation angle of the camera relative to the skeleton conveyor belt is measured, including the pitch angle, yaw angle and roll angle, etc. For example, the pitch angle of the camera determines its shooting angle of the top and side of the skeleton, and the yaw angle affects the coverage range in the length direction of the skeleton; then the arrangement of the camera modules is studied, whether it is a linear arrangement, a matrix arrangement or other special layouts. For example, in the defect detection platform, the cameras may be arranged linearly along both sides of the conveyor belt to achieve comprehensive shooting of both sides of the skeleton, or a matrix layout may be adopted to simultaneously shoot the skeleton from multiple angles; reasonable The spacing and overlapping area settings help ensure full coverage of the skeleton surface and avoid detection blind spots. Then, the camera module is analyzed to track and shoot the moving skeleton in real time during the movement of the skeleton conveyor belt. For example, the camera's shooting frequency and triggering timing are adjusted according to the speed and acceleration of the conveyor belt to ensure that the captured skeleton image is clear and complete, avoiding detection errors caused by motion blur or untimely shooting. It is also evaluated how to compensate for the visual deviation that may be caused by the movement of the conveyor belt by optimizing the camera distribution. For example, when the skeleton accelerates or decelerates on the conveyor belt, the angle or position of the camera is adjusted to ensure that the position and posture of the skeleton in the image are relatively stable. Then, theoretical calculations and experiments are combined to determine the optimal spacing between adjacent cameras and the degree of overlap of their shooting areas to ensure that complete skeleton image information can be obtained efficiently and accurately under different conveying postures.

[0028] The posture fitting module 30 is used to perform skeleton transmission posture fitting based on the standard skeleton three-dimensional model to construct multiple vehicle skeleton transmission postures that meet preset stability requirements.

[0029] Preferably, the preset stability requirement means that the skeleton cannot have excessive shaking, rolling or displacement during the transmission process, so as to ensure that the camera module can accurately capture various parts of the skeleton and the captured images can be used for accurate defect detection. The skeleton transmission posture fitting is performed based on the standard skeleton three-dimensional model. Specifically, the precise size, shape and structural information of the standard skeleton three-dimensional model is used to simulate various postures that the skeleton may have on the conveyor belt, that is, considering the distribution of the center of gravity of the skeleton, the position of the support points and the contact method with the conveyor belt, etc., and analyzing the stability of the skeleton under different postures. Then, different transmission postures are calculated based on the analysis results of the standard skeleton three-dimensional model and the simulation of the transmission process. For example, the optimal placement angle of the skeleton on the conveyor belt may be determined so that its center of gravity maintains a suitable relationship with the movement direction of the conveyor belt to reduce the instability factors caused by the offset of the center of gravity; at the same time, the relative position of each part of the skeleton and the camera module is considered, and finally a plurality of automobile skeleton transmission postures that meet the preset stability requirements are constructed to adapt to different production scenarios and detection requirements, and ensure that the skeleton can be effectively captured and detected by the camera module while being stably transmitted, thereby improving the accuracy and reliability of defect detection.

[0030] Further, such as Figure 2 As shown, the specific configuration of the posture fitting module 30 also includes collecting the structural material characteristics of each position of the target automobile seat frame to construct the skeleton weight distribution information; performing contact surface fitting with the skeleton conveyor belt based on the standard skeleton three-dimensional model to determine the contact support posture set; extracting the first contact support posture in the contact support posture set, and performing adaptation judgment between the center of gravity and the support point in combination with the skeleton weight distribution information to generate a first adaptation judgment result; if the first adaptation judgment result is adaptation, the first contact support posture is added to the multiple automobile skeleton transmission postures.

[0031] Preferably, the structural material characteristics of each position of the target car seat frame are collected by detection equipment (such as material analyzers, etc.), including information such as the material type (such as metal, plastic, etc.), material density, thickness, shape and size of different parts of the frame. For example, the crossbeam part of the frame may be made of high-strength steel, and the seat back material may be made of lightweight plastic. Then, based on the collected structural material characteristics, the material properties of each part and its position in the overall frame are considered to construct the weight distribution information of the car seat frame, that is, to determine the weight proportion and approximate position of the center of gravity at different positions; then, based on the standard frame three-dimensional model, the surface shape, material and outer contour of the conveyor belt are combined to simulate the contact between the car frame and the frame conveyor belt, determine the various postures in which the car seat frame and the conveyor belt can achieve effective contact, and combine them into a contact support posture set, which includes a variety of different contact support postures, each posture They all represent a way in which the skeleton contacts the conveyor belt; then one is selected from the contact support posture set as the analysis object, namely the first contact support posture, and combined with the constructed skeleton weight distribution information, the center of gravity and the support point of the skeleton under the first contact support posture are adapted and judged, that is, whether the center of gravity of the skeleton can be reasonably supported by the support point under this support posture to ensure the stability of the skeleton during the transmission process. If the center of gravity position exceeds the stable support range that the support point can provide, it is judged as unfit, which may cause the skeleton to tilt, shake or even fall during the transmission process; if the first adaptation judgment result is adaptation, it means that under the first contact support posture, the center of gravity of the car skeleton can be reasonably supported by the support point, and the skeleton has good stability during the transmission process. At this time, the first contact support posture is added to multiple car skeleton transmission postures for conveying car seat skeletons in production, thereby improving production efficiency and the accuracy of defect detection.

[0032] Furthermore, the specific configuration of the posture fitting module 30 also includes constructing a center of gravity recognition channel, which trains a convolutional neural network model based on historical contact posture data, historical weight distribution data and corresponding center of gravity identification information; using the center of gravity recognition channel to analyze the contact support posture and the skeleton weight distribution information, and output a first center of gravity distribution position; based on the first center of gravity distribution position, predicting the matching support point, judging whether the first prediction result is compatible with the support point in the first contact support posture, and generating the first adaptation judgment result.

[0033] Preferably, relevant data on the contact posture of the historical automobile seat frame with the conveyor belt during the transmission process, such as the contact angle, position, etc., are obtained, and the contact status of the frame and the conveyor belt at different times is recorded as historical contact posture data. Detailed information on the weight distribution of each part of the historical automobile seat frame is obtained, including the weight of different parts and the overall weight distribution pattern, as historical weight distribution data. The exact position of the center of gravity of the frame corresponding to each set of historical contact posture data and historical weight distribution data is identified to obtain the center of gravity identification information, which together constitute a training data set. Then, a recognition model is constructed based on a convolutional neural network model, and the recognition model is trained using the training data set to learn the relationship between the contact posture data, weight distribution data and the center of gravity position, thereby obtaining a center of gravity recognition channel, which can accurately predict the center of gravity position based on the input contact posture data and weight distribution data; then the first contact support The posture and skeleton weight distribution information are used as input and input into the center of gravity identification channel for analysis and processing, and the first center of gravity distribution position corresponding to the first contact support posture is identified to accurately reflect the specific position of the skeleton center of gravity under the current contact posture and weight distribution; finally, based on the output first center of gravity distribution position, the support point position that can stably support the center of gravity is predicted to obtain a first prediction result, and the first prediction result is compared and judged with the actual support point in the first contact support posture. If the predicted support point position matches the actual support point position, it means that under the current contact posture, the center of gravity of the skeleton can be reasonably supported, and the skeleton has good stability during the transmission process. At this time, the first adaptation judgment result is adaptation; conversely, if the predicted support point position does not match the actual support point position, the first adaptation judgment result is mismatch, thereby accurately screening out a stable automobile skeleton transmission posture.

[0034] Furthermore, the specific configuration of the posture fitting module 30 also includes extracting skeleton edge fulcrums in various directions in the standard skeleton three-dimensional model; performing contact surface fitting in various directions based on the skeleton edge fulcrums in various directions to generate a fitted contact surface set, wherein the fitted contact surface is constructed by N fulcrums, and N is an integer greater than or equal to 3; based on the standard skeleton three-dimensional model, performing three-dimensional model rotation with the contact surfaces in the fitted contact surface set to generate the contact support posture set.

[0035] Preferably, a comprehensive analysis is performed on the standard skeleton three-dimensional model to identify and extract skeleton edge pivots from different directions (such as up and down, left and right, front and back, and other spatial dimensions), that is, points on the skeleton surface that are at the edge position in each direction and have supporting significance. For example, when viewed from above, the protruding and stable points on the top of the skeleton may be extracted as edge pivots; when viewed from the side, some key connection points on the side edges of the skeleton may also become edge pivots; then, contact surface fitting is performed in each direction based on the skeleton edge pivots in each direction, that is, for the skeleton edge pivots extracted from each direction, they are combined and calculated in each direction respectively, and different pivot combinations are tried to construct planes. The plane determined by each group of qualified pivot combinations is a fitting contact surface, and each fitting contact surface is constructed by N (N is an integer greater than or equal to 3) pivots. All contact surfaces obtained by pivot combination fitting in each direction are summarized to form a fitting contact surface set, which includes fitting contact surfaces in multiple directions and positions. Finally, for each contact surface in the fitted contact surface set, the standard skeleton three-dimensional model is rotated based on this contact surface. After each rotation with a different contact surface, the skeleton model is in a new posture, namely the contact support posture, which represents a possible state when the skeleton contacts the support surface such as the conveyor belt; all the contact support postures obtained by rotating with different contact surfaces are combined to generate a contact support posture set, which includes various possibilities of the car skeleton contacting the support surface in different directions and positions, so that a posture suitable for the transmission process and that can ensure the stability of the skeleton can be screened out from this set.

[0036] The coverage area fitting module 40 is used to perform image coverage area fitting for the multiple vehicle skeleton transmission postures based on the camera module distribution characteristics, and construct a target transmission posture combination and a corresponding target skeleton coverage area combination that meets the skeleton coverage requirements.

[0037] Preferably, image coverage areas are fitted for multiple car skeleton transmission postures based on the distribution characteristics of camera modules. When the car skeleton has many curved surfaces, recessed parts, hidden parts or staggered support frames, there will still be parts that cannot be clearly photographed under the multi-view combination. Specifically, the shooting coverage of the car skeleton in different transmission postures by each camera module is analyzed, including simulating which parts of the car skeleton can be photographed by each camera, and the specific positions and ranges of these shooting areas on the car skeleton. Then, the shooting coverage areas of all camera modules are integrated, and at the same time, the shooting range, viewing angle and posture changes of each camera are combined to obtain the car skeleton in the transmission posture. the overall image coverage area; then, from multiple car frame transmission postures, the postures that can meet the skeleton coverage requirements are screened out, that is, all key parts of the car frame and areas where defects may exist are ensured to be clearly captured by the camera for effective defect detection, and these transmission postures that meet the requirements are combined to form a target transmission posture combination; corresponding to the target transmission posture combination, for each selected target transmission posture, its corresponding image coverage area is determined, and these image coverage areas can completely cover the relevant parts of the car frame and meet the requirements of defect detection, and then the image coverage areas are combined to form a target skeleton coverage area combination; thereby ensuring that various defects on the car frame can be accurately detected.

[0038] Furthermore, the specific configuration of the coverage area fitting module 40 also includes collecting the camera parameters of the camera module, performing digital twin modeling in combination with the distribution characteristics of the camera module, and constructing a twin camera model; using the twin camera model to perform camera simulation on the multiple car skeleton transmission postures, and determining multiple image coverage areas based on the simulation results; performing combination optimization based on the multiple image coverage areas, determining the optimal combination that covers the entire surface of the car skeleton and has the least number of combinations, and generating the target transmission posture combination and the target skeleton coverage area combination.

[0039] Preferably, the camera parameters of the acquisition camera module, including but not limited to resolution, focal length, aperture, shooting angle, field of view, frame rate, etc., directly affect the quality and coverage of the camera image, among which the resolution determines the clarity of the image, the focal length affects the shooting distance and the size of the object, and the shooting angle and field of view determine the scene area that the camera can capture; the distribution characteristics of the camera module (such as the position and arrangement of the camera, etc.) and the acquired camera parameters are combined for digital modeling to construct a virtual model corresponding to the actual camera module, namely the twin camera model, which is used to accurately simulate the shooting behavior and effect of the actual camera module under different conditions; the twin camera model is used to transmit posture information to multiple car skeletons. The system simulates shooting in different states. According to the parameters and distribution characteristics of the camera module in the twin camera model, as well as the position and angle of the car skeleton in different transmission postures, the area of ​​the car skeleton that each camera can capture in each posture is determined. For example, the shooting angle and range of each camera in the camera module for the car skeleton, as well as the specific parts captured, are simulated in a certain transmission posture. Then, based on the results of the camera simulation, multiple image coverage areas of the car skeleton in different transmission postures are determined. Each image coverage area represents the part of the car skeleton captured by the camera under specific transmission posture and camera shooting conditions, which may be different surfaces, components or areas of the car skeleton, so that its boundaries and range can be accurately determined.

[0040] Preferably, multiple image coverage areas are finally combined and optimized to determine the optimal combination, wherein the optimal combination needs to meet two conditions: first, the combination can cover the complete surface of the car skeleton, ensuring that all parts of the car skeleton can be captured by the camera, thereby meeting the comprehensiveness requirements of defect detection; second, the number of combinations is the least, and under the premise of ensuring the detection effect, the number of required transmission postures is minimized to improve detection efficiency and reduce detection costs; finally, the optimal combination is determined, and the car skeleton transmission posture corresponding to the optimal combination is the target transmission posture combination, and the image coverage areas corresponding to these transmission postures constitute the target skeleton coverage area combination, and the car skeleton is transmitted according to the target transmission posture combination, and the coverage range determined by the target skeleton coverage area combination is used for image acquisition and defect detection, thereby achieving efficient and accurate detection effects.

[0041] Furthermore, the specific configuration of the coverage area fitting module 40 also includes extracting any area from the multiple image coverage areas, determining the missing area of ​​any area relative to the complete surface of the car skeleton; performing missing matching on the missing area in other areas except the any area, and generating a missing matching result; generating a region combination result set with the any area and the missing matching result; and screening the region combination result that covers the complete surface of the car skeleton and has the least number of combinations in the region combination result set as the optimal combination.

[0042] Preferably, an area is randomly selected from multiple image coverage areas and compared with the complete surface of the car frame to determine the part that is not covered by the area relative to the complete surface of the car frame, which is defined as the missing area. For example, the selected image coverage area only captures the top and one side of the car frame, and the bottom, the other side and other parts of the frame that are not captured constitute the missing area; then, with the determined missing area as the target, search and match are performed in all other image coverage areas to find other image coverage areas that can cover the missing area, and record all other image coverage areas that can match the missing area to form a missing matching result, which may include one or more image coverage areas; then, the initially selected image coverage area is compared with the missing area. All image coverage areas recorded in the mismatch results are combined to form multiple area combinations, and then combined into an area combination result set, which includes a variety of different area combination methods; finally, each area combination in the area combination result set is evaluated, including evaluating whether the area combination can cover the complete surface of the car frame, that is, whether all the image coverage areas in the combination are combined to cover all parts of the car frame, and evaluating whether the number of image coverage areas contained in the area combination is the least while meeting the complete coverage requirement; find the area combination that can fully cover the surface of the car frame and minimize the number of combinations as the optimal combination for car frame production defect detection, and ensure detection efficiency, accuracy and comprehensiveness.

[0043] Furthermore, the specific configuration of the coverage area fitting module 40 also includes performing missing matching in other areas except any one area using the missing area, including combined matching of more than one other areas and single matching of one other area.

[0044] Preferably, when searching for a portion of a missing area on the complete surface of the vehicle skeleton using a certain image coverage area relative to the missing area in other image coverage areas, it is possible that a single other area cannot completely cover the missing area, i.e., a combined match of more than one other area is required. For example, assuming that the missing area of ​​the vehicle skeleton is a portion of its side and a small portion of its bottom, one area in the other image coverage areas may only cover a portion of the side, and another area may only cover a small portion of the bottom. To completely cover the missing area, these two (or more) other areas need to be combined so that their combined coverage areas can completely cover the missing area. The overlap between the areas and how to stitch their coverage areas together are also considered to achieve the best coverage effect. Alternatively, there may be another image coverage area that can completely cover the determined missing area, i.e., a single match of one other area. For example, for the missing area of ​​the vehicle skeleton, if the shooting range of another image coverage area completely overlaps with (or can completely contain) the missing area, this other area can be directly used to match the missing area, thereby achieving more comprehensive and flexible coverage of the complete surface of the vehicle skeleton.

[0045] The cross detection module 50 is used to perform cross detection on the target automobile seat frame in multiple postures based on the target transmission posture combination and the target frame coverage area combination, and generate a target defect detection result.

[0046] Preferably, the target automobile seat frame is cross-detected in multiple postures according to the target transmission posture combination and the target skeleton coverage area combination, wherein the multiple postures refer to different postures in the target transmission posture combination. Specifically, the automobile seat frame is detected using different transmission postures, the skeleton is observed from different angles and directions, and the information obtained under different postures is cross-checked and comprehensively analyzed. For example, a defect on one side of the skeleton is detected in one posture, while the relevant features of the defect inside or on the other side of the skeleton are detected in another posture, so as to more accurately judge the nature, location and severity of the defect. Image information under various postures is collected through cross-detection, and possible defects on the automobile seat frame, such as cracks, deformation, poor welds, etc., are identified, and finally the target defect detection results are generated to ensure the accuracy and comprehensiveness of the target automobile seat frame defect detection.

[0047] Furthermore, the specific configuration of the cross-type detection module 50 also includes extracting multiple target postures in the target transmission posture combination and multiple skeleton coverage areas in the target skeleton coverage area combination; collecting the total production volume and sampling inspection indicators of the production batch corresponding to the target automobile seat frame, and calculating the number of sampling inspections of multiple target postures based on the defect occurrence probability of the multiple skeleton coverage areas to generate a first cross-type detection scheme; based on the first cross-type detection scheme, performing image acquisition according to the multiple target postures to generate multiple image acquisition data sets; performing defect detection and marking based on the multiple image acquisition data sets to generate the target defect detection results.

[0048] Preferably, a plurality of different target postures are selected from the target transmission posture combination, representing different placement and motion states of the automobile seat frame during the transmission process, each posture allowing the camera to capture the frame from a different angle. A plurality of frame coverage areas are extracted from the target frame coverage area combination, each frame coverage area being a portion of the surface area of ​​the automobile seat frame captured by the camera under a specific target posture. The total number of production batches corresponding to the target automobile seat frames, i.e., the total number of automobile seat frames produced in the batch, is collected. At the same time, sampling inspection indicators are obtained, which may include a sampling ratio (e.g., requiring a certain percentage of the total number of production batches to be inspected) and the strictness of the inspection. The defect occurrence probabilities of the plurality of frame coverage areas are then analyzed, i.e., the probability of defects occurring in each frame coverage area is assessed based on historical inspection data. Then, based on the total number of production batches, the sampling inspection indicators, and the defect occurrence probabilities of the plurality of frame coverage areas, the number of sampling inspections for each of the plurality of target postures is calculated. For target postures and corresponding frame coverage areas with a higher probability of defect occurrence, the number of sampling inspections is increased to more accurately detect potential defects. For target postures and corresponding frame coverage areas with a lower probability of defect occurrence, the sampling number is appropriately reduced. The sampling inspection numbers and related inspection requirements for the plurality of target postures are then organized into a first cross-type inspection scheme.

[0049] Preferably, according to the first cross-type detection scheme, the target automobile seat frame is transmitted in sequence according to multiple target postures. Under each target posture, a corresponding number of automobile seat frames are selected according to the prescribed sampling detection quantity, and image acquisition is performed on them using a camera module to ensure that clear and accurate automobile seat frame images can be obtained under different postures, forming multiple image acquisition data sets, each data set corresponding to a target posture; then the images in the multiple image acquisition data sets are analyzed, specifically, by comparing with the standard skeleton three-dimensional model, defects in the automobile seat frame image, such as cracks, holes, deformations, etc., are identified, and if a defect is detected, it is marked, and detailed information such as the location, type, size, etc. of the defect is recorded, and finally a target defect detection result is generated, which comprehensively reflects the defect situation of the target automobile seat frame under different target postures.

[0050] Furthermore, the specific configuration of the cross-type detection module 50 also includes determining the preset image acquisition environment light information, and obtaining multiple defect-free target sample images in combination with the multiple target postures; constructing multiple calibrated pixel distribution features of the multiple skeleton coverage areas based on the multiple target sample images; performing pixel distribution difference analysis on the multiple image acquisition data sets with the multiple calibrated pixel distribution features, determining samples with pixel distribution differences greater than or equal to a preset difference threshold, marking defects, and generating the target defect detection results.

[0051] Preferably, preset image acquisition environment light conditions are determined, including light intensity, color (color temperature), direction of the light source, etc. For example, it is stipulated that image acquisition is performed under a white uniform light source of a specific intensity to ensure that each acquired image has similar lighting conditions, reduce image differences caused by light changes, and thus make defect detection more accurate and reliable; according to multiple target postures, multiple automobile seat skeletons that are determined to be free of defects are selected as target samples, and under the preset image acquisition environment light conditions, image acquisition is performed on the target samples according to each target posture to obtain defect-free target sample images under multiple target postures; then, the multiple target sample images are analyzed and processed, and various features of pixels in each skeleton coverage area image, such as pixel brightness distribution, color distribution, texture features, etc., are statistically calculated to construct a calibrated pixel distribution feature of the skeleton coverage area, which represents the pixel distribution of the skeleton coverage area in the absence of defects. For example, statistical quantities such as the mean value and variance of the pixel brightness in a skeleton coverage area image, as well as color hue, saturation, etc., are calculated, and combined to form the calibrated pixel distribution feature of the area, and finally multiple calibrated pixel distribution features of multiple skeleton coverage areas are obtained.

[0052] Preferably, multiple image acquisition data sets (including sampled detected car seat skeleton images) are compared and analyzed with corresponding multiple calibrated pixel distribution features. Specifically, for each image in the image acquisition data set, the difference between the pixel distribution feature of the image and the corresponding calibrated pixel distribution feature is calculated according to its corresponding skeleton coverage area, and then the calculated pixel distribution difference is compared with a preset threshold. If the pixel distribution difference of a sample image is greater than or equal to the preset difference threshold, it means that the pixel distribution of the sample image has a large deviation from the calibrated pixel distribution feature in the defect-free case, and there is a high possibility of defects. Then, the defect is marked and the location of the defect and related information are recorded. Finally, all sample information marked as defective after pixel distribution difference analysis is combined to generate the target defect detection result.

[0053] Furthermore, the specific configuration of the cross-type detection module 50 also includes fitting the camera module position and posture change position on the skeleton conveyor belt based on the multiple target postures to generate a second cross-type detection scheme; based on the second cross-type detection scheme, continuous transformation detection of multiple postures of the same target automobile seat skeleton is performed to generate corresponding detection images, and defect detection is performed using the multiple calibrated pixel distribution features.

[0054] Preferably, multiple camera modules are arranged on a skeleton conveyor belt, and the positions of the camera modules on the conveyor belt are fitted based on multiple target postures, that is, it is determined at what position on the conveyor belt each camera module should be under different target postures in order to best capture the car seat frame under the corresponding posture. As the car seat frame is conveyed, the camera module is subjected to posture transformation by a robot and then detected, including comprehensively considering the conveying speed of the skeleton, the amplitude and method of posture change, etc., to determine which positions on the conveyor belt are most suitable for posture transformation of the camera module. For example, when the skeleton is conveyed to a specific position, the robot adjusts the camera module from one shooting angle to another to adapt to the different postures of the skeleton, and then determines the specific position on the conveyor belt for posture transformation, that is, the posture transformation position. Finally, the camera module position and the posture transformation position are combined to generate a second cross-type detection scheme, which specifies in detail the setting of the camera module at different positions on the skeleton conveyor belt, the timing and method of posture transformation.

[0055] Preferably, according to the second cross-type detection scheme, the same target automobile seat frame is subjected to continuous transformation detection of multiple postures during the transmission process, that is, according to the position and timing set in the scheme, the camera module is transformed in posture by the robot, so that the camera can capture images of the skeleton in different postures from different angles and positions. For example, when the skeleton is transmitted to the first posture transformation position, the robot adjusts the posture of the camera module and captures the image of the skeleton in the first target posture; then when the skeleton is transmitted to the next position, the posture of the camera module is changed again to capture the image of the skeleton in the second target posture, and so on, to achieve continuous detection of multiple postures of the same skeleton and generate corresponding detection images; and then use multiple calibrated pixel distribution features of multiple skeleton coverage areas to analyze the generated detection image, that is, for each detection image, according to its corresponding skeleton coverage area, the pixel distribution characteristics of the image are compared with the corresponding calibrated pixel distribution characteristics. If the pixel distribution difference is large, it is judged that the skeleton area may have defects, thereby realizing defect detection of the automobile seat frame.

[0056] Although the present application makes various references to certain modules in the system according to the embodiments of the present application, any number of different modules may be used and run on the user terminal and / or server, and the various units and modules included are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other and are not used to limit the scope of protection of the present invention.

[0057] The above specific embodiments do not constitute a limitation on the scope of protection of this application. Those skilled in the art should understand that various modifications, combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application shall be included within the scope of protection of this application.

Claims

1. The automatic detection system for automobile frame production defects based on visual recognition is characterized by: include: A 3D modeling module is used to perform 3D modeling on the target car seat frame to be tested and generate a standard frame 3D model; A camera distribution analysis module is used to determine the distribution characteristics of the camera modules in the defect detection platform relative to the skeleton conveyor belt; A posture fitting module, configured to perform skeleton transmission posture fitting based on the standard skeleton three-dimensional model, and construct multiple vehicle skeleton transmission postures that meet preset stability requirements; A coverage area fitting module is used to perform image coverage area fitting for the multiple vehicle skeleton transmission postures based on the distribution characteristics of the camera modules, and to construct a target transmission posture combination and a corresponding target skeleton coverage area combination that meets the skeleton coverage requirements; A cross detection module is used to perform cross detection on the target automobile seat frame in multiple postures based on the target transmission posture combination and the target frame coverage area combination, and generate a target defect detection result.

2. The automatic detection system for automobile frame production defects based on visual recognition according to claim 1 is characterized in that: The steps performed by the posture fitting module include: Collecting structural material characteristics of various positions of the target car seat frame to construct frame weight distribution information; Performing contact surface fitting with the skeleton conveyor belt based on the standard skeleton three-dimensional model to determine a contact support posture set; Extracting a first contact support posture from the contact support posture set, performing adaptation judgment on the center of gravity and the support point in combination with the skeleton weight distribution information, and generating a first adaptation judgment result; If the first adaptation judgment result is adaptation, the first contact support posture is added to the multiple automobile frame transmission postures.

3. The automatic detection system for automobile frame production defects based on visual recognition according to claim 2 is characterized in that: The steps performed by the posture fitting module include: Constructing a center of gravity recognition channel, wherein the center of gravity recognition channel trains a convolutional neural network model based on historical contact posture data, historical weight distribution data, and corresponding center of gravity identification information; Analyzing the contact support posture and the skeleton weight distribution information using the center of gravity identification channel to output a first center of gravity distribution position; A matching support point is predicted based on the first center of gravity distribution position, and it is determined whether the first prediction result is compatible with the support point in the first contact support posture, thereby generating the first adaptation determination result.

4. The automatic detection system for automobile frame production defects based on visual recognition according to claim 2 is characterized in that: The steps performed by the posture fitting module also include: Extracting skeleton edge support points in various directions from the standard skeleton three-dimensional model; Perform contact surface fitting in each direction based on the skeleton edge support points in each direction to generate a set of fitted contact surfaces, wherein the fitted contact surface is constructed by N support points, where N is an integer greater than or equal to 3; Based on the standard skeleton three-dimensional model, the contact surfaces in the fitting contact surface set are respectively used to perform three-dimensional model rotation to generate the contact support posture set.

5. The automatic detection system for automobile frame production defects based on visual recognition according to claim 1 is characterized in that: The steps performed by the coverage area fitting module include: Collecting the camera parameters of the camera module, performing digital twin modeling based on the distribution characteristics of the camera module, and constructing a twin camera model; Performing camera simulation on the transmission postures of the multiple vehicle skeletons using the twin camera model, and determining multiple image coverage areas according to the simulation results; Based on the coverage areas of the multiple images, a combination optimization is performed to determine the optimal combination that covers the complete surface of the car skeleton and has the least number of combinations, and the target transmission posture combination and the target skeleton coverage area combination are generated.

6. The automatic detection system for automobile frame production defects based on visual recognition according to claim 5 is characterized in that: The steps performed by the coverage area fitting module include: Extracting any area from the multiple image coverage areas, and determining a missing area of ​​any area relative to a complete surface of the vehicle skeleton; Performing missing matching on the missing region in other regions except any one of the regions to generate a missing matching result; generating a region combination result set using any one of the regions and the missing matching result; The regional combination result that covers the complete surface of the car frame and has the least number of combinations is selected from the regional combination result set as the optimal combination.

7. The automatic detection system for automobile frame production defects based on visual recognition according to claim 6 is characterized in that: The steps performed by the coverage area fitting module include: performing missing matching on the missing area in other areas except the any one area, including combined matching of more than one other areas and single matching of one other area.

8. The automatic detection system for automobile frame production defects based on visual recognition according to claim 1 is characterized in that: The steps performed by the cross detection module include: Extracting a plurality of target postures from the target transmission posture combination and a plurality of skeleton coverage areas from the target skeleton coverage area combination; Collecting the total number of production batches and sampling inspection indicators corresponding to the target automobile seat frames, calculating the number of sampling inspections for multiple target postures based on the probability of defects in the areas covered by the multiple frames, and generating a first cross-type inspection plan; Based on the first cross detection scheme, performing image acquisition according to the multiple target postures to generate multiple image acquisition data sets; Defect detection and marking are performed based on the multiple image acquisition data sets to generate the target defect detection result.

9. The automatic detection system for automobile frame production defects based on visual recognition according to claim 8, characterized in that: The steps performed by the cross detection module include: Determining preset image acquisition ambient light information, and acquiring multiple defect-free target sample images in combination with the multiple target postures; constructing a plurality of calibrated pixel distribution features of the plurality of skeleton coverage areas based on the plurality of target sample images; The plurality of image acquisition data sets are subjected to pixel distribution difference analysis using the plurality of calibration pixel distribution features, samples having pixel distribution differences greater than or equal to a preset difference threshold are determined, defect marking is performed, and the target defect detection result is generated.

10. The automatic detection system for automobile frame production defects based on visual recognition according to claim 9, characterized in that: The steps performed by the cross detection module also include: Fitting the camera module position and the posture transformation position on the skeleton conveyor belt based on the multiple target postures to generate a second cross detection scheme; Based on the second cross detection scheme, the same target automobile seat frame is subjected to continuous transformation detection in multiple postures, and corresponding detection images are generated, and defect detection is performed using the multiple calibration pixel distribution features.

Images