A vehicle-associated component detection method, device, equipment and readable medium

Abstract

The application provides a kind of detection method, device, equipment and readable medium of vehicle associated component, applied to vehicle production detection system, for detecting vehicle associated component.Based on the type of vehicle associated component, obtain initial shooting position and at least one position change shooting position;Control the camera to move to the initial shooting position, take a photo, get the initial shooting image;If the initial shooting image does not meet the preset shooting requirement, control the camera to move to any position change shooting position, take a photo, get the position change shooting image.If the position change shooting image meets the preset shooting requirement, the position change shooting image is taken as the target shooting image, and the initial coordinate system and the position change coordinate system of the camera are established.Based on the target shooting image, the initial coordinate system and the position change coordinate system, the spatial position of the feature point on the vehicle associated component is obtained;Based on the spatial position of the feature point, whether the vehicle associated component is qualified is detected, to solve the problem that the detection of vehicle associated component fails due to the fixation of the camera.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of vehicles, in particular to a vehicle associated component detection method, a vehicle associated component detection device, an electronic device and a computer readable medium. BACKGROUND

[0002] In the production process of a vehicle, a vehicle associated component can move on a conveying line. In order to detect whether the vehicle associated component is qualified, a positioning camera is needed to determine the position of the vehicle associated component on the conveying line, and a component detection robot is moved to the surrounding of the vehicle associated component, so that the component detection robot can detect the vehicle associated component. However, the position of the existing positioning camera is usually fixed, which means that once the deviation of the vehicle associated component on the conveying line exceeds the detection range of the positioning camera, the positioning camera on the detection station of the vehicle associated component will not be able to accurately position the vehicle associated component, thereby affecting the accuracy and efficiency of the detection of the vehicle associated component. SUMMARY

[0003] The embodiments of the present application provide a vehicle associated component detection method, device, electronic device and computer readable storage medium, to solve the problem that the vehicle associated component at least includes a material frame and a vehicle body, the visual position of the 3D camera is usually fixed, and some columns or beams of the material frame cannot be captured, so that the deviation value of the material frame cannot be accurately determined, and the position of the positioning camera is usually fixed, once the deviation of the vehicle body on the conveying line exceeds the detection range of the positioning camera, the positioning camera on the gap detection station of the vehicle body will not be able to accurately position the vehicle body, thereby affecting the accuracy and efficiency of the detection of the vehicle body.

[0004] The embodiments of the present application disclose a vehicle associated component detection method, applied to a vehicle production detection system, the vehicle production detection system is used for detecting vehicle associated components in a preset vehicle production workshop, and the method comprises:

[0005] Based on the type of the vehicle associated component, an initial shooting position and at least one variable position shooting position of the vehicle associated component are obtained;

[0006] The camera in the vehicle production workshop is controlled to move to the initial shooting position and take a picture to obtain an initial shooting image;

[0007] If the initial shooting image does not meet the preset shooting requirement, the camera is controlled to move to any variable position shooting position and take a picture to obtain a variable position shooting image;

[0008] if the initial photograph does not meet the preset photographing requirement, moving the camera to any of the variable-position photographing positions and taking a photograph to obtain a variable-position photograph;

[0009] based on the target photograph, the initial coordinate system, and the variable-position coordinate system, obtaining a spatial position of at least one feature point on the vehicle-associated component; the feature point is a point on the vehicle-associated component that has a visual feature;

[0010] based on the spatial position of the feature point, detecting whether the vehicle-associated component is qualified.

[0011] Optionally, after the step of, if the initial photograph does not meet the preset photographing requirement, moving the camera to any of the variable-position photographing positions and taking a photograph to obtain a variable-position photograph, the method further comprises:

[0012] if the variable-position photograph meets the preset photographing requirement, stopping moving the camera;

[0013] if the variable-position photograph does not meet the preset photographing requirement, repeatedly performing the step of moving the camera to any of the variable-position photographing positions and taking a photograph to obtain a variable-position photograph until a variable-position photograph that meets the preset photographing requirement is obtained.

[0014] Optionally, the step of, based on the target photograph, the initial coordinate system, and the variable-position coordinate system, obtaining a spatial position of at least one feature point on the vehicle-associated component, comprises:

[0015] based on the target photograph, obtaining coordinate information of the feature point on the vehicle-associated component in the variable-position coordinate system;

[0016] based on the coordinate information of the feature point in the variable-position coordinate system, obtaining coordinate information of the feature point in the initial coordinate system.

[0017] Optionally, the step of, based on the spatial position of the feature point, detecting whether the vehicle-associated component is qualified, comprises:

[0018] based on the coordinate information of the feature point in the initial coordinate system, obtaining a position of the vehicle-associated component;

[0019] controlling a component detection device in the vehicle production workshop to move to the vicinity of the position of the vehicle-associated component, and detecting whether the vehicle-associated component is qualified.

[0020] Optionally, the vehicle-related component comprises a material frame; the material frame is located in a storage location of the vehicle production workshop; the material frame comprises at least one clamping groove and at least one limiting structure; the vehicle production detection system is configured to detect whether the clamping groove and the limiting structure of the material frame can enable a part of the vehicle to be placed in the material frame; the step of detecting whether the vehicle-related component is qualified based on the spatial position of the feature point comprises:

[0021] obtaining a storage location coordinate system of the storage location; the storage location coordinate system is a coordinate system with any point of the storage location as the origin;

[0022] obtaining clamping groove position parameters of the at least one clamping groove in the storage location coordinate system based on the coordinate information of the feature point in the initial coordinate system;

[0023] using a preset least square method to fit the clamping groove position parameters to obtain a target fitting model;

[0024] determining predicted position parameters of the clamping groove based on the target fitting model;

[0025] determining a target clamping groove from the at least one clamping groove according to the difference between the clamping groove position parameters and the predicted position parameters;

[0026] establishing a material frame coordinate system based on the clamping groove position parameters of the target clamping groove.

[0027] Optionally, the step of detecting whether the vehicle-related component is qualified based on the spatial position of the feature point comprises:

[0028] assuming that the material frame is qualified and that a preset assumed part is placed in the material frame, obtaining a part coordinate system of the assumed part in the material frame; the part coordinate system is a coordinate system with any point of the assumed part as the origin;

[0029] adjusting the part coordinate system so that the part coordinate system coincides with the material frame coordinate system;

[0030] treating the remaining clamping grooves except the target clamping groove in the at least one clamping groove as to-be-processed clamping grooves;

[0031] obtaining part coordinate parameters of the assumed part in the material frame coordinate system, clamping groove coordinate parameters of the to-be-processed clamping grooves in the material frame coordinate system, and limiting structure coordinate parameters of the limiting structure in the material frame coordinate system;

[0032] judging whether the clamping groove and the limiting structure can enable the assumed part to be placed in the material frame based on the part coordinate parameters, the clamping groove coordinate parameters of the to-be-processed clamping grooves, and the limiting structure coordinate parameters.

[0033] If the card slot and the limiting structure can enable the assumed part to be placed in the frame, it is confirmed that the frame is qualified.

[0034] Optionally, the method comprises:

[0035] If the frame is qualified, based on the coordinate information of the feature points in the initial coordinate system, position information of the frame in the storage location is obtained;

[0036] Based on the position information and the card slot position parameter, a placement position of the part is determined;

[0037] The robot in the vehicle production workshop is controlled to place the part at the placement position.

[0038] The embodiment of the application also discloses a detection device of a vehicle-related component, which is applied to a vehicle production detection system, and the vehicle production detection system is used for detecting a vehicle-related component in a preset vehicle production workshop.

[0039] A shooting position acquisition module is used to acquire an initial shooting position and at least one variable-position shooting position of the vehicle-related component based on the type of the vehicle-related component;

[0040] A first moving module is used to control a camera in the vehicle production workshop to move to the initial shooting position and take a picture to obtain an initial shooting image;

[0041] A second moving module is used to control the camera to move to any variable-position shooting position and take a picture to obtain a variable-position shooting image if the initial shooting image does not meet a preset shooting requirement;

[0042] A coordinate system establishment module is used to take the variable-position shooting image as a target shooting image and establish an initial coordinate system and a variable-position coordinate system of the camera if the variable-position shooting image meets the preset shooting requirement; the initial coordinate system is a coordinate system with the initial shooting position as an origin; and the variable-position coordinate system is a coordinate system with a variable-position shooting position corresponding to the target shooting image as an origin;

[0043] A space position acquisition module is used to acquire a space position of at least one feature point on the vehicle-related component based on the target shooting image, the initial coordinate system and the variable-position coordinate system; the feature point is a point with a visual feature on the vehicle-related component;

[0044] A detection module is used to detect whether the vehicle-related component is qualified based on the space position of the feature point.

[0045] Optionally, the device comprises:

[0046] A stopping moving module is configured to stop moving the camera if the variable-position photographing image meets the preset photographing requirement.

[0047] A repeating module is configured to repeat the steps of controlling the camera to move to any variable-position photographing position and taking a photograph to obtain a variable-position photographing image until a variable-position photographing image meeting the preset photographing requirement is obtained if the variable-position photographing image does not meet the preset photographing requirement.

[0048] Optionally, the spatial position acquisition module comprises:

[0049] A first coordinate information acquisition submodule is configured to acquire coordinate information of the feature point on the vehicle-associated component in the variable-position coordinate system based on the target photographing image.

[0050] A second coordinate information acquisition submodule is configured to acquire coordinate information of the feature point in the initial coordinate system based on the coordinate information of the feature point in the variable-position coordinate system.

[0051] Optionally, the detection module comprises:

[0052] A vehicle body position acquisition submodule is configured to acquire the position of the vehicle-associated component based on the coordinate information of the feature point in the initial coordinate system.

[0053] A moving submodule is configured to control the component detection equipment in the vehicle production workshop to move to the surroundings of the position of the vehicle-associated component to detect whether the vehicle-associated component is qualified.

[0054] Optionally, the vehicle-associated component comprises a material frame; the material frame is located in a storage location of the vehicle production workshop; the material frame comprises at least one clamping groove and at least one limiting structure; the vehicle production detection system is configured to detect whether the clamping groove and the limiting structure of the material frame can enable a part of the vehicle to be placed in the material frame; and the detection module comprises:

[0055] A storage location coordinate system acquisition submodule is configured to acquire a storage location coordinate system of the storage location; the storage location coordinate system is a coordinate system taking any point of the storage location as an origin.

[0056] A clamping groove position parameter acquisition submodule is configured to acquire clamping groove position parameters of the at least one clamping groove in the storage location coordinate system based on the coordinate information of the feature point in the initial coordinate system.

[0057] A fitting submodule is configured to perform fitting on the clamping groove position parameters by using a preset least square method to obtain a target fitting model.

[0058] a predicted position parameter determination sub-module, configured to determine a predicted position parameter of the card slot based on the target fitting model;

[0059] a target card slot determination sub-module, configured to determine a target card slot from the at least one card slot according to a difference between the card slot position parameter and the predicted position parameter;

[0060] a frame coordinate system establishment sub-module, configured to establish a frame coordinate system based on the card slot position parameter of the target card slot.

[0061] Optionally, the detection module comprises:

[0062] a part coordinate system acquisition sub-module, configured to acquire a part coordinate system of a hypothetical part in the frame under the assumption that the frame is qualified and the hypothetical part is placed in the frame; the part coordinate system is a coordinate system with any point of the hypothetical part as an origin;

[0063] a part coordinate system adjustment sub-module, configured to adjust the part coordinate system so that the part coordinate system coincides with the frame coordinate system;

[0064] a to-be-processed card slot serving sub-module, configured to serve as a to-be-processed card slot the remaining card slots in the at least one card slot except the target card slot;

[0065] a part coordinate parameter acquisition sub-module, configured to acquire part coordinate parameters of the hypothetical part in the frame coordinate system, card slot coordinate parameters of the to-be-processed card slot in the frame coordinate system, and limiting structure coordinate parameters of the limiting structure in the frame coordinate system;

[0066] a judgment sub-module, configured to judge whether the card slot and the limiting structure can enable the hypothetical part to be placed in the frame based on the part coordinate parameters, the card slot coordinate parameters of the to-be-processed card slot, and the limiting structure coordinate parameters;

[0067] a qualification confirmation sub-module, configured to confirm that the frame is qualified if the card slot and the limiting structure can enable the hypothetical part to be placed in the frame.

[0068] Optionally, the device comprises:

[0069] a position information acquisition module, configured to acquire position information of the frame in the storage location based on coordinate information of the feature points in the initial coordinate system if the frame is qualified;

[0070] a placement position determination module, configured to determine a placement position of the part based on the position information and the card slot position parameter;

[0071] a placement module for controlling a robot in the vehicle production plant to place the part at the placement location.

[0072] The embodiments of the present application also disclose an electronic device, comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory complete mutual communication through the communication bus;

[0073] The memory is used for storing a computer program.

[0074] The processor is used for executing the program stored on the memory, and the method as described in the embodiments of the present application is realized.

[0075] The embodiments of the present application also disclose one or more computer readable media having instructions stored thereon, which, when executed by one or more processors, cause the processors to perform the method as described in the embodiments of the present application.

[0076] The embodiments of the present application have the following advantages:

[0077] In an embodiment of the present invention, a vehicle production inspection system is used to inspect vehicle-related components in a pre-defined vehicle production workshop. Based on the type of the vehicle-related component, the vehicle production inspection system obtains an initial shooting position and at least one shifted shooting position for the vehicle-related component. The system controls a camera in the vehicle production workshop to move to the initial shooting position and take a picture, thereby obtaining an initial captured image. If the initial captured image does not meet the preset shooting requirements, the system controls the camera to move to any shifted shooting position and take a picture, thereby obtaining a shifted captured image. If the shifted captured image meets the preset shooting requirements, the shifted captured image is used as the target captured image, and an initial coordinate system and a shifted coordinate system are established for the camera. The initial coordinate system is a coordinate system with the initial shooting position as its origin, while the shifted coordinate system is a coordinate system with the shifted shooting position corresponding to the target captured image as its origin. Based on the target captured image, the initial coordinate system, and the shifted coordinate system, the vehicle production inspection system obtains the spatial position of at least one feature point on the vehicle-related component. A feature point is a point on the vehicle-related component that has visual characteristics. Based on the spatial position of the feature point, the vehicle production inspection system can determine whether the vehicle-related component is qualified. In an embodiment of the present invention, the camera in the vehicle production workshop is movable, which solves the problem that the visual position of the 3D camera in the vehicle production workshop is usually fixed, resulting in the inability to capture the columns or beams of certain material frames, and thus the inability to accurately determine the deviation value of the material frame. The position of the positioning camera used to determine the position of the vehicle body or material frame on the conveyor line is usually fixed. Once the deviation of the vehicle body or material frame on the conveyor line exceeds the detection range of the positioning camera, the positioning camera will not be able to accurately locate the vehicle body or material frame, thereby affecting the accuracy and efficiency of the vehicle body or material frame detection. In addition, in an embodiment of the present invention, the spatial position of at least one feature point on the vehicle-related component is obtained through the target captured image of the vehicle-related component, the initial coordinate system of the camera, and the displaced coordinate system, and the spatial position of the vehicle-related component and at least one structure on the vehicle-related component is obtained. BRIEF DESCRIPTION OF THE DRAWINGS

[0078] Figure 1 This is a flowchart of a method for detecting vehicle-related components provided in an embodiment of the present invention;

[0079] Figure 2 is a schematic diagram of a material frame in a vehicle production workshop provided in an embodiment of the present invention;

[0080] Figure 3 is a schematic diagram of a vehicle in a vehicle production workshop provided in an embodiment of the present invention;

[0081] Figure 4 is a schematic diagram of a vehicle body detection system provided in an embodiment of the present invention;

[0082] Figure 5 is a schematic diagram of still another vehicle body detection system provided in an embodiment of the present application;

[0083] Figure 6 is a flowchart of a vehicle body detection method provided in an embodiment of the present application;

[0084] Figure 7 is a flowchart of a method for obtaining a spatial position of a vehicle body based on an initial coordinate system and a displacement coordinate system provided in an embodiment of the present application;

[0085] Figure 8 is a schematic diagram of a triangular geometric relationship provided in an embodiment of the present application;

[0086] Figure 9 is a schematic diagram of a parallax provided in an embodiment of the present application;

[0087] Figure 10 is a schematic diagram of a material frame provided in an embodiment of the present application;

[0088] Figure 11 is a flowchart of a material frame detection system controlling a camera to capture a material frame provided in an embodiment of the present application;

[0089] Figure 12 is a flowchart of a method for determining whether a material frame is qualified provided in an embodiment of the present application;

[0090] Figure 13 is a flowchart of a method for a robot to load a part into a material frame provided in an embodiment of the present application;

[0091] Figure 14 is a structural block diagram of a vehicle-related component detection device provided in an embodiment of the present application;

[0092] Figure 15 is a block diagram of an electronic device provided in an embodiment of the present application;

[0093] Figure 16 is a schematic diagram of a computer-readable medium provided in an embodiment of the present application. DETAILED DESCRIPTION

[0094] In order to make the above objectives, features and advantages of the present application more apparent, further specific embodiments of the present application will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0095] In order to facilitate understanding of the technical solutions and technical effects of the embodiments of the present application, a brief description of the prior art of the present application will be given below.

[0096] In the production process of a vehicle, it is necessary to use a robot to put the parts of the vehicle into a frame to achieve efficient material handling and accurate material positioning, ensuring the smooth operation of the production line. After the parts are assembled into a vehicle, the gap and surface difference of the vehicle body need to be detected to determine whether the vehicle is qualified.

[0097] If the parts of the vehicle are stamping sheet metal parts, in the operation of the frame guiding of the stamping sheet metal part automatic framing production line, the non-guiding technology or visual guiding technology is usually used to ensure the accurate guiding of the frame, and then ensure that the stamping sheet metal part is accurately loaded into the frame.

[0098] The non-guiding technology relies on the robot to move along the preset path and place the stamping sheet metal part into the frame. In order to achieve this purpose, the position of the frame on the storage location or tray must be accurately fixed by the positioning pin of the storage location or tray to ensure that the robot can accurately place the stamping sheet metal part into the frame along the preset path. Since the action path of the robot is fixed, the position of the frame on the storage location or tray must be very accurate, and the accuracy requirement of repeated positioning is ≤±5mm, that is, the error range of each positioning of the frame must be controlled within 5mm. This high accuracy requirement makes the cost of the frame in the automatic framing production line at least 1-2 times that of the ordinary frame, significantly increasing the development cost of the frame.

[0099] In addition, the frame may be worn and deformed during use, which may affect the success rate of automatic framing; and it is difficult to predict the degree of influence of wear and deformation on framing success, and regular repair of the frame also increases maintenance costs. Moreover, the position of each frame on each storage location or tray needs to be adjusted separately, and for a complete vehicle model, the number of adjustments may exceed 1000 times, which is a large amount of work.

[0100] The visual guiding technology captures the image of the frame column by a 3D (three-dimensional) camera and compares it with the standard frame to determine the deviation value of the frame. These deviation values are then transmitted to the robot, which adjusts its action trajectory according to these deviation values to ensure that the stamping sheet metal part can be accurately loaded into the frame. However, the visual guiding technology also has limitations, such as the fixed visual position of the 3D camera, which results in insufficient flexibility and may not be able to capture the column or beam of some frames, thereby failing to accurately determine the deviation value of the frame. In addition, the poor quality and long calculation time of the 3D point cloud data of the 3D camera may not be able to meet the high-speed operation requirements of the production line. At the same time, the deviation limit value for each detection point of the frame needs to be set according to experience, and whether the frame is qualified is determined by the deviation limit value. Since the number of detection points of each frame is large, and each detection point involves angle deviation in multiple directions, such as angle deviation in the X-axis, Y-axis, and Z-axis directions of the three-dimensional coordinate system with the detection point as the origin, this leads to the problem of large data combination and complex setting of the deviation limit value.

[0101] In the detection of the gap and the face difference of the vehicle body, the manual detection, the automatic detection of the fixed vehicle body, and the automatic detection of the moving vehicle body can be adopted. The automatic detection system of the fixed vehicle body is composed of a positioning camera and a detection robot, and the automatic detection system of the moving vehicle body is composed of a positioning camera, a detection robot, and a moving encoder. The positioning camera in the automatic detection system of the fixed vehicle body and the automatic detection system of the moving vehicle body can also be other positioning sensors for determining the position of the vehicle body.

[0102] The steps of the automatic detection of the moving vehicle body include: first, placing the assembled vehicle accurately on the conveying line. Then, the specific position information of the vehicle body on the conveying line is captured by the positioning camera. According to the position information, the gap and face difference detection robot is controlled to move around the vehicle body. Then, the robot will move with the vehicle body and detect the gap and face difference of each part of the vehicle body.

[0103] In the actual production process of the vehicle, the vehicle body of the same vehicle model may deviate to a certain extent when the vehicle moves on the conveying line for the detection of the vehicle body. The deviation range can reach ±40mm (plus or minus 40mm). However, the position of the existing positioning camera is usually fixed, and the detection range of the positioning camera is limited to ±25mm. This means that once the vehicle body deviates beyond the detection range of the positioning camera, the positioning camera on the gap detection station of the vehicle body will not be able to accurately position the vehicle body, thereby affecting the accuracy and efficiency of the vehicle body detection.

[0104] In addition, when the vehicles of different wheelbase models move on the conveying line for the detection of the vehicle body, the wheelbase range of the vehicle is 1500-1900mm. Therefore, the wheelbase range can reach 0-400mm, which is far beyond the detection range of the positioning camera, which causes the positioning camera on the gap detection station of the vehicle body to also fail to accurately position the vehicle body of these vehicle models.

[0105] At the same time, the material frame can also move on the conveying line. The detection of the material frame on the conveying line can also use the positioning camera to determine the position of the material frame on the conveying line, and move the material frame detection robot around the material frame, so that the material frame detection robot can detect the material frame. However, the position of the existing positioning camera is usually fixed, which means that once the deviation of the material frame on the conveying line exceeds the detection range of the positioning camera, the positioning camera on the detection station of the material frame will not be able to accurately position the material frame, thereby affecting the accuracy and efficiency of the material frame detection.

[0106] Reference Figure 1, a step flow chart of a detection method of a vehicle-related component provided in an embodiment of the present application is shown, which is applied to a vehicle production detection system for detecting vehicle-related components in a preset vehicle production workshop, and can specifically include the following steps:

[0107] In step 101, based on the type of the vehicle-related component, an initial shooting position and at least one position-changing shooting position of the vehicle-related component are obtained.

[0108] In an embodiment of the present application, for the production process of a vehicle in a vehicle production workshop, a vehicle production detection system can detect vehicle-related components in the vehicle production workshop. The vehicle-related components at least include a frame and / or a body of the vehicle, and the vehicle production detection system can determine whether the frame in the vehicle production workshop is qualified, and whether the face difference and the gap of the body of the vehicle are qualified.

[0109] Specifically, the vehicle production detection system includes a frame detection system. The frame in the vehicle production workshop is used to store parts of the vehicle, such as stamping sheet metal parts, etc. The frame can be placed on a storage location in the vehicle production workshop. The frame usually includes at least one clamping groove and at least one limiting structure, and the frame may be worn and deformed during use. If the deformation of the clamping groove and the limiting structure in the frame is large, the frame cannot store the parts of the vehicle. If the frame cannot store the parts of the vehicle, the frame is unqualified. The frame detection system can detect whether the frame is qualified. In the case that the frame is qualified, the frame detection system can control a robot in the vehicle production workshop to put the parts into the frame.

[0110] In addition, during the detection of the frame, the frame can also be placed on a conveying line and transported. During the transportation of the frame on the conveying line, the frame may be offset. In order to obtain an accurate frame position, a positioning camera can be used to take a picture of the frame to obtain an accurate frame position. Then, the frame detection robot in the vehicle production workshop is controlled to move around the frame position, and during the movement of the frame along with the conveying line, the frame detection robot is controlled to follow the movement of the frame to realize the detection of the frame by the frame detection robot, and determine whether the frame is qualified.

[0111] The vehicle production detection system further comprises a vehicle body detection system. In a vehicle production workshop, a vehicle on which parts have been assembled can be transported on a conveying line. During the transportation of the vehicle on the conveying line, the vehicle can deviate. In order to obtain an accurate vehicle body position, a positioning camera can be used to take a picture of the vehicle body to obtain an accurate vehicle body position. Then, a gap and surface difference detection robot in the vehicle production workshop is controlled to move to the surroundings of the vehicle body position, and during the movement of the vehicle along with the conveying line, the gap and surface difference detection robot is controlled to follow the movement of the vehicle body, so as to realize the detection of the gap and surface difference of the vehicle body by the gap and surface difference detection robot.

[0112] In the embodiments of the present application, the vehicle production detection system can obtain an initial shooting position and at least one position change shooting position of the vehicle associated part based on the type of the vehicle associated part. Specifically, in the two cases that the frame is located in the storage location or follows the conveying line to move, the frame detection system can obtain the initial shooting position and at least one position change shooting position of the frame based on the type of the frame. The vehicle body detection system can obtain an initial shooting position and at least one position change shooting position of the vehicle based on the type of the vehicle.

[0113] Reference is made to Figure 2 , a schematic diagram of a frame in a vehicle production workshop is shown. The frame is located in a storage location in the vehicle production workshop, and the frame is used to place parts of a vehicle. A frame detection system is used to detect whether the parts of the vehicle can be placed in the frame. An upper part of the storage location is arranged with a track, and a 3D camera is arranged on the track. The 3D camera can move on the track to meet the shooting needs of different types of frames. For different types of frames, the frame detection system can obtain an initial shooting position and at least one position change shooting position of the frame based on the type of the frame.

[0114] Reference is made to Figure 3 , a schematic diagram of a vehicle in a vehicle production workshop is shown. The vehicle in the vehicle production workshop moves on a conveying line according to the direction of movement of the vehicle shown in Figure 3 . At least one position change device is arranged around the conveying line, and a positioning camera is arranged on the position change device. For different types of vehicles, the vehicle body detection system can obtain an initial shooting position and at least one position change shooting position of the vehicle based on the type of the vehicle. The vehicle body detection system can also take the initial shooting position of the positioning camera as an origin to establish an initial coordinate system of the positioning camera. Figure 3 The directions of the longitudinal axis (Y axis) and the height axis (Z axis) of the initial coordinate system are shown in Figure 3The direction of the vehicle shown can also be the direction of the frame. At least one displacement device can also be provided around the conveying line of the frame, and a positioning camera is arranged on the displacement device. For different types of frames, the frame detection system can obtain the initial shooting position and at least one displacement shooting position of the frame based on the type of the frame.

[0115] In step 102, the camera in the vehicle production workshop is controlled to move to the initial shooting position and take a picture to obtain an initial shooting image.

[0116] In the embodiment of the present application, for the frame located on the storage site, the frame detection system can control the 3D camera on the track to move to the initial shooting position of the frame, take a picture of the frame, and obtain an initial shooting image of the frame.

[0117] For the frame located on the conveying line, the frame detection system can control the positioning camera to move to the initial shooting position of the frame through the displacement device, take a picture of the frame, and obtain an initial shooting image of the frame.

[0118] In the embodiment of the present application, the body detection system can control the positioning camera to move to the initial shooting position of the vehicle through the displacement device, take a picture of the body of the vehicle, and obtain an initial shooting image of the body.

[0119] In step 103, if the initial shooting image does not meet the preset shooting requirement, the camera is controlled to move to any displacement shooting position and take a picture to obtain a displacement shooting image.

[0120] In the embodiment of the present application, for the frame located on the storage site, after the frame detection system obtains the initial shooting image of the frame, if the initial shooting image of the frame does not meet the preset shooting requirement, the frame detection system can control the 3D camera on the track to move to any displacement shooting position of the frame and take a picture to obtain a displacement shooting image of the frame. The initial shooting image of the frame not meeting the preset shooting requirement can refer to the initial shooting image of the frame not including the image of all structures of the frame, or the initial shooting image being unclear, etc.

[0121] For the frame located on the conveying line, after the frame detection system obtains the initial shooting image of the frame, if the initial shooting image of the frame does not meet the preset shooting requirement, the frame detection system can control the positioning camera to move to any displacement shooting position of the frame through the displacement device, take a picture of the frame, and obtain a displacement shooting image of the frame. The initial shooting image of the frame not meeting the preset shooting requirement can refer to the initial shooting image of the frame not including the frame, or the initial shooting image being unclear, etc.

[0122] In the embodiment of the present application, after the vehicle body detection system acquires the initial shooting image of the vehicle body, if the initial shooting image of the vehicle body does not meet the preset shooting requirement, the vehicle body detection system can control the positioning camera to move to any variable-position shooting position of the vehicle through the variable-position device, take a photo of the vehicle body, and obtain a variable-position shooting image of the vehicle body. The initial shooting image of the vehicle body not meeting the preset shooting requirement can refer to the initial shooting image of the vehicle body not including the vehicle body, or the initial shooting image being unclear, etc.

[0123] In step 104, if the variable-position shooting image meets the preset shooting requirement, the variable-position shooting image is taken as a target shooting image, and an initial coordinate system and a variable-position coordinate system of the camera are established; the initial coordinate system is a coordinate system with the initial shooting position as the origin; and the variable-position coordinate system is a coordinate system with the variable-position shooting position corresponding to the target shooting image as the origin.

[0124] In the embodiment of the present application, if the variable-position shooting image meets the preset shooting requirement, the vehicle production detection system can take the variable-position shooting image as a target shooting image. The vehicle production detection system can also establish an initial coordinate system and a variable-position coordinate system of the 3D camera and / or the positioning camera. The initial coordinate system is a coordinate system with the initial shooting position of the 3D camera and / or the positioning camera as the origin, and the variable-position coordinate system is a coordinate system with the variable-position shooting position of the 3D camera and / or the positioning camera when the target shooting image is shot as the origin.

[0125] In step 105, based on the target shooting image, the initial coordinate system, and the variable-position coordinate system, the spatial position of at least one feature point on the vehicle-related component is acquired; the feature point is a point on the vehicle-related component with visual features.

[0126] In the embodiment of the present application, based on the target shooting image, the initial coordinate system, and the variable-position coordinate system, the vehicle production detection system can acquire the spatial position of at least one feature point on the vehicle-related component. The feature point is a point on the vehicle-related component with visual features. Taking the vehicle body on the conveying line as an example, the vehicle body detection system can acquire the coordinate information of at least one feature point on the vehicle body in the variable-position coordinate system of the positioning camera based on the target shooting image of the vehicle body. Then, based on the coordinate information of the feature point in the variable-position coordinate system of the positioning camera, the vehicle body detection system can acquire the coordinate information of the feature point in the initial coordinate system of the positioning camera. The steps of acquiring the spatial position of the feature point on the storage location or the material frame on the conveying line are similar to the steps of acquiring the spatial position of the feature point on the vehicle body on the conveying line, and the present application will not be described here.

[0127] In step 106, based on the spatial position of the feature point, whether the vehicle-related component is qualified is detected.

[0128] In the embodiment of the present application, based on the spatial positions of the at least one feature point on the vehicle-related component, the vehicle production detection system can detect whether the vehicle-related component is qualified.

[0129] In the case that the vehicle-related component is a frame on a storage location, the frame usually includes at least one clamping groove and at least one limiting structure. Based on the spatial positions of the feature points on the frame, the frame detection system can obtain the spatial positions of the clamping groove and the limiting structure, and further determine whether the clamping groove and the limiting structure can make the part be put into the frame. In the case that the clamping groove and the limiting structure can make the part be put into the frame, the frame detection system can confirm that the frame is qualified; in the case that the clamping groove and the limiting structure are severely deformed and cannot make the part be put into the frame, the frame detection system can confirm that the frame is unqualified.

[0130] In the case that the vehicle-related component is a frame on a conveying line, the frame moves along with the conveying line in the vehicle production workshop. Based on the spatial positions of the feature points on the frame, the frame detection system can determine the position of the frame on the conveying line. The frame detection system can control the frame detection robot to move to the surroundings of the frame position, and follow the frame to move during the movement of the frame along with the conveying line, to detect the frame and determine whether the frame is qualified.

[0131] In the case that the vehicle-related component is a vehicle body of a vehicle on a conveying line, the vehicle moves along with the conveying line in the vehicle production workshop. Based on the spatial positions of the feature points on the vehicle body, the vehicle body detection system can determine the position of the vehicle body of the vehicle on the conveying line. The vehicle body detection system can control the gap and surface difference detection robot to move to the surroundings of the vehicle body position, and follow the vehicle body to move during the movement of the vehicle body along with the conveying line, to detect the gap and surface difference of the vehicle body and determine whether the vehicle body is qualified.

[0132] In the embodiment of the present application, the vehicle production detection system is used to detect vehicle-related components in a preset vehicle production workshop. The vehicle production detection system obtains an initial shooting position and at least one variable shooting position of the vehicle-related components based on the type of the vehicle-related components, controls a camera in the vehicle production workshop to move to the initial shooting position and take a picture to obtain an initial shooting image. If the initial shooting image does not meet the preset shooting requirement, the vehicle production detection system controls the camera to move to any variable shooting position and take a picture to obtain a variable shooting image. If the variable shooting image meets the preset shooting requirement, the variable shooting image is taken as a target shooting image, and an initial coordinate system and a variable coordinate system of the camera are established. The initial coordinate system is a coordinate system with the initial shooting position as the origin, and the variable coordinate system is a coordinate system with the variable shooting position corresponding to the target shooting image as the origin. Based on the target shooting image, the initial coordinate system and the variable coordinate system, the vehicle production detection system obtains the spatial position of at least one feature point on the vehicle-related components, and the feature point is a point on the vehicle-related components with visual features. Based on the spatial position of the feature point, the vehicle production detection system can detect whether the vehicle-related components are qualified. In the embodiment of the present application, the camera in the vehicle production workshop is movable, which solves the problem that the visual position of the 3D camera in the vehicle production workshop is usually fixed, which cannot capture the upright column or cross beam of some material frames, so that the deviation value of the material frame cannot be accurately determined, and the position of the positioning camera used to determine the position of the vehicle body or the material frame on the conveying line is usually fixed. Once the offset of the vehicle body or the material frame on the conveying line exceeds the detection range of the positioning camera or the wheel track of the vehicle changes to cause the vehicle body not to be in the detection range of the positioning camera, the positioning camera cannot accurately position the vehicle body or the material frame, thereby affecting the accuracy and efficiency of the vehicle body or the material frame detection. In addition, in the embodiment of the present application, the spatial position of at least one feature point on the vehicle-related components is obtained through the target shooting image of the vehicle-related components, the initial coordinate system and the variable coordinate system of the camera, and the spatial position of the vehicle-related components and at least one structure on the vehicle-related components is obtained.

[0133] Further, in any of the above embodiments, after step 103, comprising:

[0134] Sub-step S11, if the variable shooting image meets the preset shooting requirement, stop moving the camera;

[0135] Sub-step S12, if the variable shooting image does not meet the preset shooting requirement, repeat the step of controlling the camera to move to any variable shooting position and taking a picture to obtain a variable shooting image until a variable shooting image meeting the preset shooting requirement is obtained.

[0136] In the embodiment of the present application, if the initial photographing image of the vehicle-associated component does not satisfy the preset photographing requirement, the camera is controlled to move to any variable-position photographing position and photographing is performed to obtain a variable-position photographing image of the vehicle-associated component. If the variable-position photographing image satisfies the preset photographing requirement, the vehicle production detection system stops moving the camera. If the variable-position photographing image does not satisfy the preset photographing requirement, the vehicle production detection system repeatedly controls the camera to move to any variable-position photographing position and perform photographing to obtain a variable-position photographing image until a variable-position photographing image satisfying the preset photographing requirement is obtained.

[0137] For the vehicle body of the vehicle on the conveying line, the vehicle body detection system can control the positioning camera to move to an initial photographing position of the vehicle through the variable-position device, perform photographing on the vehicle body to obtain an initial photographing image of the vehicle body. If the initial photographing image of the vehicle body does not satisfy the preset photographing requirement, the positioning camera is controlled to move to any variable-position photographing position through the variable-position device and photographing is performed to obtain a variable-position photographing image of the vehicle body. If the variable-position photographing image satisfies the preset photographing requirement, the vehicle body detection system stops moving the positioning camera. If the variable-position photographing image does not satisfy the preset photographing requirement, the vehicle body detection system repeatedly controls the camera to move to any variable-position photographing position and perform photographing to obtain a variable-position photographing image until a variable-position photographing image satisfying the preset photographing requirement is obtained. The steps of obtaining a variable-position photographing image of the material frame on the conveying line satisfying the preset photographing requirement are similar to the steps of obtaining a variable-position photographing image of the vehicle body of the vehicle on the conveying line satisfying the preset photographing requirement, which will not be described herein.

[0138] For the material frame on the storage location, the material frame detection system can control the 3D camera to move to an initial photographing position of the material frame through the track, perform photographing on the material frame to obtain an initial photographing image of the material frame. If the initial photographing image of the material frame does not satisfy the preset photographing requirement, the 3D camera is controlled to move to any variable-position photographing position through the track and photographing is performed to obtain a variable-position photographing image of the material frame. If the variable-position photographing image satisfies the preset photographing requirement, the material frame detection system stops moving the 3D camera. If the variable-position photographing image does not satisfy the preset photographing requirement, the material frame detection system repeatedly controls the camera to move to any variable-position photographing position and perform photographing to obtain a variable-position photographing image until a variable-position photographing image satisfying the preset photographing requirement is obtained.

[0139] In the embodiment of the present application, if the variable-position photographing image does not satisfy the preset photographing requirement, the step of repeatedly controlling the camera to move to any variable-position photographing position and perform photographing to obtain a variable-position photographing image until a variable-position photographing image satisfying the preset photographing requirement is obtained is repeated, which realizes obtaining a variable-position photographing image satisfying the preset photographing requirement and provides support for obtaining the spatial position of the vehicle-associated component and any structure of the vehicle-associated component based on the variable-position photographing image.

[0140] Further, in any of the above embodiments, step 105 comprises:

[0141] Sub-step S21, based on the target photographing image, obtaining coordinate information of the feature point on the vehicle-related component in the displacement coordinate system;

[0142] Sub-step S22, based on the coordinate information of the feature point in the displacement coordinate system, obtaining coordinate information of the feature point in the initial coordinate system.

[0143] In the embodiments of the present application, the vehicle production detection system can obtain the spatial position of at least one feature point on the vehicle-related component based on the target photographing image, the initial coordinate system, and the displacement coordinate system. Specifically, for the material frame on the storage location, the material frame detection system can obtain the coordinate information of at least one feature point on the material frame in the displacement coordinate system of the 3D camera based on the target photographing image of the material frame. Then, based on the coordinate information of the feature point in the displacement coordinate system of the 3D camera, the material frame detection system can obtain the coordinate information of the feature point in the initial coordinate system of the 3D camera.

[0144] For the vehicle body of the vehicle on the conveying line, the vehicle body detection system can obtain the coordinate information of at least one feature point on the vehicle body in the displacement coordinate system of the positioning camera based on the target photographing image of the vehicle body. Then, based on the coordinate information of the feature point in the displacement coordinate system of the positioning camera, the vehicle body detection system can obtain the coordinate information of the feature point in the initial coordinate system of the positioning camera. The steps of obtaining the coordinate information of the feature point on the material frame on the conveying line in the initial coordinate system of the positioning camera and the steps of obtaining the coordinate information of the feature point on the vehicle body of the vehicle on the conveying line in the initial coordinate system of the positioning camera are similar, and the present application will not be described again here.

[0145] In the embodiments of the present application, the vehicle production detection system can obtain the coordinate information of the feature point on the vehicle-related component in the displacement coordinate system based on the target photographing image, and can obtain the coordinate information of the feature point in the initial coordinate system based on the coordinate information of the feature point in the displacement coordinate system, thereby realizing the obtaining of the spatial position of the vehicle body and / or the material frame, and further realizing the detection of the vehicle body and / or the material frame based on the spatial position of the vehicle body and / or the material frame.

[0146] Further, in any of the above embodiments, step 106 comprises:

[0147] Sub-step S31, based on the coordinate information of the feature point in the initial coordinate system, obtaining the position of the vehicle-related component;

[0148] Sub-step S32, controlling the component detection device in the vehicle production workshop to move to the surroundings of the position of the vehicle-related component, and detecting whether the vehicle-related component is qualified.

[0149] In the embodiment of the present application, based on the coordinate information of at least one feature point on the vehicle-related component in the initial coordinate system, the position of the vehicle-related component can be obtained. For the vehicle body or the material frame of the vehicle on the conveying line, the vehicle production detection system can control the component detection device in the vehicle production workshop to move to the surrounding of the position of the vehicle-related component to detect whether the vehicle-related component is qualified.

[0150] If the vehicle-related component is the vehicle body of the vehicle on the conveying line, the component detection device can be a gap and surface difference detection robot, and the vehicle moves along the conveying line in the vehicle production workshop. The vehicle body detection system can control the gap and surface difference detection robot to move to the surrounding of the position of the vehicle body, and in the process of the vehicle body moving along the conveying line, follow the vehicle body to move, detect the gap and surface difference of the vehicle body, and judge whether the vehicle body is qualified.

[0151] If the vehicle-related component is the material frame on the conveying line, the component detection device can be a material frame detection robot, and the material frame moves along the conveying line in the vehicle production workshop. The material frame detection system can control the material frame detection robot to move to the surrounding of the position of the material frame, and in the process of the material frame moving along the conveying line, follow the material frame to move, detect the material frame, and judge whether the material frame is qualified.

[0152] Reference Figure 4 , a schematic diagram of a vehicle body detection system provided in an embodiment of the present application is shown. The vehicle body detection system comprises: 4 robots, 4 sets of gap and surface difference detection systems, 4 positioning cameras, an industrial computer, a programmable logic controller (PLC), and 4 sets of encoders. The robot is connected with the gap and surface difference detection system, and the gap and surface difference detection system and the robot jointly constitute a gap and surface difference detection robot. The positioning camera is also called a 3D camera and a line edge positioning camera. The encoder is in communication connection with the industrial computer.

[0153] The gap and surface difference detection system can detect the gap and surface difference on the vehicle body, that is, the distance difference between the parts on the vehicle body and the difference in surface smoothness. The positioning camera is used to shoot the image of the vehicle body, and based on the image of the vehicle body, the positioning data of the vehicle body is obtained and sent to the gap and surface difference detection robot. Based on the positioning data, the gap and surface difference detection robot can move to the surrounding of the vehicle body.

[0154] The vehicle follows the conveying line to move, and the encoder is used to collect the moving speed of the vehicle body of the vehicle. The industrial computer is used to obtain the moving speed of the vehicle body collected by the encoder, and calculate the positioning result of the moving vehicle body, and send the positioning result to the gap and surface difference detection robot. After the gap and surface difference detection robot moves to the surrounding of the vehicle body, the gap and surface difference detection robot can follow the vehicle body to move based on the positioning result, and detect the gap and surface difference of the vehicle body.

[0155] The gap and flushness detection robot can send the detection result of the vehicle body gap and flushness to the industrial computer. The industrial computer further comprises a screen. The industrial computer can acquire and record the detection result of the vehicle body gap and flushness, and can also display the detection result on the screen.

[0156] The PLC can control the gap and flushness detection robot to start and stop the vehicle body detection. The PLC further comprises vision software. The vision software can also acquire the image captured by the positioning camera, process the image, acquire the position of the vehicle body, and send the position of the vehicle body to the gap and flushness detection robot, so that the gap and flushness detection robot can move around the vehicle body. It should be noted that, Figure 4 The gap and flushness detection system in the embodiment can also be used for detecting the material frame. When the gap and flushness detection system is used for detecting the material frame, the gap and flushness detection system and the robot jointly constitute a material frame detection robot, Figure 4 The vehicle in the embodiment travels in the same direction as the material frame. In the embodiment, Figure 4 When the gap and flushness detection system in the embodiment is used for detecting the material frame, Figure 4 The vehicle body detection system in the embodiment is also a material frame detection system for detecting the material frame on the conveying line.

[0157] Referring to Figure 5 , a schematic diagram of another vehicle body detection system provided in the embodiment is shown. The vehicle body detection system comprises four robots, four sets of gap and flushness detection systems, four positioning cameras, an industrial computer, a programmable logic controller (PLC), four sets of encoders, and a displacement device. The robot is connected with the gap and flushness detection system, and the gap and flushness detection system and the robot jointly constitute a gap and flushness detection robot. The encoder is in communication connection with the industrial computer. The positioning camera is arranged on the displacement device, and the positioning camera can move through the displacement device. The PLC is in communication connection with the displacement device, and the PLC can send the vehicle model information of the vehicle body to the displacement device. It should be noted that, Figure 5 The gap and flushness detection system in the embodiment can also be used for detecting the material frame. When the gap and flushness detection system in the embodiment is used for detecting the material frame, Figure 5 When the gap and flushness detection system in the embodiment is used for detecting the material frame, Figure 5 The vehicle body detection system in the embodiment is also a material frame detection system for detecting the material frame on the conveying line.

[0158] Referring to Figure 6 , a flowchart of a vehicle body detection method provided in the embodiment is shown.

[0159] The first part, the line positioning camera is automatically positioned according to the vehicle type. The line positioning camera is a positioning camera. The MOM (Manufacturing Operations Management) sends the vehicle type data of the vehicle body entering the conveying line to the PLC, and the PLC sends the vehicle type data to the positioning device. The positioning device automatically positions the positioning camera according to the vehicle type data, and moves the positioning camera to an initial shooting position.

[0160] The second part, the vehicle body deviation is determined. When the vehicle follows the conveying line to move, the vehicle can trigger the positioning and shooting trigger sensor. After the positioning and shooting trigger sensor is triggered, the line positioning camera can shoot at the initial shooting position. If the initial shooting image of the line positioning camera meets the preset shooting requirement, the line positioning camera detects successfully, and the initial shooting image of the line positioning camera at the initial shooting position can be taken as a target shooting image. The line positioning camera can output the position of the vehicle body, i.e., the deviation of the vehicle body on the conveying line, according to the target shooting image.

[0161] The third part, the line positioning camera is automatically positioned. Since the detection range of the positioning camera is limited to ±25mm, when the deviation of the vehicle body on the conveying line exceeds the detection range of ±25mm, the image shot by the line positioning camera does not include the vehicle body, i.e., the line positioning camera detects unsuccessfully. The line positioning camera can send a positioning instruction to the PLC, and the PLC can forward the positioning instruction to the positioning device. The positioning device can move the line positioning camera to the second or third or multiple variable shooting positions to shoot the vehicle body until the line positioning camera detects successfully, i.e., the image shot by the line positioning camera meets the preset shooting requirement. The image shot by the line positioning camera at the variable shooting position meeting the preset shooting requirement can be taken as a target shooting image, and the variable shooting position shooting the target shooting image can be taken as a target variable shooting position. It should be noted that since the vehicle body has a certain length, even if the vehicle body follows the conveying line to move, the line positioning camera can still shoot the vehicle body after spending a certain time to position.

[0162] When the line edge positioning camera is moved to the target position by the positioner and the target image captured at the target position meets the preset shooting requirement, the line edge positioning camera can establish an initial coordinate system of the line edge positioning camera and a positioner coordinate system according to the moving direction and distance of the positioner. The origin of the initial coordinate system is the initial shooting position, and the origin of the positioner coordinate system is the target position. According to the target image, the spatial position of the vehicle body in the positioner coordinate system can be obtained, and the spatial position of the vehicle body in the initial coordinate system can also be obtained by conversion. The line edge positioning camera and the vision software in the PLC can obtain the deviation value of the vehicle body on the conveying line, i.e., the position of the vehicle body on the conveying line, according to the spatial position of the vehicle body in the initial coordinate system.

[0163] The fourth part: vehicle body gap and face difference detection. The gap and face difference detection robot can obtain the deviation value of the vehicle body on the conveying line sent by the line edge positioning camera or the vision software in the PLC, and start the gap and face difference detection of the vehicle body after triggering the vehicle body detection. The gap and face difference detection robot can move around the vehicle body to detect the vehicle body. In addition, the industrial computer in the vehicle body detection system can obtain the moving speed of the vehicle body collected by the encoder, calculate the positioning result of the moving vehicle body, and send the positioning result to the gap and face difference detection robot. After the gap and face difference detection robot moves around the vehicle body, the gap and face difference detection robot can follow the vehicle body based on the positioning result to detect the gap and face difference of the vehicle body.

[0164] The fifth part: the gap and face difference detection is completed, and the robot exits. The vision software can obtain the vehicle body detection result of the gap and face difference detection robot, and display the vehicle body detection result on the screen of the industrial computer in real time. After the gap and face difference detection robot detects, the gap and face difference detection robot exits, and the screen can wait to display the next vehicle body detection result. It should be noted that the detection method of the material frame on the conveying line is similar to the detection method of the vehicle body, and the present application will not be repeated here. Figure 6

[0165] Referring to Figure 7 , a flowchart for obtaining the spatial position of the vehicle body based on the initial coordinate system and the positioner coordinate system is shown.

[0166] ​Firstly, the vehicle body detection system can obtain the initial shooting position of the positioning camera corresponding to different vehicle models according to different vehicle models. It should be noted that the vehicle widths of different vehicle models are different. The vehicle body detection system can receive the vehicle model information of the vehicle sent by the MOM, then control the positioning camera to move to the initial shooting position of the positioning camera corresponding to the vehicle model information, and establish an initial coordinate system T0. The origin of the initial coordinate system is the initial shooting position. If the initial shooting image captured by the positioning camera at the initial shooting position meets the preset shooting requirement, the vehicle body detection system can take the initial shooting image as the target shooting image.

[0167] Secondly, if the initial shooting image does not meet the preset shooting requirement, the vehicle body detection system can control the positioning camera to move to any variable position shooting position through the variable position device and perform shooting. The variable position device is also called a servo moving mechanism. The variable position device moves within the range of 15-50mm and -15--50mm relative to the initial shooting position. Referring to the initial coordinate system shown in FIG. 1, when the quality of the initial shooting image captured by the positioning camera at the initial shooting position is poor, the initial shooting image does not meet the preset shooting requirement at this time. If the positioning camera detects that the offset of the vehicle body on the conveying line is greater than 25mm according to the initial shooting image, which leads to poor image quality, the positioning camera moves to a position 15-50mm away from the initial shooting position along the Y-axis direction of the initial coordinate system through the variable position device; if the positioning camera detects that the offset of the vehicle body on the conveying line is less than -25mm according to the initial shooting image, which leads to poor image quality, the positioning camera moves to a position -15--50mm away from the initial shooting position along the Y-axis direction of the initial coordinate system through the variable position device. Figure 3

[0168] If the variable position shooting image captured by the positioning camera at a certain variable position shooting position meets the preset shooting requirement, the variable position shooting position is regarded as the target variable position shooting position, and the variable position shooting image is taken as the target shooting image. According to the initial coordinate system and the moving distance and direction of the variable position device, a variable position coordinate system T1 can be created. The origin of the variable position coordinate system is the target variable position shooting position. The vehicle body detection system can obtain the spatial point position of the feature point on the vehicle body in T1 based on the target shooting image, and also can convert the spatial point position of the feature point on the vehicle body in T1 to the spatial point position of the feature point on the vehicle body in T0.

[0169] In a specific example, the variable position device moves along the X-axis direction, the Y-axis direction and the Z-axis direction of the initial coordinate system. If the moving components of the three coordinate axes of the variable position coordinate system T1 relative to the three coordinate axes of the initial coordinate system T0 are dx=Tx, dy=Ty and dz=Tz respectively, the relationship formula between T1 and T0 is:

[0170] T1=T0*T ​

[0171]

[0172] Wherein, T is the coordinate system conversion matrix from T0 to T1.

[0173] If the spatial point positions of the feature points on the vehicle body in the variable position coordinate system are P1-Pn, the spatial point positions of the feature points in the initial coordinate system can be obtained by converting the spatial point positions of the feature points in the variable position coordinate system.

[0174] The conversion formula between the spatial point position P101 and the spatial point position P1 is:

[0175] P101=T -1 P1

[0176] Wherein, T -1 is the inverse matrix of T. The conversion formula of the remaining spatial point positions is similar to the conversion formula between P101 and P1, which will not be described here.

[0177] Then, the positioning camera calculates the spatial positions of the feature points on the vehicle body in the target shooting image based on the parallax method through the triangular geometric relationship according to the target shooting image shot at the initial shooting position or the target variable position shooting position.

[0178] Referring to Figure 8 , a principle diagram of a triangular geometric relationship provided in the embodiment of the application is shown. Referring to Figure 9 , a parallax principle diagram provided in the embodiment of the application is shown. The vehicle body is surrounded by two positioning cameras, which are a left positioning camera and a right positioning camera. The target shooting image shot by the left positioning camera is a left image, and the target shooting image shot by the right positioning camera is a right image. Figure 8 The triangular measurement principle shown in

[0179] Z=b*(f / x)

[0180] Wherein, Z is the distance from the feature point to the camera imaging plane, b is the baseline distance between the two positioning cameras, i.e. the distance between the two camera center points, f is the focal length of the camera lens, and x is twice the parallax.

[0181] Figure 9 The parallax principle in

[0182] Parallax (Disparity)=X left -X right

[0183]

[0184] Wherein, X leftis the horizontal position of the feature point on the imaging plane of the left positioning camera, X right is the horizontal position of the feature point on the imaging plane of the right positioning camera, P(x c , y c , z c ) is the spatial position of the feature point on the vehicle body, and Y is the vertical position of the feature point on the imaging plane of any positioning camera.

[0185] Finally, the deviation of the vehicle body on the conveying line can be calculated using the spatial position of the feature point on the vehicle body. The vehicle body detection system can include four positioning cameras, and the four positioning cameras can obtain multiple spatial positions of the feature points on the vehicle body, and then obtain multiple vehicle body positions. The vehicle body detection system performs spatial fitting calculation on the multiple vehicle body positions and the reference vehicle body, and obtains the rotation matrix of the current vehicle body relative to the reference vehicle body. The vehicle body detection system performs 3D affine transformation on the rotation matrix of the vehicle body, and obtains the offset value of the current vehicle body relative to the reference vehicle body. It should be noted that the reference vehicle body is an ideal vehicle body position that enables the positioning camera to capture an image that meets the preset shooting requirements at the initial shooting position. When determining the reference vehicle body of the vehicle, the vehicle body detection system can obtain multiple vehicle body positions of the reference vehicle body under the positioning camera, which will be saved and used for spatial fitting calculation of the multiple vehicle body positions and the reference vehicle body.

[0186] In a specific example, the four positioning cameras are distributed on both sides of the vehicle body, and the offset value of the current vehicle body relative to the reference vehicle body is 6 degrees of freedom. It should be noted that the steps of obtaining the spatial position of the material frame on the conveying line or the storage location based on the initial coordinate system and the displacement coordinate system are similar to the steps of obtaining the spatial position of the vehicle body based on the initial coordinate system and the displacement coordinate system as shown in Figure 7 , and the principles of the triangular geometric relationship as shown in Figure 8 and the parallax principle as shown in Figure 9 can also be used in the steps of obtaining the spatial position of the material frame on the conveying line or the storage location based on the initial coordinate system and the displacement coordinate system, which will not be described herein again.

[0187] In the embodiment of the present application, based on the coordinate information of the feature points in the initial coordinate system, the position of the vehicle body is obtained, and the vehicle body detection equipment in the vehicle production workshop is controlled to move to the surrounding of the position of the vehicle body to detect whether the vehicle body is qualified, thereby realizing the detection of the face difference and the gap of the vehicle body.

[0188] In this embodiment of the present invention, the use of a positioning device significantly improves vehicle model compatibility and line compatibility for vehicle body inspection. Because the positioning device allows the positioning camera to be moved to different positions for imaging, it is compatible with the positioning of vehicles with different wheelbases. This solves the problem of the positioning camera being unable to accurately locate the vehicle body due to a change in wheelbase, which could cause the vehicle body to fall outside the positioning camera's detection range. This significantly reduces subsequent hardware investment.

[0189] Furthermore, in any of the above embodiments, the vehicle-related components include a material frame; the material frame is located in a storage location of the vehicle production workshop; the material frame includes at least one slot and at least one limiting structure; the vehicle production detection system is used to detect whether the slot and the limiting structure of the material frame enable the vehicle parts to be placed in the material frame; step 106 includes:

[0190] Sub-step S41, obtaining a storage location coordinate system of the storage location; the storage location coordinate system is a coordinate system with any point of the storage location as the origin;

[0191] Sub-step S42, based on the coordinate information of the feature point in the initial coordinate system, obtaining the slot position parameters of the at least one slot in the storage location coordinate system;

[0192] Sub-step S43, fitting the slot position parameters using a preset least squares method to obtain a target fitting model;

[0193] Sub-step S44, determining the predicted position parameters of the card slot based on the target fitting model;

[0194] Sub-step S45, determining a target card slot from the at least one card slot according to a difference between the card slot position parameter and the predicted position parameter;

[0195] Sub-step S46: establishing a material frame coordinate system based on the slot position parameters of the target slot.

[0196] In an embodiment of the present invention, the vehicle-related components further include a material frame, which is located in a storage location of a vehicle production workshop. The material frame detection system can obtain a storage location coordinate system of the storage location, wherein the storage location coordinate system is a coordinate system with any point in the storage location as the origin.

[0197] Reference Figure 10, shows a schematic diagram of a material frame provided in an embodiment of the present application. The material frame includes a reference plate, at least one clamping slot and at least one limiting structure. The material frame detection system is used to detect whether the clamping slot and the limiting structure of the material frame can enable the vehicle parts to be placed in the material frame. If the clamping slot and the limiting structure of the material frame can enable the vehicle parts to be placed in the material frame, the material frame is qualified; if the clamping slot and the limiting structure of the material frame cannot enable the vehicle parts to be placed in the material frame, the material frame is unqualified.

[0198] Referring to Figure 11 , shows a flowchart of a process in which a material frame detection system controls a camera to take pictures of a material frame. On a production line of vehicle parts, the material frame detection system will automatically select and load corresponding parameter settings according to the type of the produced parts, and control the camera to move to a corresponding initial shooting position, in preparation for shooting.

[0199] After the material frame enters the storage location, the material frame will trigger the light grid. After the material frame triggers the light grid, the material frame detection system sends a shooting signal to the camera, and the camera takes pictures of the reference plate, the clamping slot and the limiting structure of the material frame to obtain an initial shooting image. The limiting structure refers to the limiting structure.

[0200] If the initial shooting image does not meet the preset shooting requirements, the camera is controlled to move to any variable position shooting position and take pictures to obtain a variable position shooting image. If the variable position shooting image meets the preset shooting requirements, the variable position shooting image is taken as a target shooting image, and an initial coordinate system of the camera and a variable position coordinate system are established. The initial coordinate system is a coordinate system with the initial shooting position as the origin, and the variable position coordinate system is a coordinate system with the variable position shooting position corresponding to the target shooting image as the origin. If the variable position shooting image does not meet the preset shooting requirements, the steps of controlling the camera to move to any variable position shooting position and taking pictures to obtain a variable position shooting image are repeated until a variable position shooting image that meets the preset shooting requirements is obtained.

[0201] Based on the target shooting image, the initial coordinate system and the variable position coordinate system, the spatial position of at least one feature point on the material frame can be obtained. Based on the spatial position of at least one feature point on the material frame, the position parameters of the reference plate, the at least one clamping slot and the at least one limiting structure in the storage location coordinate system can be obtained. The position parameters of the clamping slot in the storage location coordinate system can be referred to as clamping slot position parameters. The method of obtaining the spatial position of at least one feature point on the material frame in the storage location is similar to the method of obtaining the spatial position of at least one feature point on the vehicle body, which will not be described here.

[0202] In addition, the material frame detection system can preprocess the point cloud data obtained by shooting through straight-through filtering, departure point filtering and voxel network filtering, and identify the processed point cloud data to obtain the position parameters of the reference plate, the clamping slot and the limiting structure in the storage location coordinate system.

[0203] In the embodiments of the present application, the frame detection system can use a preset least square method to fit the card slot position parameters to obtain a target fitting model. The target fitting model is the best fitting model. Based on the best fitting model, the frame detection system can determine the predicted position parameters of the card slot in the best fitting model. According to the difference between the card slot position parameters and the predicted position parameters, i.e., the residual between the card slot and the best fitting model, the target card slot can be determined from at least one card slot. Specifically, the different card slot position parameters are sorted according to the size of the difference, and in the sorted data set, the three card slots located in the middle position are taken as the target card slot. Finally, based on the card slot position parameters of the target card slot, the frame coordinate system can be established. The origin of the frame coordinate system is any position of the target card slot.

[0204] In the embodiments of the present application, the storage location coordinate system of the storage location is obtained, and the storage location coordinate system is a coordinate system taking any point of the storage location as the origin. Based on the coordinate information of the feature points in the initial coordinate system, the card slot position parameters of at least one card slot in the storage location coordinate system are obtained. A preset least square method is used to fit the card slot position parameters to obtain a target fitting model, and based on the target fitting model, the predicted position parameters of the card slot are determined. According to the difference between the card slot position parameters and the predicted position parameters, the target card slot is determined from at least one card slot, and based on the card slot position parameters of the target card slot, the frame coordinate system is established, which realizes the establishment of the frame coordinate system based on the target card slot and provides support for judging whether the frame is qualified based on the frame coordinate system.

[0205] In the embodiments of the present application, the frame detection system only collects the position data of the card slot, the limiting structure and the reference plate, reduces the amount of data collection, improves the point cloud precision through point cloud data screening and data simplification, and reduces the data processing time, meeting the rapid placement demand of parts in the vehicle production line.

[0206] Further, in any of the above embodiments, step 106 comprises:

[0207] Sub-step S51, assuming that the frame is qualified and a preset assumed part is placed in the frame, obtaining a part coordinate system of the assumed part in the frame; the part coordinate system is a coordinate system taking any point of the assumed part as the origin;

[0208] Sub-step S52, adjusting the part coordinate system so that the part coordinate system coincides with the frame coordinate system;

[0209] Sub-step S53, taking the remaining card slots in the at least one card slot except the target card slot as to-be-processed card slots;

[0210] Sub-step S54, obtaining part coordinate parameters of the assumed part in the frame coordinate system, slot coordinate parameters of the to-be-processed slot in the frame coordinate system, and limiting structure coordinate parameters of the limiting structure in the frame coordinate system.

[0211] Sub-step S55, judging whether the slot and the limiting structure can enable the assumed part to be placed in the frame based on the part coordinate parameters, the slot coordinate parameters of the to-be-processed slot, and the limiting structure coordinate parameters.

[0212] Sub-step S56, if the slot and the limiting structure can enable the assumed part to be placed in the frame, confirming that the frame is qualified.

[0213] In the embodiment of the present application, in the case that the assumed frame is qualified and a preset assumed part has been placed in the frame, a part coordinate system of the assumed part in the frame is obtained. The part coordinate system is a coordinate system taking any point of the assumed part as an origin. Then, the part coordinate system is adjusted so as to coincide with the frame coordinate system. The remaining slots except the target slot in the at least one slot are taken as to-be-processed slots, and based on the coordinate information of the at least one feature point in the initial coordinate system, the part coordinate parameters of the assumed part in the frame coordinate system, the slot coordinate parameters of the to-be-processed slot in the frame coordinate system, and the limiting structure coordinate parameters of the limiting structure in the frame coordinate system are obtained.

[0214] In the embodiment of the present application, based on the part coordinate parameters, the slot coordinate parameters of the to-be-processed slot, and the limiting structure coordinate parameters, it can be judged whether the slot and the limiting structure can enable the assumed part to be placed in the frame. Specifically, based on the part coordinate parameters, the slot coordinate parameters of the to-be-processed slot, and the limiting structure coordinate parameters, it can be judged whether the slot coordinate parameters of the to-be-processed slot, the limiting structure coordinate parameters of the limiting structure coincide with the part coordinate parameters of the assumed part, and whether the assumed part is located within the range of the to-be-processed slot and the limiting structure. If the slot coordinate parameters of the to-be-processed slot, the limiting structure coordinate parameters of the limiting structure do not coincide with the part coordinate parameters of the assumed part, and the assumed part is located within the range of the to-be-processed slot and the limiting structure, the slot and the limiting structure can enable the assumed part to be placed in the frame, and it is confirmed that the frame is qualified. If the slot coordinate parameters of the to-be-processed slot, the limiting structure coordinate parameters of the limiting structure have a coinciding part with the part coordinate parameters of the assumed part, and the assumed part cannot be located within the range of the to-be-processed slot and the limiting structure, the slot and the limiting structure cannot enable the assumed part to be placed in the frame, and it is confirmed that the frame is unqualified.

[0215] Reference Figure 12, shows a flowchart of judging whether the material frame is qualified. First, the material frame detection system can use the least square method to process the card slot position parameters in the storage location coordinate system to obtain the best fitting model. Based on the best fitting model, the material frame detection system can obtain the residual between the card slot and the best fitting model, and sort different card slot position parameters according to the size of the residual. In the sorted data set, the three card slots located in the middle position are taken as the target card slots. Based on the target card slots, the material frame detection system can establish the material frame coordinate system.

[0216] In the case that the material frame is qualified and the preset assumed part has been placed in the material frame, the part coordinate system of the assumed part in the material frame can be established based on the position parameters of the assumed part in the storage location coordinate system. Then, the part coordinate system is fitted to the material frame coordinate system, that is, the part coordinate system is adjusted so that the part coordinate system coincides with the material frame coordinate system. The part coordinate parameters of the assumed part in the material frame coordinate system are obtained, and whether the assumed part is within the range of the card slot is judged based on the part coordinate parameters.

[0217] If the assumed part is not within the range of the card slot, it is determined that the material frame is unqualified, and the material frame is taken offline and repaired. If the assumed part is within the range of the card slot, it is determined that the material frame is qualified, and the robot is controlled to load the part into the material frame.

[0218] In the embodiment of the present application, in the case that the assumed material frame is qualified and the preset assumed part is placed in the material frame, the part coordinate system of the assumed part in the material frame is obtained; the part coordinate system is a coordinate system taking any point of the assumed part as the origin; the part coordinate system is adjusted so that the part coordinate system coincides with the material frame coordinate system; at least one remaining card slot except the target card slot in the card slot is taken as a to-be-processed card slot; the part coordinate parameters of the assumed part in the material frame coordinate system, the card slot coordinate parameters of the to-be-processed card slot in the material frame coordinate system, and the limiting structure coordinate parameters of the limiting structure in the material frame coordinate system are obtained; whether the card slot and the limiting structure can enable the assumed part to be placed in the material frame is judged based on the part coordinate parameters, the card slot coordinate parameters of the to-be-processed card slot, and the limiting structure coordinate parameters; if the card slot and the limiting structure can enable the assumed part to be placed in the material frame, it is confirmed that the material frame is qualified. By adjusting the part coordinate system so that the part coordinate system coincides with the material frame coordinate system, whether the material frame is qualified is judged based on the part coordinate parameters of the assumed part in the material frame coordinate system, the card slot coordinate parameters of the to-be-processed card slot, and the limiting structure coordinate parameters, which eliminates the confirmation process of the limit deviation value of the card slot, the limiting structure and other structures of the material frame, reduces the debugging and optimization time of the material frame, realizes that the artificial can load the frame, the robot can load the frame, greatly reduces the precision requirement of the material frame, reduces the development cost of the material frame, and can repair the unqualified material frame, which greatly reduces the maintenance cost of the material frame.

[0219] Further, in any of the above embodiments, the method comprises:

[0220] In sub-step S61, if the frame is qualified, position information of the frame in the storage location is obtained based on the coordinate information of the feature points in the initial coordinate system.

[0221] In sub-step S62, the placement position of the part is determined based on the position information and the slot position parameter.

[0222] In sub-step S63, the robot in the vehicle production workshop is controlled to place the part at the placement position.

[0223] In the embodiment, if the frame is qualified, the position information of the frame in the storage location can be obtained based on the coordinate information of the feature points of the frame in the initial coordinate system, and then the position information of the frame in the storage location is obtained. Since the frame is placed in the slot, the placement position of the part can be determined based on the position information of the frame in the storage location and the position information of the slot in the frame coordinate system. The frame detection system generates the part placement trajectory according to the position information of the frame in the storage location and the position information of the slot in the frame coordinate system, and then controls the robot to place the part at the placement position.

[0224] Referring to Figure 13 FIG. 1 shows a flowchart of a process of placing a part into a frame by a robot according to an embodiment of the present application. If the frame is qualified, the frame detection system can obtain the frame position parameter of the reference plate of the frame in the storage location coordinate system based on the coordinate information of the feature points of the frame in the initial coordinate system, and determine the position of the frame in the storage location. Since the frame is placed in the slot, the frame detection system can plan the frame loading trajectory, generate the part placement trajectory, and then control the robot to place the part at the placement position based on the position of the frame in the storage location and the position information of the slot in the frame coordinate system. It should be noted that the position information of the slot in the frame coordinate system is the part loading coordinate, and the frame detection system is also called a control system. The reference plate in the frame is used for positioning the frame.

[0225] In the embodiment, if the frame is qualified, the position information of the frame in the storage location is obtained based on the coordinate information of the feature points in the initial coordinate system, the placement position of the part is determined based on the position information and the slot position parameter, and the robot in the vehicle production workshop is controlled to place the part at the placement position, so that the part is successfully loaded into the qualified frame. The frame detection system automatically plans the robot loading trajectory, and each frame only needs to be debugged once, which greatly reduces the debugging workload and debugging time.

[0226] It should be noted that for the method embodiments, the methods can be described as a series of acts combined to achieve the stated purpose, however, those skilled in the art should know that the present application is not limited to the described act sequence, since some steps can be performed in other sequences or at the same time according to the present application. In addition, those skilled in the art should know that the embodiments described in the specification are all preferred embodiments, and the involved acts are not necessarily required by the present application.

[0227] With reference to Figure 14 , a structural block diagram of a detection device for vehicle-related components provided in an embodiment of the present application is shown, which is applied to a vehicle production detection system for detecting vehicle-related components in a preset vehicle production workshop, and can specifically include the following modules:

[0228] A shooting position acquisition module 1401 is configured to acquire an initial shooting position and at least one variable position shooting position of the vehicle-related component based on the type of the vehicle-related component;

[0229] A first moving module 1402 is configured to control a camera in the vehicle production workshop to move to the initial shooting position and take a picture to obtain an initial shooting image;

[0230] A second moving module 1403 is configured to control the camera to move to any variable position shooting position and take a picture to obtain a variable position shooting image if the initial shooting image does not meet a preset shooting requirement;

[0231] A coordinate system establishment module 1404 is configured to take the variable position shooting image as a target shooting image and establish an initial coordinate system and a variable coordinate system of the camera if the variable position shooting image meets the preset shooting requirement; the initial coordinate system is a coordinate system with the initial shooting position as the origin; and the variable coordinate system is a coordinate system with a variable position shooting position corresponding to the target shooting image as the origin;

[0232] A spatial position acquisition module 1405 is configured to acquire a spatial position of at least one feature point on the vehicle-related component based on the target shooting image, the initial coordinate system and the variable coordinate system; the feature point is a point on the vehicle-related component having a visual feature;

[0233] A detection module 1406 is configured to detect whether the vehicle-related component is qualified based on the spatial position of the feature point.

[0234] In an optional embodiment of the present application, the device includes:

[0235] A stopping moving module is configured to stop moving the camera if the position-shifted image meets the preset shooting requirement.

[0236] A repeating module is configured to repeat the steps of controlling the camera to move to any position-shifted shooting position and taking a picture to obtain a position-shifted image until a position-shifted image meeting the preset shooting requirement is obtained if the position-shifted image does not meet the preset shooting requirement.

[0237] In an optional embodiment of the present application, the spatial position acquisition module comprises:

[0238] A first coordinate information acquisition sub-module is configured to acquire coordinate information of the feature point on the vehicle-related component in the position-shifted coordinate system based on the target shooting image.

[0239] A second coordinate information acquisition sub-module is configured to acquire coordinate information of the feature point in the initial coordinate system based on the coordinate information of the feature point in the position-shifted coordinate system.

[0240] In an optional embodiment of the present application, the detection module comprises:

[0241] A vehicle body position acquisition sub-module is configured to acquire the position of the vehicle-related component based on the coordinate information of the feature point in the initial coordinate system.

[0242] A moving sub-module is configured to control the component detection device in the vehicle production workshop to move to the surrounding of the position of the vehicle-related component to detect whether the vehicle-related component is qualified.

[0243] In an optional embodiment of the present application, the vehicle-related component comprises a material frame; the material frame is located in a storage location of the vehicle production workshop; the material frame comprises at least one clamping groove and at least one limiting structure; the vehicle production detection system is configured to detect whether the clamping groove and the limiting structure of the material frame can enable a part of the vehicle to be placed in the material frame; and the detection module comprises:

[0244] A storage location coordinate system acquisition sub-module is configured to acquire a storage location coordinate system of the storage location; the storage location coordinate system is a coordinate system taking any point of the storage location as an origin.

[0245] A clamping groove position parameter acquisition sub-module is configured to acquire clamping groove position parameters of the at least one clamping groove in the storage location coordinate system based on the coordinate information of the feature point in the initial coordinate system.

[0246] A fitting sub-module is configured to fit the clamping groove position parameters by using a preset least square method to obtain a target fitting model.

[0247] A predicted position parameter determination submodule is configured to determine a predicted position parameter of the card slot based on the target fitting model;

[0248] A target card slot determination submodule is configured to determine a target card slot from the at least one card slot according to a difference between the card slot position parameter and the predicted position parameter;

[0249] A frame coordinate system establishment submodule is configured to establish a frame coordinate system based on a card slot position parameter of the target card slot.

[0250] In an optional embodiment of the present application, the detection module comprises:

[0251] A part coordinate system acquisition submodule is configured to acquire a part coordinate system of a hypothetical part in the frame under the assumption that the frame is qualified and the hypothetical part is placed in the frame; the part coordinate system is a coordinate system with any point of the hypothetical part as an origin;

[0252] A part coordinate system adjustment submodule is configured to adjust the part coordinate system so that the part coordinate system coincides with the frame coordinate system;

[0253] A to-be-processed card slot as submodule is configured to take the remaining card slots in the at least one card slot except the target card slot as to-be-processed card slots;

[0254] A part coordinate parameter acquisition submodule is configured to acquire part coordinate parameters of the hypothetical part in the frame coordinate system, card slot coordinate parameters of the to-be-processed card slots in the frame coordinate system, and limiting structure coordinate parameters of the limiting structure in the frame coordinate system;

[0255] A judgment submodule is configured to judge whether the card slots and the limiting structure can enable the hypothetical part to be placed in the frame based on the part coordinate parameters, the card slot coordinate parameters of the to-be-processed card slots, and the limiting structure coordinate parameters;

[0256] A qualification confirmation submodule is configured to confirm that the frame is qualified if the card slots and the limiting structure can enable the hypothetical part to be placed in the frame.

[0257] In an optional embodiment of the present application, the device comprises:

[0258] A position information acquisition module is configured to acquire position information of the frame in the storage location based on coordinate information of the feature points in the initial coordinate system if the frame is qualified;

[0259] A placement position determination module is configured to determine a placement position of the part based on the position information and the card slot position parameter;

[0260] a placing module configured to control a robot in the vehicle production plant to place the part at the placing position.

[0261] For the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant part can be referred to the part of the method embodiment.

[0262] In addition, the embodiment of the present application further provides an electronic device, such as Figure 15 As shown in the figure, the electronic device comprises a processor 1501, a communication interface 1502, a memory 1503 and a communication bus 1504, wherein the processor 1501, the communication interface 1502 and the memory 1503 complete mutual communication through the communication bus 1504,

[0263] The memory 1503 is used for storing a computer program.

[0264] The processor 1501 is used for executing the program stored in the memory 1503, and realizes the following steps:

[0265] Based on the type of the vehicle-related component, an initial shooting position and at least one variable position shooting position of the vehicle-related component are obtained;

[0266] The camera in the vehicle production plant is controlled to move to the initial shooting position and take a picture to obtain an initial shooting image;

[0267] If the initial shooting image does not meet the preset shooting requirement, the camera is controlled to move to any variable position shooting position and take a picture to obtain a variable position shooting image;

[0268] If the variable position shooting image meets the preset shooting requirement, the variable position shooting image is taken as a target shooting image, and an initial coordinate system and a variable position coordinate system of the camera are established; the initial coordinate system is a coordinate system with the initial shooting position as the origin; the variable position coordinate system is a coordinate system with the variable position shooting position corresponding to the target shooting image as the origin;

[0269] Based on the target shooting image, the initial coordinate system and the variable position coordinate system, the spatial position of at least one feature point on the vehicle-related component is obtained; the feature point is a point on the vehicle-related component with visual features;

[0270] Based on the spatial position of the feature point, it is detected whether the vehicle-related component is qualified.

[0271] In an optional embodiment of the present application, after the step of if the initial shooting image does not meet the preset shooting requirement, the camera is controlled to move to any variable position shooting position and take a picture to obtain a variable position shooting image, the following steps are included:

[0272] If the variable position photograph meets the preset photographing requirement, stop moving the camera;

[0273] If the variable position photograph does not meet the preset photographing requirement, repeat the steps of controlling the camera to move to any variable position photographing position and taking a photograph to obtain a variable position photograph until a variable position photograph meeting the preset photographing requirement is obtained.

[0274] In an optional embodiment of the present application, the step of obtaining the spatial position of at least one feature point on the vehicle-related component based on the target photograph, the initial coordinate system and the variable position coordinate system comprises:

[0275] obtaining the coordinate information of the feature point on the vehicle-related component in the variable position coordinate system based on the target photograph;

[0276] obtaining the coordinate information of the feature point in the initial coordinate system based on the coordinate information of the feature point in the variable position coordinate system.

[0277] In an optional embodiment of the present application, the step of detecting whether the vehicle-related component is qualified based on the spatial position of the feature point comprises:

[0278] obtaining the position of the vehicle-related component based on the coordinate information of the feature point in the initial coordinate system;

[0279] controlling the component detection device in the vehicle production workshop to move to the vicinity of the position of the vehicle-related component to detect whether the vehicle-related component is qualified.

[0280] In an optional embodiment of the present application, the vehicle-related component comprises a material frame; the material frame is located in a storage location of the vehicle production workshop; the material frame comprises at least one clamping groove and at least one limiting structure; the vehicle production detection system is used to detect whether the clamping groove and the limiting structure of the material frame can enable the parts of the vehicle to be placed in the material frame; the step of detecting whether the vehicle-related component is qualified based on the spatial position of the feature point comprises:

[0281] obtaining a storage location coordinate system of the storage location; the storage location coordinate system is a coordinate system taking any point of the storage location as an origin;

[0282] obtaining a clamping groove position parameter of the at least one clamping groove in the storage location coordinate system based on the coordinate information of the feature point in the initial coordinate system;

[0283] adopting a preset least square method to fit the clamping groove position parameter to obtain a target fitting model;

[0284] determine a predicted position parameter of the card slot based on the target fitting model;

[0285] determine a target card slot from the at least one card slot according to a difference between the card slot position parameter and the predicted position parameter;

[0286] establish a material frame coordinate system based on a card slot position parameter of the target card slot.

[0287] In an optional embodiment of the present application, the step of detecting whether the vehicle-related component is qualified based on the spatial position of the feature point comprises:

[0288] In the case that the material frame is assumed to be qualified and a preset assumed part is placed in the material frame, obtain a part coordinate system of the assumed part in the material frame; the part coordinate system is a coordinate system with any point of the assumed part as an origin;

[0289] adjust the part coordinate system so that the part coordinate system coincides with the material frame coordinate system;

[0290] regard the remaining card slots in the at least one card slot except the target card slot as to-be-processed card slots;

[0291] obtain a part coordinate parameter of the assumed part in the material frame coordinate system, a card slot coordinate parameter of the to-be-processed card slot in the material frame coordinate system, and a limiting structure coordinate parameter of the limiting structure in the material frame coordinate system;

[0292] judge whether the card slot and the limiting structure can enable the assumed part to be placed in the material frame based on the part coordinate parameter, the card slot coordinate parameter of the to-be-processed card slot, and the limiting structure coordinate parameter;

[0293] if the card slot and the limiting structure can enable the assumed part to be placed in the material frame, confirm that the material frame is qualified.

[0294] In an optional embodiment of the present application, the method comprises:

[0295] if the material frame is qualified, obtain position information of the material frame in the storage location based on coordinate information of the feature point in the initial coordinate system;

[0296] determine a placement position of the part based on the position information and the card slot position parameter;

[0297] control a robot in the vehicle production workshop to place the part at the placement position.

[0298] The communication bus mentioned above can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For the convenience of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or only one type of bus.

[0299] The communication interface is used for communication between the terminal and other devices.

[0300] The memory can include a Random Access Memory (RAM) and can also include a non-volatile memory, such as at least one disk memory. Optionally, the memory can also be at least one storage device located away from the processor.

[0301] The processor mentioned above can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc. It can also be a Digital Signal Processing (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component.

[0302] As shown in FIG. 1, in another embodiment provided by the present application, a terminal 1000 is provided, which comprises a processor 1001, a communication interface 1002, a memory 1003 and a communication bus 1004. Figure 16 As shown in FIG. 1, in another embodiment provided by the present application, a terminal 1000 is provided, which comprises a processor 1001, a communication interface 1002, a memory 1003 and a communication bus 1004.

[0303] In another embodiment provided by the present application, a computer program product containing instructions is provided, which, when running on a computer, causes the computer to execute the detection method of the vehicle-related component mentioned in the above embodiment.

[0304] In the embodiments described above, all or some of the steps can be implemented by software, hardware, firmware or any combination thereof. When implemented in software, all or some of the steps can be implemented in the form of one or more computer programs which are stored in a computer readable storage medium. The computer readable storage medium can be located in a computing device which is in operation. These computer programs (which may

[0305] It is to be noted that, in the present document, the terms such as first and second, etc. are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between such entities or operations. Moreover, the terms "comprising", "including", or any other variant are intended to cover a non-exclusive inclusion, such that processes, methods, articles, or apparatuses that comprise a list of elements are not required to include only those elements in the list, but can include other elements not expressly listed, or also include elements inherent in such processes, methods, articles, or apparatuses. Without more limitations, an element defined by the statement "comprising a..." does not exclude the existence of additional identical elements in the process, method, article, or apparatus that includes the element.

[0306] Each of the embodiments in the present document is described in a related manner, and the same or similar parts among the embodiments can be referred to each other. Each of the embodiments focuses on the difference from other embodiments. In particular, for the system embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the part of the description of the method embodiments.

[0307] The above merely provides the preferred embodiments of the application, and not intended to limit the protection scope of the application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the application shall fall within the protection scope of the application.

Claims

1. A method for detecting vehicle-related components, characterized in that: Applied to a vehicle production inspection system, the vehicle production inspection system is used to inspect vehicle-related components in a preset vehicle production workshop, and the method includes: Based on the type of the vehicle-related component, acquiring an initial shooting position and at least one displaced shooting position of the vehicle-related component; Controlling the camera in the vehicle production workshop to move to the initial shooting position and take a picture to obtain an initial shot image; If the initial captured image does not meet the preset capturing requirements, the camera is controlled to move to any of the displaced capturing positions and take a picture to obtain a displaced captured image; If the displaced captured image meets the preset capturing requirements, the displaced captured image is used as the target captured image, and an initial coordinate system and a displaced coordinate system of the camera are established; the initial coordinate system is a coordinate system with the initial capturing position as the origin; the displaced coordinate system is a coordinate system with the displaced capturing position corresponding to the target captured image as the origin; Based on the target captured image, the initial coordinate system, and the displaced coordinate system, obtaining the spatial position of at least one feature point on the vehicle-related component; the feature point is a point on the vehicle-related component having a visual feature; Based on the spatial positions of the feature points, it is detected whether the vehicle-related components are qualified.

2. The method according to claim 1, characterized in that If the initial captured image does not meet the preset shooting requirements, the camera is controlled to move to any of the displaced shooting positions and take a picture, and after the step of obtaining the displaced captured image, the method further includes: If the displaced image meets the preset shooting requirement, stop moving the camera; If the displaced photographed image does not meet the preset photographing requirements, the steps of controlling the camera to move to any of the displaced photographing positions and taking pictures to obtain the displaced photographed image are repeated until a displaced photographed image that meets the preset photographing requirements is obtained.

3. The method according to claim 1, characterized in that The step of acquiring the spatial position of at least one feature point on the vehicle-related component based on the target captured image, the initial coordinate system, and the displaced coordinate system includes: Based on the target captured image, obtaining coordinate information of the feature point on the vehicle-related component in the displacement coordinate system; Based on the coordinate information of the feature point in the displaced coordinate system, the coordinate information of the feature point in the initial coordinate system is acquired.

4. The method according to claim 3, characterized in that The step of detecting whether the vehicle-related components are qualified based on the spatial positions of the feature points includes: Acquiring the position of the vehicle-related component based on the coordinate information of the feature point in the initial coordinate system; The component inspection equipment in the vehicle production workshop is controlled to move to the vicinity of the position of the vehicle-related component to inspect whether the vehicle-related component is qualified.

5. The method according to claim 3, characterized in that The vehicle-related components include a material frame; the material frame is located in a storage location of the vehicle production workshop; the material frame includes at least one slot and at least one limiting structure; the vehicle production detection system is used to detect whether the slot and the limiting structure of the material frame enable the vehicle parts to be placed in the material frame; The step of detecting whether the vehicle-related components are qualified based on the spatial positions of the feature points includes: Obtaining a storage location coordinate system of the storage location; the storage location coordinate system is a coordinate system with any point of the storage location as the origin; Based on the coordinate information of the feature point in the initial coordinate system, obtaining the slot position parameters of the at least one slot in the storage location coordinate system; Fitting the slot position parameters using a preset least squares method to obtain a target fitting model; Determining predicted position parameters of the card slot based on the target fitting model; determining a target card slot from the at least one card slot according to a difference between the card slot position parameter and the predicted position parameter; A material frame coordinate system is established based on the slot position parameters of the target slot.

6. The method according to claim 5, characterized in that The step of detecting whether the vehicle-related components are qualified based on the spatial positions of the feature points includes: Under the assumption that the material frame is qualified and a preset assumed part is placed in the material frame, a part coordinate system of the assumed part in the material frame is obtained; the part coordinate system is a coordinate system with any point of the assumed part as the origin; Adjusting the part coordinate system so that the part coordinate system coincides with the material frame coordinate system; The remaining card slots of the at least one card slot except the target card slot are used as the card slots to be processed; Obtaining part coordinate parameters of the assumed part in the material frame coordinate system, slot coordinate parameters of the to-be-processed slot in the material frame coordinate system, and position-limiting structure coordinate parameters of the position-limiting structure in the material frame coordinate system; Based on the part coordinate parameters, the slot coordinate parameters of the slot to be processed, and the coordinate parameters of the limiting structure, determining whether the slot and the limiting structure can enable the assumed part to be placed in the material frame; If the slot and the limiting structure enable the assumed part to be placed in the material frame, the material frame is confirmed to be qualified.

7. The method according to claim 6, characterized in that The method comprises: If the material frame is qualified, obtaining the position information of the material frame in the storage location based on the coordinate information of the feature point in the initial coordinate system; Determining a placement position of the component based on the position information and the slot position parameters; Controlling a robot in the vehicle production workshop to place the part at the placement position.

8. A detection device for vehicle-related parts, characterized in that: Applied to a vehicle production inspection system, the vehicle production inspection system is used to inspect vehicle-related components in a preset vehicle production workshop, and the device includes: a shooting position acquisition module, configured to acquire an initial shooting position and at least one displaced shooting position of the vehicle-related component based on the type of the vehicle-related component; A first moving module is used to control the camera in the vehicle production workshop to move to the initial shooting position and take a picture to obtain an initial shooting image; A second moving module is configured to control the camera to move to any of the displaced shooting positions and take a picture to obtain a displaced shooting image if the initial shot image does not meet the preset shooting requirements; a coordinate system establishment module, configured to use the displaced captured image as a target captured image if the displaced captured image meets the preset capturing requirements, and to establish an initial coordinate system and a displaced coordinate system for the camera; the initial coordinate system is a coordinate system with the initial capturing position as its origin; and the displaced coordinate system is a coordinate system with the displaced capturing position corresponding to the target captured image as its origin; a spatial position acquisition module, configured to acquire the spatial position of at least one feature point on the vehicle-related component based on the target captured image, the initial coordinate system, and the displaced coordinate system; the feature point being a point on the vehicle-related component having a visual feature; The detection module is used to detect whether the vehicle-related components are qualified based on the spatial positions of the feature points.

9. An electronic device, characterized in that: comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus; The memory is used to store computer programs; The processor is configured to implement the method according to any one of claims 1 to 7 when executing a program stored in the memory.

10. One or more computer-readable media having instructions stored thereon, which, when executed by one or more processors, cause the processors to perform the method of any one of claims 1-7.

Images