Detection method, device and equipment for vehicle related parts and readable medium
By obtaining and establishing coordinate systems of different shooting positions in the vehicle production detection system, the problem of fixing the visual position of the 3D camera and fixing the position of the position of the position of the position of the vehicle related components is solved, and the accurate detection and efficient detection of the vehicle related components is achieved, reducing the accuracy and efficiency of the vehicle related components. The cost.
Patent Information
- Application Number
- CN202411194266.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-28
- Publication Date
- 2024-11-29
- Estimated Expiration
- 2044-08-28
AI Technical Summary
In the existing vehicle-related component detection system, the visual position of the 3D camera is fixed, and the columns or beams of some material frames cannot be captured, resulting in the inability to accurately determine the deviation value of the material frame. The position of the positioning camera is usually fixed, resulting in the vehicle body. Or when the offset of the material frame on the conveying line exceeds the detection range, it cannot be accurately positioned, which affects the accuracy and efficiency of the detection.
In the vehicle production detection system, the initial shooting position and displacement shooting position of the vehicle-related components are obtained, the camera is controlled to move to different positions for shooting, the initial coordinate system and displacement coordinate system are established, and the characteristic points on the vehicle-related components are obtained. The space position is to check whether it is qualified and ensure accurate positioning and inspection in the vehicle production workshop.
It improves the accuracy and efficiency of vehicle-related components detection, solves the problem of fixed visual position and fixed positioning camera position of 3D cameras, is compatible with the detection of different models and material frames, and reduces hardware investment and maintenance costs.
Smart Images

Figure CN119043658A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of vehicle technology, and in particular to a method for detecting vehicle-related components, a device for detecting vehicle-related components, an electronic device, and a computer-readable medium. Background Art
[0002] During the production process of vehicles, vehicle-related parts can move on the conveyor line. In order to detect whether the vehicle-related parts are qualified, it is necessary to use a positioning camera to determine the position of the vehicle-related parts on the conveyor line, and move the part inspection robot around the vehicle-related parts so that the part inspection robot can inspect the vehicle-related parts. However, the position of existing positioning cameras is usually fixed, which means that once the displacement of the vehicle-related parts on the conveyor line exceeds the detection range of the positioning camera, the positioning camera on the inspection station of the vehicle-related parts will not be able to accurately locate the vehicle-related parts, thereby affecting the accuracy and efficiency of the vehicle-related parts inspection. Summary of the invention
[0003] The embodiments of the present invention provide a method, device, electronic device and computer-readable storage medium for detecting vehicle-related parts, so as to solve the problem that the vehicle-related parts at least include a material frame and a vehicle body, the visual position of a 3D camera is usually fixed, and may not be able to capture some columns or beams of the material frame, so that the deviation value of the material frame cannot be accurately determined, and the position of the positioning camera is usually fixed. Once the deviation of the vehicle body on the conveyor line exceeds the detection range of the positioning camera, the positioning camera on the gap detection station of the vehicle body will not be able to accurately position the vehicle body, thereby affecting the accuracy and efficiency of the vehicle body detection.
[0004] An embodiment of the present invention discloses a method for detecting vehicle-related components, which is applied to a vehicle production detection system. The vehicle production detection system is used to detect vehicle-related components in a preset vehicle production workshop. The method includes:
[0005] Based on the type of the vehicle-related component, acquiring an initial shooting position and at least one displaced shooting position of the vehicle-related component;
[0006] Controlling the camera in the vehicle production workshop to move to the initial shooting position and take a picture to obtain an initial shooting image;
[0007] If the initial captured image does not meet the preset shooting requirements, the camera is controlled to move to any of the displaced shooting positions and take a picture to obtain a displaced captured image;
[0008] If the displaced captured image meets the preset shooting requirements, the displaced captured image is used as the target captured image, and an initial coordinate system and a displaced coordinate system of the camera are established; the initial coordinate system is a coordinate system with the initial shooting position as the origin; the displaced coordinate system is a coordinate system with the displaced shooting position corresponding to the target captured image as the origin;
[0009] Based on the target captured image, the initial coordinate system and the displaced coordinate system, the spatial position of at least one feature point on the vehicle-related component is acquired; the feature point is a point on the vehicle-related component with visual features;
[0010] Based on the spatial position of the feature point, it is detected whether the vehicle-related component is qualified.
[0011] Optionally, if the initial captured image does not meet the preset shooting requirements, the camera is controlled to move to any of the displaced shooting positions and take a picture, and after the step of obtaining the displaced captured image, the method further comprises:
[0012] If the displaced photographed image meets the preset photographing requirement, stopping moving the camera;
[0013] If the displaced image does not meet the preset shooting requirements, the steps of controlling the camera to move to any of the displaced shooting positions and taking pictures to obtain the displaced image are repeated until a displaced image meeting the preset shooting requirements is obtained.
[0014] Optionally, the step of acquiring the spatial position of at least one feature point on the vehicle-related component based on the target captured image, the initial coordinate system and the displaced coordinate system comprises:
[0015] Based on the target captured image, obtaining coordinate information of the feature point on the vehicle-related component in the displacement coordinate system;
[0016] Based on the coordinate information of the feature point in the displaced coordinate system, the coordinate information of the feature point in the initial coordinate system is acquired.
[0017] Optionally, the step of detecting whether the vehicle-related components are qualified based on the spatial position of the feature point includes:
[0018] Based on the coordinate information of the feature point in the initial coordinate system, obtaining the position of the vehicle-related component;
[0019] The component inspection equipment in the vehicle production workshop is controlled to move to the vicinity of the position of the vehicle-related component to inspect whether the vehicle-related component is qualified.
[0020] Optionally, the vehicle-related component includes a material frame; the material frame is located in a storage location of the vehicle production workshop; the material frame includes at least one card slot and at least one limit structure; the vehicle production detection system is used to detect whether the card slot and the limit structure of the material frame can enable the vehicle parts to be placed in the material frame; the step of detecting whether the vehicle-related component is qualified based on the spatial position of the feature point includes:
[0021] Obtaining a storage location coordinate system of the storage location; the storage location coordinate system is a coordinate system with any point of the storage location as the origin;
[0022] Based on the coordinate information of the feature point in the initial coordinate system, obtaining the slot position parameters of the at least one slot in the storage location coordinate system;
[0023] Using a preset least square method to fit the slot position parameters to obtain a target fitting model;
[0024] Determining predicted position parameters of the card slot based on the target fitting model;
[0025] determining a target card slot from the at least one card slot according to a difference between the card slot position parameter and the predicted position parameter;
[0026] A material frame coordinate system is established based on the slot position parameters of the target slot.
[0027] Optionally, the step of detecting whether the vehicle-related components are qualified based on the spatial position of the feature point includes:
[0028] Under the assumption that the material frame is qualified and the preset assumed part is placed in the material frame, a part coordinate system of the assumed part in the material frame is obtained; the part coordinate system is a coordinate system with any point of the assumed part as the origin;
[0029] Adjusting the part coordinate system so that the part coordinate system coincides with the material frame coordinate system;
[0030] Using the remaining card slots of the at least one card slot except the target card slot as the card slots to be processed;
[0031] Acquire part coordinate parameters of the assumed part in the material frame coordinate system, slot coordinate parameters of the to-be-processed slot in the material frame coordinate system, and position-limiting structure coordinate parameters of the position-limiting structure in the material frame coordinate system;
[0032] Based on the part coordinate parameters, the slot coordinate parameters of the slot to be processed and the limit structure coordinate parameters, determining whether the slot and the limit structure can enable the assumed part to be placed in the material frame;
[0033] If the slot and the limiting structure enable the assumed part to be placed in the material frame, the material frame is confirmed to be qualified.
[0034] Optionally, the method comprises:
[0035] If the material frame is qualified, obtaining the position information of the material frame in the storage location based on the coordinate information of the feature point in the initial coordinate system;
[0036] Based on the position information and the slot position parameters, determining the placement position of the part;
[0037] Controlling a robot in the vehicle production workshop to place the part at the placement position.
[0038] The embodiment of the present invention further discloses a detection device for vehicle-related components, which is applied to a vehicle production detection system. The vehicle production detection system is used to detect vehicle-related components in a preset vehicle production workshop. The device includes:
[0039] A shooting position acquisition module, used for acquiring an initial shooting position and at least one displaced shooting position of the vehicle-related component based on the type of the vehicle-related component;
[0040] A first moving module is used to control the camera in the vehicle production workshop to move to the initial shooting position and take pictures to obtain an initial shooting image;
[0041] A second moving module is used to control the camera to move to any of the displaced shooting positions and take pictures to obtain a displaced shooting image if the initial shot image does not meet the preset shooting requirements;
[0042] A coordinate system establishment module, for taking the displaced photographed image as a target photographed image if the displaced photographed image meets the preset photographing requirement, and establishing an initial coordinate system and a displaced coordinate system of the camera; the initial coordinate system is a coordinate system with the initial photographing position as the origin; the displaced coordinate system is a coordinate system with the displaced photographing position corresponding to the target photographed image as the origin;
[0043] A spatial position acquisition module, used to acquire the spatial position of at least one feature point on the vehicle-related component based on the target captured image, the initial coordinate system and the displaced coordinate system; the feature point is a point on the vehicle-related component with visual features;
[0044] The detection module is used to detect whether the vehicle-related components are qualified based on the spatial positions of the feature points.
[0045] Optionally, the device comprises:
[0046] A stop movement module, used for stopping moving the camera if the displaced captured image meets the preset capture requirement;
[0047] A repetition module is used to repeatedly control the camera to move to any of the displaced shooting positions and take pictures to obtain the displaced shooting image if the displaced shooting image does not meet the preset shooting requirements, until a displaced shooting image that meets the preset shooting requirements is obtained.
[0048] Optionally, the spatial position acquisition module includes:
[0049] A first coordinate information acquisition submodule, used for acquiring coordinate information of the feature point on the vehicle-related component in the displacement coordinate system based on the target captured image;
[0050] The second coordinate information acquisition submodule is used to acquire the coordinate information of the feature point in the initial coordinate system based on the coordinate information of the feature point in the displaced coordinate system.
[0051] Optionally, the detection module includes:
[0052] A vehicle body position acquisition submodule, used for acquiring the position of the vehicle-related component based on the coordinate information of the feature point in the initial coordinate system;
[0053] The moving submodule is used to control the component detection equipment in the vehicle production workshop to move to the vicinity of the position of the vehicle-related component to detect whether the vehicle-related component is qualified.
[0054] Optionally, the vehicle-related components include a material frame; the material frame is located in a storage location of the vehicle production workshop; the material frame includes at least one card slot and at least one limit structure; the vehicle production detection system is used to detect whether the card slot and the limit structure of the material frame can enable the parts of the vehicle to be placed in the material frame; the detection module includes:
[0055] A storage location coordinate system acquisition submodule is used to acquire the storage location coordinate system of the storage location; the storage location coordinate system is a coordinate system with any point of the storage location as the origin;
[0056] A slot position parameter acquisition submodule, used for acquiring a slot position parameter of the at least one slot in the storage location coordinate system based on the coordinate information of the feature point in the initial coordinate system;
[0057] A fitting submodule, used to fit the slot position parameters using a preset least squares method to obtain a target fitting model;
[0058] A predicted position parameter determination submodule, used to determine the predicted position parameters of the card slot based on the target fitting model;
[0059] a target card slot determination submodule, configured to determine a target card slot from the at least one card slot according to a difference between the card slot position parameter and the predicted position parameter;
[0060] The material frame coordinate system establishment submodule is used to establish the material frame coordinate system based on the slot position parameters of the target slot.
[0061] Optionally, the detection module includes:
[0062] A part coordinate system acquisition submodule is used to acquire a part coordinate system of the assumed part in the material frame, assuming that the material frame is qualified and the preset assumed part is placed in the material frame; the part coordinate system is a coordinate system with any point of the assumed part as the origin;
[0063] A part coordinate system adjustment submodule, used for adjusting the part coordinate system so that the part coordinate system coincides with the material frame coordinate system;
[0064] The card slot to be processed is used as a submodule, and is used to use the remaining card slots in the at least one card slot except the target card slot as the card slot to be processed;
[0065] A part coordinate parameter acquisition submodule, used to acquire part coordinate parameters of the assumed part in the material frame coordinate system, slot coordinate parameters of the to-be-processed slot in the material frame coordinate system, and position-limiting structure coordinate parameters of the position-limiting structure in the material frame coordinate system;
[0066] A judgment submodule, configured to judge whether the slot and the limiting structure can enable the assumed part to be placed in the material frame based on the part coordinate parameters, the slot coordinate parameters of the slot to be processed, and the limiting structure coordinate parameters;
[0067] The qualified confirmation submodule is used to confirm that the material frame is qualified if the card slot and the limiting structure can allow the assumed part to be placed in the material frame.
[0068] Optionally, the device comprises:
[0069] A position information acquisition module, configured to acquire position information of the material frame in the storage location based on coordinate information of the feature point in the initial coordinate system if the material frame is qualified;
[0070] A placement position determination module, used to determine the placement position of the part based on the position information and the slot position parameters;
[0071] A placement module is used to control the robot in the vehicle production workshop to place the parts at the placement position.
[0072] The embodiment of the present invention further discloses an electronic device, comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus;
[0073] The memory is used to store computer programs;
[0074] The processor is used to implement the method described in the embodiment of the present invention when executing the program stored in the memory.
[0075] The embodiment of the present invention further discloses one or more computer-readable media on which instructions are stored. When executed by one or more processors, the processors are enabled to execute the method described in the embodiment of the present invention.
[0076] The embodiments of the present invention include the following advantages:
[0077] In an embodiment of the present invention, a vehicle production inspection system is used to inspect vehicle-related components in a preset vehicle production workshop. Based on the type of the vehicle-related component, the vehicle production inspection system obtains the initial shooting position and at least one displacement shooting position of the vehicle-related component, controls the camera in the vehicle production workshop to move to the initial shooting position, and takes a picture to obtain an initial shooting image. If the initial shooting image does not meet the preset shooting requirements, the vehicle production inspection system controls the camera to move to any displacement shooting position, and takes a picture to obtain a displacement shooting image. If the displacement shooting image meets the preset shooting requirements, the displacement shooting image is used as the target shooting image, and the initial coordinate system and displacement coordinate system of the camera are established. The initial coordinate system is a coordinate system with the initial shooting position as the origin, and the displacement coordinate system is a coordinate system with the displacement shooting position corresponding to the target shooting image as the origin. Based on the target shooting image, the initial coordinate system and the displacement coordinate system, the vehicle production inspection system obtains the spatial position of at least one feature point on the vehicle-related component, and the feature point is a point with visual features on the vehicle-related component. Based on the spatial position of the feature point, the vehicle production inspection system can detect whether the vehicle-related component is qualified. In an embodiment of the present invention, the camera in the vehicle production workshop is movable, which solves the problem that the visual position of the 3D camera in the vehicle production workshop is usually fixed, resulting in the inability to capture the columns or beams of certain material frames, thereby failing to accurately determine the deviation value of the material frame, and the position of the positioning camera used to determine the position of the vehicle body or material frame on the conveyor line is usually fixed. Once the deviation of the vehicle body or material frame on the conveyor line exceeds the detection range of the positioning camera, the positioning camera will not be able to accurately locate the vehicle body or material frame, thereby affecting the accuracy and efficiency of the vehicle body or material frame detection. In addition, in an embodiment of the present invention, the spatial position of at least one feature point on the vehicle-related component is obtained through the target captured image of the vehicle-related component, the initial coordinate system of the camera, and the displacement coordinate system, and the spatial position of the vehicle-related component and at least one structure on the vehicle-related component is obtained. BRIEF DESCRIPTION OF THE DRAWINGS
[0078] Figure 1 is a flowchart of a method for detecting vehicle-related components provided in an embodiment of the present invention;
[0079] Figure 2 is a schematic diagram of a material frame in a vehicle production workshop provided in an embodiment of the present invention;
[0080] Figure 3 is a schematic diagram of a vehicle in a vehicle production workshop provided in an embodiment of the present invention;
[0081] Figure 4 is a schematic diagram of a vehicle body detection system provided in an embodiment of the present invention;
[0082] Figure 5 is a schematic diagram of another vehicle body detection system provided in an embodiment of the present invention;
[0083] Figure 6 is a flow chart of a vehicle body detection method provided in an embodiment of the present invention;
[0084] Figure 7 is a schematic diagram of a process for obtaining the spatial position of a vehicle body based on an initial coordinate system and a displacement coordinate system provided in an embodiment of the present invention;
[0085] Figure 8 is a principle diagram of a triangular geometric relationship provided in an embodiment of the present invention;
[0086] Fig. 9 is a parallax principle diagram provided in an embodiment of the present invention;
[0087] Fig.10 is a schematic diagram of a material frame provided in an embodiment of the present invention;
[0088] Fig.11 It is a schematic diagram of a process of controlling a camera to shoot a material frame by a material frame detection system provided in an embodiment of the present invention;
[0089] Fig.12 It is a schematic diagram of a process for judging whether a material frame is qualified provided in an embodiment of the present invention;
[0090] Fig.13 is a schematic diagram of a process of loading parts into a material frame by a robot provided in an embodiment of the present invention;
[0091] Fig.14 is a structural block diagram of a detection device for vehicle-related components provided in an embodiment of the present invention;
[0092] Fig.15 is a block diagram of an electronic device provided in an embodiment of the present invention;
[0093] Fig.16 is a schematic diagram of a computer-readable medium provided in an embodiment of the present invention. DETAILED DESCRIPTION
[0094] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.
[0095] To facilitate understanding of the technical solutions and technical effects of the embodiments of the present invention, the prior art of the present invention is briefly described below.
[0096] During the vehicle production process, robots are needed to place vehicle parts into material frames to achieve efficient material handling and accurate material positioning to ensure smooth operation of the production line. After the parts are assembled into a vehicle, the gap and flushness of the vehicle body need to be tested to determine whether the vehicle is qualified.
[0097] If the parts of the vehicle are stamped sheet metal parts, in the material frame guidance operation of the stamped sheet metal automatic framing production line, unguided technology or visual guidance technology is usually used to ensure the precise guidance of the material frame, thereby ensuring that the stamped sheet metal parts are accurately loaded into the material frame.
[0098] Unguided technology relies on the robot to move along a preset path and place the stamped sheet metal parts into the material frame. To achieve this goal, the position of the material frame on the storage location or pallet must be accurately fixed by the positioning pins of the storage location or pallet to ensure that the robot can accurately place the stamped sheet metal parts into the material frame along the preset path. Since the robot's movement path is fixed, the position of the material frame on the storage location or pallet must be very precise, and the accuracy of repeated positioning must be ≤±5mm, that is, the error range of each positioning of the material frame must be controlled within 5 mm. This high-precision requirement makes the cost of the material frame in the automatic framing production line at least 1-2 times that of ordinary material frames, significantly increasing the development cost of the material frame.
[0099] In addition, the material frame may be worn and deformed during use, which will affect the success rate of automatic frame installation. It is difficult to predict the impact of wear and deformation on the success of frame installation, and regular repair of the material frame also increases maintenance costs. Moreover, each material frame needs to be debugged separately at each storage location or pallet position. For a complete model of the material frame, the number of debugging times may exceed 1,000, which is a large workload.
[0100] Vision guidance technology uses a 3D (three-dimensional) camera to capture the column image of the material frame and compare it with the standard material frame to determine the deviation value of the material frame. These deviation values are then transmitted to the robot, which adjusts its movement trajectory according to these deviation values to ensure that the stamped sheet metal parts can be accurately loaded into the material frame. However, vision guidance technology also has limitations, such as the fixed visual position of the 3D camera, which leads to insufficient flexibility and may not be able to capture the columns or beams of some material frames, so that the deviation value of the material frame cannot be accurately determined. In addition, the poor quality and long calculation time of the 3D point cloud data of the 3D camera may be difficult to meet the high-speed operation requirements of the production line. At the same time, it is necessary to set the deviation limit value for each detection point of the material frame based on experience, and judge whether the material frame is qualified by the deviation limit value. Since there are many detection points for each material frame, and each detection point involves angle deviations in multiple directions, such as angle deviations in the three directions of the X-axis, Y-axis, and Z-axis of the three-dimensional coordinate system with the detection point as the origin, this leads to the problem of large data combinations and complex settings of the deviation limit value.
[0101] When inspecting the gap and flushness of the vehicle body, manual inspection, vehicle body fixed automatic inspection, and vehicle body accompanying automatic inspection can be used. The vehicle body fixed automatic inspection system is composed of a positioning camera, an inspection robot, etc., and the vehicle body accompanying automatic inspection system is composed of a positioning camera, an inspection robot, an accompanying encoder, etc. Among them, the positioning camera in the vehicle body fixed automatic inspection system and the vehicle body accompanying automatic inspection system can also be other positioning sensors for determining the position of the vehicle body.
[0102] The steps of the automatic vehicle body inspection include: first, accurately place the assembled vehicle on the conveyor line. Then, use the positioning camera to capture the specific position information of the vehicle body on the conveyor line. Based on this position information, control the gap and flushness inspection robot to move around the vehicle body. Then, the robot will move with the vehicle body and perform gap and flushness inspection on various parts of the vehicle body.
[0103] In the actual production process of vehicles, when vehicles of the same model move on the conveyor line to realize the vehicle body inspection, the vehicle body may have a certain degree of deviation, and the deviation range can reach ±40mm (plus or minus 40 mm). However, the position of the existing positioning camera is usually fixed, and the detection range of the positioning camera is limited to ±25mm. This means that once the body deviation exceeds the detection range of the positioning camera, the positioning camera at the body gap detection station will not be able to accurately locate the body, thereby affecting the accuracy and efficiency of the body inspection.
[0104] In addition, when vehicles with different wheelbases move on the conveyor line to detect the vehicle body, the wheelbase ranges from 1500 to 1900 mm. Therefore, the wheelbase can vary from 0 to 400 mm, far exceeding the detection range of the positioning camera. This results in the positioning camera on the body gap detection station also being unable to accurately locate the body of these models.
[0105] At the same time, the material frame can also move on the conveyor line. For the detection of the material frame on the conveyor line, the positioning camera can also be used to determine the position of the material frame on the conveyor line, and the material frame detection robot is moved to the vicinity of the material frame so that the material frame detection robot can detect the material frame. However, the position of the existing positioning camera is usually fixed, which means that once the deviation of the material frame on the conveyor line exceeds the detection range of the positioning camera, the positioning camera on the detection station of the material frame will not be able to accurately locate the material frame, thereby affecting the accuracy and efficiency of the material frame detection.
[0106] Reference Figure 1, shows a flowchart of a method for detecting vehicle-related components provided in an embodiment of the present invention, which is applied to a vehicle production detection system. The vehicle production detection system is used to detect vehicle-related components in a preset vehicle production workshop, and specifically may include the following steps:
[0107] Step 101, based on the type of the vehicle-related component, obtaining an initial shooting position and at least one displaced shooting position of the vehicle-related component;
[0108] In an embodiment of the present invention, in a vehicle production workshop, the vehicle production detection system can detect vehicle-related components in the vehicle production workshop for the vehicle production process. The vehicle-related components include at least a material frame and / or a vehicle body, and the vehicle production detection system can determine whether the material frame in the vehicle production workshop is qualified, and whether the face difference and gap of the vehicle body are qualified.
[0109] Specifically, the vehicle production inspection system includes a material frame inspection system. The material frame in the vehicle production workshop is used to store vehicle parts, such as stamped sheet metal parts. The material frame can be placed in the storage location of the vehicle production workshop. The material frame usually includes at least one card slot and at least one limiting structure, and the material frame may be worn and deformed during use. If the deformation degree of the card slot and the limiting structure in the material frame is large, it will result in that the vehicle parts cannot be stored in the material frame. If the vehicle parts cannot be stored in the material frame, the material frame is unqualified. The material frame inspection system can detect whether the material frame is qualified. If the material frame is qualified, the material frame inspection system can control the robot in the vehicle production workshop to put the parts into the material frame.
[0110] In addition, during the inspection of the material frame, the material frame can also be placed on the conveyor line for transmission. During the material frame's transmission on the conveyor line, the material frame may be offset. In order to obtain the accurate position of the material frame, the material frame can be photographed using a positioning camera to obtain the accurate position of the material frame. Then, the material frame inspection robot in the vehicle production workshop is controlled to move to the vicinity of the material frame position, and when the material frame moves along the conveyor line, the material frame inspection robot is controlled to follow the material frame to realize the inspection of the material frame by the material frame inspection robot and determine whether the material frame is qualified.
[0111] The vehicle production inspection system also includes a body inspection system. In the vehicle production workshop, the vehicle that has completed the parts assembly can be transferred on the conveyor line. During the process of the vehicle being transferred on the conveyor line, the vehicle may be offset. In order to obtain the accurate body position of the vehicle, the body can be photographed by a positioning camera to obtain the accurate body position. Then, the gap and flushness detection robot in the vehicle production workshop is controlled to move around the body position, and when the vehicle moves along the conveyor line, the gap and flushness detection robot is controlled to move with the body to realize the gap and flushness detection of the body by the gap and flushness detection robot.
[0112] In an embodiment of the present invention, the vehicle production detection system can obtain the initial shooting position and at least one displacement shooting position of the vehicle-related parts based on the type of the vehicle-related parts. Specifically, in both cases where the material frame is located in the storage location or moves along the conveyor line, the material frame detection system can obtain the initial shooting position and at least one displacement shooting position of the material frame based on the type of the material frame. The body detection system can obtain the initial shooting position and at least one displacement shooting position of the vehicle based on the type of the vehicle.
[0113] Reference Figure 2 , shows a schematic diagram of a material frame in a vehicle production workshop provided in an embodiment of the present invention. The material frame is located in a storage location in a vehicle production workshop, and the material frame is used to place vehicle parts. The material frame detection system is used to detect whether the vehicle parts can be placed in the material frame. A track is arranged at the upper part of the storage location, and a 3D camera is arranged on the track. The 3D camera can meet the photography requirements of different types of material frames by moving on the track. For different types of material frames, the material frame detection system can obtain the initial shooting position and at least one displacement shooting position of the material frame based on the type of the material frame.
[0114] Reference Figure 3 , which shows a schematic diagram of a vehicle in a vehicle production workshop provided in an embodiment of the present invention. The vehicles in the vehicle production workshop are on a conveyor line according to Figure 3 The vehicle body detection system moves in the direction of the vehicle shown. At least one displacement device is arranged around the conveyor line, and a positioning camera is arranged on the displacement device. For different types of vehicles, the vehicle body detection system can obtain the initial shooting position and at least one displacement shooting position of the vehicle based on the type of the vehicle. The vehicle body detection system can also use the initial shooting position of the positioning camera as the origin to establish the initial coordinate system of the positioning camera. Figure 3 The directions of the vertical axis (Y axis) and the height axis (Z axis) of the initial coordinate system are shown in the figure. The plane formed by the vertical axis and the height axis of the initial coordinate system is perpendicular to the horizontal axis (X axis) of the initial coordinate system. It should be noted that if the material frame in the vehicle production workshop is on the conveyor line, Figure 3The vehicle travel direction shown may also be the travel direction of the material frame. At least one displacement device may also be provided around the conveyor line of the material frame, and a positioning camera may be provided on the displacement device. For different types of material frames, the material frame detection system may obtain the initial shooting position and at least one displacement shooting position of the material frame based on the type of the material frame.
[0115] Step 102, controlling the camera in the vehicle production workshop to move to the initial shooting position and taking a picture to obtain an initial shooting image;
[0116] In the embodiment of the present invention, for a material frame located in a storage location, the material frame detection system can control the 3D camera on the track to move to an initial shooting position of the material frame, take a picture of the material frame, and obtain an initial shooting image of the material frame.
[0117] For the material frame located on the conveyor line, the material frame detection system can control the positioning camera to move to the initial shooting position of the material frame through the displacement device, take a picture of the material frame, and obtain the initial shooting image of the material frame.
[0118] In the embodiment of the present invention, the vehicle body detection system can control the positioning camera to move to the initial shooting position of the vehicle through the displacement device, take a picture of the vehicle body, and obtain an initial shooting image of the vehicle body.
[0119] Step 103, if the initial captured image does not meet the preset shooting requirements, the camera is controlled to move to any of the displaced shooting positions, and a photograph is taken to obtain a displaced captured image;
[0120] In an embodiment of the present invention, for a material frame located in a storage location, after the material frame detection system obtains an initial captured image of the material frame, if the initial captured image of the material frame does not meet the preset shooting requirements, the material frame detection system can control the 3D camera on the track to move to any displacement shooting position of the material frame, and take a picture to obtain a displacement captured image of the material frame. The initial captured image of the material frame not meeting the preset shooting requirements may refer to the problem that the initial captured image of the material frame does not include images of all structures of the material frame, or the initial captured image is not clear.
[0121] For the material frame on the conveyor line, after the material frame detection system obtains the initial captured image of the material frame, if the initial captured image of the material frame does not meet the preset shooting requirements, the material frame detection system can control the positioning camera to move to any displacement shooting position of the material frame through the displacement device, take a picture of the material frame, and obtain the displacement captured image of the material frame. The initial captured image of the material frame not meeting the preset shooting requirements may mean that the initial captured image of the material frame does not include the material frame, or the initial captured image is not clear, etc.
[0122] In an embodiment of the present invention, after the vehicle body detection system obtains the initial captured image of the vehicle body, if the initial captured image of the vehicle body does not meet the preset shooting requirements, the vehicle body detection system can control the positioning camera to move to any displacement shooting position of the vehicle through the displacement device, take a picture of the vehicle body, and obtain the displacement captured image of the vehicle body. The initial captured image of the vehicle body not meeting the preset shooting requirements may refer to the initial captured image of the vehicle body not including the vehicle body, or the initial captured image is not clear, etc.
[0123] Step 104: if the displaced captured image meets the preset capture requirement, the displaced captured image is used as the target captured image, and an initial coordinate system and a displaced coordinate system of the camera are established; the initial coordinate system is a coordinate system with the initial capturing position as the origin; the displaced coordinate system is a coordinate system with the displaced capturing position corresponding to the target captured image as the origin;
[0124] In an embodiment of the present invention, if the displaced captured image meets the preset capture requirements, the vehicle production inspection system may use the displaced captured image as the target captured image. The vehicle production inspection system may also establish an initial coordinate system and a displaced coordinate system of the 3D camera and / or the positioning camera. The initial coordinate system is a coordinate system with the initial capture position of the 3D camera and / or the positioning camera as the origin, and the displaced coordinate system is a coordinate system with the displaced capture position of the 3D camera and / or the positioning camera when capturing the target captured image as the origin.
[0125] Step 105, based on the target captured image, the initial coordinate system and the displaced coordinate system, obtaining the spatial position of at least one feature point on the vehicle-related component; the feature point is a point on the vehicle-related component with visual features;
[0126] In an embodiment of the present invention, a vehicle production detection system can obtain the spatial position of at least one feature point on a vehicle-related component based on a target shot image, an initial coordinate system, and a displaced coordinate system. A feature point is a point on a vehicle-related component that has a visual feature. Taking the body of a vehicle on a conveyor line as an example, the body detection system can obtain the coordinate information of at least one feature point on the body in the displaced coordinate system of a positioning camera based on the target shot image of the body. Then, based on the coordinate information of the feature point in the displaced coordinate system of the positioning camera, the body detection system can obtain the coordinate information of the feature point in the initial coordinate system of the positioning camera. The steps of obtaining the spatial position of a feature point on a material frame in a storage location or a conveyor line are similar to the steps of obtaining the spatial position of a feature point on the body of a vehicle on a conveyor line, and the present invention will not be repeated here.
[0127] Step 106: Based on the spatial position of the feature point, detect whether the vehicle-related components are qualified.
[0128] In the embodiment of the present invention, based on the spatial position of at least one feature point on the vehicle-related component, the vehicle production inspection system can detect whether the vehicle-related component is qualified.
[0129] In the case where the vehicle-related component is a material frame on a storage location, the material frame usually includes at least one slot and at least one limit structure. Based on the spatial position of the characteristic points on the material frame, the material frame detection system can obtain the spatial position of the slot and the limit structure, and then determine whether the slot and the limit structure can allow the part to be placed in the material frame. In the case where the slot and the limit structure can allow the part to be placed in the material frame, the material frame detection system can confirm that the material frame is qualified; in the case where the slot and the limit structure are severely deformed and cannot allow the part to be placed in the material frame, the material frame detection system can confirm that the material frame is unqualified.
[0130] When the vehicle-related component is a material frame on a conveyor line, the material frame moves along the conveyor line in the vehicle production workshop. The material frame detection system can determine the position of the material frame on the conveyor line based on the spatial position of the feature points on the material frame. The material frame detection system can control the material frame detection robot to move around the material frame position, and follow the material frame when the material frame moves along the conveyor line, detect the material frame, and determine whether the material frame is qualified.
[0131] In the case where the vehicle-related component is the body of the vehicle on the conveyor line, the vehicle moves along the conveyor line in the vehicle production workshop. The body detection system can determine the position of the vehicle body on the conveyor line based on the spatial position of the feature points on the body. The body detection system can control the gap and flushness detection robot to move around the body position, and follow the body while the body moves along the conveyor line, detect the gap and flushness of the body, and determine whether the body is qualified.
[0132] In an embodiment of the present invention, a vehicle production inspection system is used to inspect vehicle-related components in a preset vehicle production workshop. Based on the type of the vehicle-related component, the vehicle production inspection system obtains the initial shooting position and at least one displacement shooting position of the vehicle-related component, controls the camera in the vehicle production workshop to move to the initial shooting position, and takes a picture to obtain an initial shooting image. If the initial shooting image does not meet the preset shooting requirements, the vehicle production inspection system controls the camera to move to any displacement shooting position, and takes a picture to obtain a displacement shooting image. If the displacement shooting image meets the preset shooting requirements, the displacement shooting image is used as the target shooting image, and the initial coordinate system and displacement coordinate system of the camera are established. The initial coordinate system is a coordinate system with the initial shooting position as the origin, and the displacement coordinate system is a coordinate system with the displacement shooting position corresponding to the target shooting image as the origin. Based on the target shooting image, the initial coordinate system and the displacement coordinate system, the vehicle production inspection system obtains the spatial position of at least one feature point on the vehicle-related component, and the feature point is a point with visual features on the vehicle-related component. Based on the spatial position of the feature point, the vehicle production inspection system can detect whether the vehicle-related component is qualified. In the embodiment of the present invention, the camera in the vehicle production workshop is movable, which solves the problem that the visual position of the 3D camera in the vehicle production workshop is usually fixed, resulting in the inability to capture the columns or beams of certain material frames, thereby failing to accurately determine the deviation value of the material frame, and the position of the positioning camera used to determine the position of the vehicle body or material frame on the conveyor line is usually fixed. Once the deviation of the vehicle body or material frame on the conveyor line exceeds the detection range of the positioning camera or the wheelbase change of the vehicle causes the vehicle body to be out of the detection range of the positioning camera, the positioning camera will not be able to accurately locate the vehicle body or material frame, thereby affecting the accuracy and efficiency of the vehicle body or material frame detection. In addition, in the embodiment of the present invention, the spatial position of at least one feature point on the vehicle-related component is obtained through the target captured image of the vehicle-related component, the initial coordinate system of the camera and the displacement coordinate system, and the spatial position of the vehicle-related component and at least one structure on the vehicle-related component is obtained.
[0133] Further, in any of the above embodiments, after step 103, the following steps are included:
[0134] Sub-step S11, if the displaced photographed image meets the preset photographing requirement, stop moving the camera;
[0135] Sub-step S12, if the displaced shooting image does not meet the preset shooting requirements, repeatedly controlling the camera to move to any of the displaced shooting positions and taking pictures to obtain the displaced shooting image, until a displaced shooting image that meets the preset shooting requirements is obtained.
[0136] In an embodiment of the present invention, if the initial captured image of the vehicle-related component does not meet the preset shooting requirements, the camera is controlled to move to any displacement shooting position and take a picture to obtain the displacement captured image of the vehicle-related component. If the displacement captured image meets the preset shooting requirements, the vehicle production inspection system stops moving the camera. If the displacement captured image does not meet the preset shooting requirements, the vehicle production inspection system repeats the steps of controlling the camera to move to any displacement shooting position and take a picture to obtain the displacement captured image until the displacement captured image that meets the preset shooting requirements is obtained.
[0137] For the vehicle body on the conveyor line, the vehicle body detection system can control the positioning camera to move to the initial shooting position of the vehicle through the displacement device, take a picture of the vehicle body, and obtain the initial shooting image of the vehicle body. If the initial shooting image of the vehicle body does not meet the preset shooting requirements, the positioning camera is controlled to move to any displacement shooting position through the displacement device, and take a picture to obtain the displacement shooting image of the vehicle body. If the displacement shooting image meets the preset shooting requirements, the vehicle body detection system stops moving the positioning camera. If the displacement shooting image does not meet the preset shooting requirements, the vehicle body detection system repeats the steps of controlling the camera to move to any displacement shooting position, take a picture, and obtain the displacement shooting image until the displacement shooting image that meets the preset shooting requirements is obtained. The steps of obtaining the displacement shooting image of the material frame on the conveyor line that meets the preset shooting requirements are similar to the steps of obtaining the displacement shooting image of the vehicle body on the conveyor line that meets the preset shooting requirements, and the present invention will not be repeated here.
[0138] For the material frame on the storage location, the material frame detection system can control the 3D camera to move to the initial shooting position of the material frame through the track, take a picture of the material frame, and obtain the initial shooting image of the material frame. If the initial shooting image of the material frame does not meet the preset shooting requirements, the 3D camera is controlled to move to any displacement shooting position through the track, and take a picture to obtain the displacement shooting image of the material frame. If the displacement shooting image meets the preset shooting requirements, the material frame detection system stops moving the 3D camera. If the displacement shooting image does not meet the preset shooting requirements, the material frame detection system repeats the steps of controlling the camera to move to any displacement shooting position, take a picture, and obtain the displacement shooting image, until the displacement shooting image that meets the preset shooting requirements is obtained.
[0139] In an embodiment of the present invention, if the displaced shot image does not meet the preset shooting requirements, the steps of controlling the camera to move to any displaced shooting position and taking pictures to obtain the displaced shot image are repeated until the displaced shot image that meets the preset shooting requirements is obtained. This realizes obtaining the displaced shot image that meets the preset shooting requirements and provides support for obtaining the spatial position of vehicle-related components and any structure of vehicle-related components based on the displaced shot image.
[0140] Further, in any of the above embodiments, step 105 includes:
[0141] Sub-step S21, based on the target captured image, obtaining the coordinate information of the feature point on the vehicle-related component in the displacement coordinate system;
[0142] Sub-step S22, based on the coordinate information of the feature point in the displaced coordinate system, obtaining the coordinate information of the feature point in the initial coordinate system.
[0143] In an embodiment of the present invention, the vehicle production detection system can obtain the spatial position of at least one feature point on the vehicle-related component based on the target shot image, the initial coordinate system, and the displacement coordinate system. Specifically, for the material frame on the storage location, the material frame detection system can obtain the coordinate information of at least one feature point on the material frame in the displacement coordinate system of the 3D camera based on the target shot image of the material frame. Then, based on the coordinate information of the feature point in the displacement coordinate system of the 3D camera, the material frame detection system can obtain the coordinate information of the feature point in the initial coordinate system of the 3D camera.
[0144] For the vehicle body on the conveyor line, the vehicle body detection system can obtain the coordinate information of at least one feature point on the vehicle body in the displacement coordinate system of the positioning camera based on the target shot image of the vehicle body. Then, based on the coordinate information of the feature point in the displacement coordinate system of the positioning camera, the vehicle body detection system can obtain the coordinate information of the feature point in the initial coordinate system of the positioning camera. The step of obtaining the coordinate information of the feature point on the material frame on the conveyor line in the initial coordinate system of the positioning camera is similar to the step of obtaining the coordinate information of the feature point on the vehicle body on the conveyor line in the initial coordinate system of the positioning camera, and the present invention will not be repeated here.
[0145] In an embodiment of the present invention, a vehicle production inspection system can obtain coordinate information of feature points on vehicle-related components in a displaced coordinate system based on a target captured image, and can obtain coordinate information of feature points in an initial coordinate system based on the coordinate information of the feature points in the displaced coordinate system, thereby realizing the acquisition of the spatial position of the vehicle body and / or material frame, and further based on the spatial position of the vehicle body and / or material frame, the inspection of the vehicle body and / or material frame can be realized.
[0146] Further, in any of the above embodiments, step 106 includes:
[0147] Sub-step S31, acquiring the position of the vehicle-related component based on the coordinate information of the feature point in the initial coordinate system;
[0148] Sub-step S32, controlling the component inspection equipment in the vehicle production workshop to move to the vicinity of the location of the vehicle-related component to inspect whether the vehicle-related component is qualified.
[0149] In an embodiment of the present invention, based on the coordinate information of at least one feature point on the vehicle-related component in the initial coordinate system, the position of the vehicle-related component can be obtained. For the body or material frame of the vehicle on the conveyor line, the vehicle production inspection system can control the component inspection equipment in the vehicle production workshop to move to the vicinity of the position of the vehicle-related component to detect whether the vehicle-related component is qualified.
[0150] If the vehicle-related component is the body of the vehicle on the conveyor line, the component detection device may be a gap and flushness detection robot, and the vehicle moves along the conveyor line in the vehicle production workshop. The body detection system may control the gap and flushness detection robot to move around the body position, and follow the body as it moves along the conveyor line, detect the gap and flushness of the body, and determine whether the body is qualified.
[0151] If the vehicle-related component is a material frame on a conveyor line, the component inspection device can be a material frame inspection robot, and the material frame moves along the conveyor line in the vehicle production workshop. The material frame inspection system can control the material frame inspection robot to move to the vicinity of the material frame position, and follow the material frame as it moves along the conveyor line, inspect the material frame, and determine whether the material frame is qualified.
[0152] Reference Figure 4 , showing a schematic diagram of a vehicle body detection system provided in an embodiment of the present invention. The vehicle body detection system includes: 4 robots, 4 gap and flushness detection systems, 4 positioning cameras, an industrial computer, a programmable logic controller (PLC) and 4 encoders. The robot is connected to the gap and flushness detection system, and the gap and flushness detection system and the robot together constitute a gap and flushness detection robot. The positioning camera is also called a 3D camera and a line edge positioning camera. The encoder is communicatively connected to the industrial computer.
[0153] The gap and flush detection system can detect gaps and flushes on the vehicle body, that is, the distance differences and surface smoothness differences between parts on the vehicle body. The positioning camera is used to capture an image of the vehicle body, obtain the positioning data of the vehicle body based on the image of the vehicle body, and send the positioning data to the gap and flush detection robot. The gap and flush detection robot can move around the vehicle body based on the positioning data.
[0154] The vehicle moves along the conveyor line, and the encoder is used to collect the moving speed of the vehicle body. The industrial computer is used to obtain the moving speed of the vehicle body collected by the encoder, calculate and obtain the positioning result of the moving vehicle body, and send the positioning result to the gap and flushness detection robot. After the gap and flushness detection robot moves around the vehicle body, the gap and flushness detection robot can follow the movement of the vehicle body based on the positioning result to detect the gap and flushness of the vehicle body.
[0155] The gap and flushness detection robot can send the detection results of the gap and flushness of the vehicle body to the industrial computer. The industrial computer also includes a screen, and the industrial computer can obtain and record the detection results of the gap and flushness of the vehicle body, and can also display the detection results on the screen.
[0156] The PLC can control the start and stop of the gap and flushness detection robot for body inspection. The PLC also includes vision software. The vision software can also obtain the image taken by the positioning camera, process the image, obtain the position of the body, and send the position of the body to the gap and flushness detection robot, so that the gap and flushness detection robot can move around the body. It should be noted that Figure 4 The gap and flushness detection system in the system can also be used to detect the material frame. When the gap and flushness detection system is used to detect the material frame, the gap and flushness detection system and the robot together constitute a material frame detection robot. Figure 4 The moving direction of the car is consistent with the moving direction of the material frame. Figure 4 When the gap and flushness detection system in the material frame is used to detect the material frame, Figure 4 The body detection system in is also a material frame detection system used to detect the material frames on the conveyor line.
[0157] Reference Figure 5 , showing a schematic diagram of another vehicle body detection system provided in an embodiment of the present invention. The vehicle body detection system includes: 4 robots, 4 sets of gap and flushness detection systems, 4 positioning cameras, an industrial computer, a programmable logic controller (PLC), 4 sets of encoders and a displacement device. The robot is connected to the gap and flushness detection system, and the gap and flushness detection system and the robot together constitute a gap and flushness detection robot. The encoder is communicatively connected to the industrial computer. The positioning camera is set on the displacement device, and the positioning camera can move the positioning camera through the displacement device. The PLC is communicatively connected to the displacement device, and the PLC can send the vehicle model information of the vehicle body to the displacement device. It should be noted that, Figure 5 The gap and flushness detection system in the can also be used to detect the material frame. Figure 5 When the gap and flushness detection system in the material frame is used to detect the material frame, Figure 5 The body detection system in is also a material frame detection system used to detect the material frames on the conveyor line.
[0158] Reference Figure 6 , showing a schematic flow chart of a vehicle body detection method provided in an embodiment of the present invention.
[0159] In the first part, the line-side positioning camera is repositioned according to the vehicle model. The line-side positioning camera is the positioning camera. MOM (Manufacturing Operations Management) sends the vehicle model data of the vehicle body entering the conveyor line to the PLC, and the PLC sends the vehicle model data to the positioning device. The positioning device automatically repositions the positioning camera according to the vehicle model data and moves the positioning camera to the initial shooting position.
[0160] The second part is the determination of the body deviation. When the vehicle moves along the conveyor line, the vehicle can trigger the positioning photo trigger sensor. After the positioning photo trigger sensor is triggered, the line-side positioning camera can take a photo at the initial shooting position. If the initial shooting image of the line-side positioning camera meets the preset shooting requirements, the line-side positioning camera detection is successful, and the initial shooting image taken by the line-side positioning camera at the initial shooting position can be used as the target shooting image. The line-side positioning camera can output the position of the body according to the target shooting image, that is, the deviation of the body on the conveyor line.
[0161] The third part is the automatic displacement of the line-side positioning camera. Since the detection range of the positioning camera is limited to ±25mm, when the deviation of the vehicle body on the conveyor line exceeds the detection range of ±25mm, the image captured by the line-side positioning camera does not include the vehicle body, that is, the line-side positioning camera fails to detect. The line-side positioning camera can send a displacement instruction to the PLC, and the PLC can forward the displacement instruction to the displacement device. The displacement device can move the line-side positioning camera to multiple displacement shooting positions such as the second or third position to shoot the vehicle body until the line-side positioning camera detects successfully, that is, the image captured by the line-side positioning camera meets the preset shooting requirements, and the image captured by the line-side positioning camera at the displacement shooting position that meets the preset shooting requirements can be used as the target shooting image, and the displacement shooting position where the target shooting image is captured can be used as the target displacement shooting position. It should be noted that since the vehicle body has a certain length, even if the vehicle body moves with the conveyor line, the line-side positioning camera can still shoot the vehicle body after spending a certain amount of time to displace.
[0162] When the displacement device moves the line-side positioning camera to the target displacement shooting position, and the target shooting image shot at the target displacement shooting position meets the preset shooting requirements, the line-side positioning camera can establish the initial coordinate system and displacement coordinate system of the line-side positioning camera according to the moving direction and distance of the displacement device. Among them, the origin of the initial coordinate system is the initial shooting position, and the origin of the displacement coordinate system is the target displacement shooting position. According to the target shooting image, the spatial position of the vehicle body in the displacement coordinate system can be obtained, and the spatial position of the vehicle body in the displacement coordinate system can also be converted to obtain the spatial position of the vehicle body in the initial coordinate system. The line-side positioning camera and the visual software in the PLC can obtain the deviation value of the vehicle body on the conveyor line, that is, the position of the vehicle body on the conveyor line, according to the spatial position of the vehicle body in the initial coordinate system.
[0163] Part 4: Body gap and flushness detection. The gap and flushness detection robot can obtain the deviation value of the body on the conveyor line sent by the line-side positioning camera or the visual software in the PLC, and start the gap and flushness detection of the body after triggering the body detection. The gap and flushness detection robot can move around the body to detect the body. In addition, the industrial computer in the body detection system can obtain the body movement speed collected by the encoder, calculate and obtain the positioning result of the moving body, and send the positioning result to the gap and flushness detection robot. After the gap and flushness detection robot moves around the body, the gap and flushness detection robot can follow the body based on the positioning result to detect the body gap and flushness.
[0164] Part 5: Gap and flush detection is completed, and the robot exits. The visual software can obtain the body detection results of the gap and flush detection robot, and display the body detection results in real time on the screen of the industrial computer. After the gap and flush detection robot completes the detection, the gap and flush detection robot exits, and the screen can wait to display the next body detection result. It should be noted that the detection method of the material frame on the conveyor line is the same as that of the Figure 6 The detection method of the vehicle body shown is similar, and the present invention will not be repeated here.
[0165] Reference Figure 7 , shows a schematic diagram of a process for obtaining the spatial position of a vehicle body based on an initial coordinate system and a displaced coordinate system provided in an embodiment of the present invention.
[0166] First, the vehicle body detection system can obtain the initial shooting position of the positioning camera corresponding to different vehicle models according to different vehicle models. It should be noted that the vehicle widths of vehicles of different vehicle models are different. The vehicle body detection system can receive the vehicle model information sent by MOM, and then control the positioning camera to move to the initial shooting position of the positioning camera corresponding to the vehicle model information, and establish an initial coordinate system T0. Among them, the origin of the initial coordinate system is the initial shooting position. If the initial shooting image taken by the positioning camera at the initial shooting position meets the preset shooting requirements, the vehicle body detection system can use the initial shooting image as the target shooting image.
[0167] Secondly, if the initial captured image does not meet the preset shooting requirements, the vehicle body inspection system can control the positioning camera to move to any displacement shooting position through the displacement device and take pictures. The displacement device is also called a servo moving mechanism. The displacement device moves within the range of 15 to 50 mm and -15 to -50 mm relative to the initial shooting position. Figure 3 In the initial coordinate system shown, when the quality of the initial image taken by the positioning camera at the initial shooting position is poor, the initial image does not meet the preset shooting requirements. If the positioning camera detects that the deviation of the vehicle body on the conveyor line is greater than 25mm based on the initial shooting image, resulting in poor image quality, the positioning camera moves along the Y-axis direction of the initial coordinate system to a position 15 to 50mm away from the initial shooting position through the displacement device; if the positioning camera detects that the deviation of the vehicle body on the conveyor line is less than -25mm based on the initial shooting image, resulting in poor image quality, the positioning camera moves along the Y-axis direction of the initial coordinate system to a position -15 to -50mm away from the initial shooting position through the displacement device.
[0168] If the displacement shooting image taken by the positioning camera at a certain displacement shooting position meets the preset shooting requirements, the displacement shooting position is regarded as the target displacement shooting position, and the displacement shooting image is regarded as the target shooting image. According to the initial coordinate system and the moving distance and direction of the displacement device, a displacement coordinate system T1 can be created. Among them, the origin of the displacement coordinate system is the target displacement shooting position. The body detection system can obtain the spatial position of the feature points on the body in T1 based on the target shooting image, and can also convert the spatial position of the feature points on the body in T1 into the spatial position of the feature points on the body in T0.
[0169] In a specific example, the displacement device moves along the X-axis, Y-axis and Z-axis directions of the initial coordinate system. If the movement components of the three coordinate axes of the displacement coordinate system T1 relative to the three coordinate axes of the initial coordinate system T0 are dx=Tx, dy=Ty, dz=Tz, respectively, then the relationship between T1 and T0 is:
[0170] T1=T0*T
[0171]
[0172] Where T is the coordinate system transformation matrix from T0 to T1.
[0173] If the spatial position of the feature point on the vehicle body in the displacement coordinate system is P1-Pn, the spatial position of the feature point in the displacement coordinate system P1-Pn can be converted to obtain the spatial position of the feature point in the initial coordinate system P101-Pn1.
[0174] The conversion formula between spatial point P101 and spatial point P1 is:
[0175] P101=T -1 P1
[0176] Among them, T -1 is the inverse matrix of T. The conversion formulas for the remaining spatial points are similar to the conversion formula between P101 and P1, and the present invention will not be repeated here.
[0177] Next, the positioning camera can calculate the spatial positions of the feature points on the vehicle body in the target shooting image based on the parallax method and triangular geometric relationship according to the target shooting image shot at the initial shooting position or the target displacement shooting position.
[0178] Reference Figure 8 , which shows a principle diagram of a triangular geometric relationship provided in an embodiment of the present invention. Fig. 9 , shows a parallax principle diagram provided in an embodiment of the present invention. There are two positioning cameras distributed around the vehicle body, namely a left positioning camera and a right positioning camera. The target image captured by the left positioning camera is the left image, and the target image captured by the right positioning camera is the right image. Figure 8 The triangulation principle shown in is:
[0179] Z=b*(f / x)
[0180] Among them, Z is the distance from the feature point to the camera imaging plane, b is the baseline distance between the two positioning cameras, that is, the distance between the center points of the two positioning cameras, f is the focal length of the camera lens, and x is twice the parallax.
[0181] Fig. 9 The parallax principle in is:
[0182] Disparity = X left -X right
[0183]
[0184] Among them, X leftis the horizontal position of the feature point on the imaging plane of the left positioning camera, X right is the horizontal position of the feature point on the imaging plane of the right positioning camera, P(x c ,y c , z c ) is the spatial position of the feature point on the vehicle body, and Y is the vertical position of the feature point on the imaging plane of any positioning camera.
[0185] Finally, the deviation of the vehicle body on the conveyor line can be calculated using the spatial position of the feature points on the vehicle body. The vehicle body detection system may include 4 positioning cameras, which can obtain multiple spatial positions of the feature points on the vehicle body, and then obtain multiple vehicle body positions. The vehicle body detection system performs spatial fitting calculations on multiple vehicle body positions and reference vehicles, and can obtain the rotation matrix of the current vehicle body relative to the reference vehicle body. The vehicle body detection system performs 3D affine transformation on the rotation matrix of the vehicle body, and can obtain the offset value of the current vehicle body relative to the reference vehicle body. It should be noted that the reference vehicle body is an ideal vehicle body position that enables the positioning camera to capture the preset shooting requirements at the initial shooting position. When determining the reference vehicle body of the vehicle, the vehicle body detection system can obtain multiple vehicle body positions of the reference vehicle body under the positioning camera, and these positions will be saved and used for spatial fitting calculations of multiple vehicle body positions and the reference vehicle body.
[0186] In a specific example, four positioning cameras are distributed on both sides of the vehicle body, and the offset value of the current vehicle body relative to the reference vehicle body is 6 degrees of freedom. It should be noted that the steps of obtaining the spatial position of the material frame on the conveyor line or storage location based on the initial coordinate system and the displacement coordinate system are the same as Figure 7 The steps of obtaining the spatial position of the vehicle body based on the initial coordinate system and the displacement coordinate system are similar to those shown in FIG. Figure 8 The principles and Fig. 9 The parallax principle shown can also be used in the step of obtaining the spatial position of a material frame on a conveyor line or a storage location based on an initial coordinate system and a displacement coordinate system, and the present invention will not be described in detail here.
[0187] In an embodiment of the present invention, the position of the vehicle body is obtained based on the coordinate information of the feature points in the initial coordinate system, and the vehicle body inspection equipment in the vehicle production workshop is controlled to move to the vicinity of the position of the vehicle body to detect whether the vehicle body is qualified, thereby realizing the detection of the surface difference and gap of the vehicle body.
[0188] In the embodiment of the present invention, the use of the displacement device greatly improves the vehicle type compatibility and line compatibility of vehicle body detection. Since the displacement device can move the positioning camera to different positions for shooting, it is compatible with the positioning of the vehicle body position of vehicles with different wheelbases, and solves the problem that the vehicle body is not within the detection range of the positioning camera due to the change of the vehicle wheelbase, and the positioning camera will not be able to accurately position the vehicle body, which greatly reduces the hardware investment in the later stage.
[0189] Further, in any of the above embodiments, the vehicle-related components include a material frame; the material frame is located in a storage location of the vehicle production workshop; the material frame includes at least one card slot and at least one limit structure; the vehicle production detection system is used to detect whether the card slot and the limit structure of the material frame can enable the parts of the vehicle to be placed in the material frame; step 106 includes:
[0190] Sub-step S41, obtaining a storage location coordinate system of the storage location; the storage location coordinate system is a coordinate system with any point of the storage location as the origin;
[0191] Sub-step S42, based on the coordinate information of the feature point in the initial coordinate system, obtaining the slot position parameters of the at least one slot in the storage location coordinate system;
[0192] Sub-step S43, fitting the slot position parameters using a preset least squares method to obtain a target fitting model;
[0193] Sub-step S44, determining the predicted position parameters of the card slot based on the target fitting model;
[0194] Sub-step S45, determining a target card slot from the at least one card slot according to a difference between the card slot position parameter and the predicted position parameter;
[0195] Sub-step S46, establishing a material frame coordinate system based on the slot position parameters of the target slot.
[0196] In an embodiment of the present invention, the vehicle-related components further include a material frame, which is located in a storage location of a vehicle production workshop. The material frame detection system can obtain a storage location coordinate system of the storage location, wherein the storage location coordinate system is a coordinate system with any point of the storage location as the origin.
[0197] Reference Fig.10, shows a schematic diagram of a material frame provided in an embodiment of the present invention. The material frame includes a reference plate, at least one slot and at least one limit structure. The material frame detection system is used to detect whether the slot and limit structure of the material frame can allow vehicle parts to be placed in the material frame. If the slot and limit structure of the material frame can allow vehicle parts to be placed in the material frame, the material frame is qualified; if the slot and limit structure of the material frame cannot allow vehicle parts to be placed in the material frame, the material frame is unqualified.
[0198] Reference Fig.11 , shows a schematic diagram of a process of controlling a camera to shoot a material frame by a material frame detection system provided in an embodiment of the present invention. On a vehicle parts production line, the material frame detection system automatically selects and loads corresponding parameter settings according to the type of parts produced, and controls the camera to move to the corresponding initial shooting position to prepare for shooting.
[0199] After the material frame enters the storage location, the material frame will trigger the grating. After the material frame triggers the grating, the material frame detection system sends a photo signal to the camera, and the camera takes a photo of the reference plate, card slot and limit of the material frame to obtain the initial photo image. Among them, the limit refers to the limit structure.
[0200] If the initial captured image does not meet the preset shooting requirements, the camera is controlled to move to any displacement shooting position and take pictures to obtain the displacement captured image. If the displacement captured image meets the preset shooting requirements, the displacement captured image is used as the target captured image, and the initial coordinate system and the displacement coordinate system of the camera are established. The initial coordinate system is a coordinate system with the initial shooting position as the origin, and the displacement coordinate system is a coordinate system with the displacement shooting position corresponding to the target captured image as the origin. If the displacement captured image does not meet the preset shooting requirements, the steps of controlling the camera to move to any displacement shooting position and taking pictures to obtain the displacement captured image are repeated until a displacement captured image that meets the preset shooting requirements is obtained.
[0201] Based on the target shot image, the initial coordinate system and the displacement coordinate system, the spatial position of at least one feature point on the material frame can be obtained. Based on the spatial position of at least one feature point on the material frame, the position parameters of the reference plate, at least one card slot and at least one limit structure in the storage location coordinate system can be obtained. The position parameters of the card slot in the storage location coordinate system can be called the card slot position parameters. Among them, the method for obtaining the spatial position of at least one feature point on the material frame in the storage location is similar to the method for obtaining the spatial position of at least one feature point on the vehicle body, and the present invention will not be repeated here.
[0202] In addition, the material frame detection system can pre-process the point cloud data obtained by shooting through methods such as through-filtering, leaving point filtering and voxel network filtering, and identify the processed point cloud data to obtain the position parameters of the reference plate, card slot and limit structure in the warehouse coordinate system.
[0203] In an embodiment of the present invention, the material frame detection system can use a preset least square method to fit the slot position parameters to obtain a target fitting model. The target fitting model is the best fitting model. Based on the best fitting model, the material frame detection system can determine the predicted position parameters of the slot in the best fitting model. According to the difference between the slot position parameters and the predicted position parameters, that is, the residual between the slot and the best fitting model, the target slot can be determined from at least one slot. Specifically, different slot position parameters are sorted according to the size of the difference, and in the sorted data set, the three slots located in the middle position are used as the target slots. Finally, based on the slot position parameters of the target slot, a material frame coordinate system can be established. The origin of the material frame coordinate system is any position of the target slot.
[0204] In an embodiment of the present invention, a storage location coordinate system of a storage location is obtained, and the storage location coordinate system is a coordinate system with any point of the storage location as the origin. Based on the coordinate information of the feature point in the initial coordinate system, the slot position parameters of at least one slot in the storage location coordinate system are obtained. The slot position parameters are fitted using a preset least squares method to obtain a target fitting model, and the predicted position parameters of the slot are determined based on the target fitting model. According to the difference between the slot position parameters and the predicted position parameters, a target slot is determined from at least one slot, and a material frame coordinate system is established based on the slot position parameters of the target slot, thereby realizing the establishment of a material frame coordinate system based on the target slot, and providing support for judging whether the material frame is qualified based on the material frame coordinate system.
[0205] In an embodiment of the present invention, the material frame detection system only collects the position data of the card slot, the limit structure and the reference plate, thereby reducing the amount of data collected, and through point cloud data screening and data simplification, improves the point cloud accuracy and reduces the data processing time, thereby meeting the needs of rapid placement of parts in the vehicle production line.
[0206] Further, in any of the above embodiments, step 106 includes:
[0207] Sub-step S51, assuming that the material frame is qualified and the preset assumed part is placed in the material frame, obtaining a part coordinate system of the assumed part in the material frame; the part coordinate system is a coordinate system with any point of the assumed part as the origin;
[0208] Sub-step S52, adjusting the part coordinate system so that the part coordinate system coincides with the material frame coordinate system;
[0209] Sub-step S53, taking the remaining card slots in the at least one card slot except the target card slot as the card slots to be processed;
[0210] Sub-step S54, obtaining part coordinate parameters of the assumed part in the material frame coordinate system, slot coordinate parameters of the to-be-processed slot in the material frame coordinate system, and position-limiting structure coordinate parameters of the position-limiting structure in the material frame coordinate system;
[0211] Sub-step S55, judging whether the slot and the limiting structure can enable the assumed part to be placed in the material frame based on the part coordinate parameters, the slot coordinate parameters of the slot to be processed and the limiting structure coordinate parameters;
[0212] Sub-step S56: If the slot and the limiting structure enable the assumed part to be placed in the material frame, then the material frame is confirmed to be qualified.
[0213] In an embodiment of the present invention, when the material frame is assumed to be qualified and the preset assumed part has been placed in the material frame, the part coordinate system of the assumed part in the material frame is obtained. The part coordinate system is a coordinate system with any point of the assumed part as the origin. Then, the part coordinate system is adjusted so that the part coordinate system coincides with the material frame coordinate system. The remaining card slots in at least one card slot except the target card slot are used as the card slots to be processed, and based on the coordinate information of at least one feature point in the material frame in the initial coordinate system, the part coordinate parameters of the assumed part in the material frame coordinate system, the card slot coordinate parameters of the card slot to be processed in the material frame coordinate system, and the limit structure coordinate parameters of the limit structure in the material frame coordinate system are obtained.
[0214] In an embodiment of the present invention, based on the part coordinate parameters, the slot coordinate parameters of the slot to be processed and the coordinate parameters of the limiting structure, it can be determined whether the slot and the limiting structure can enable the assumed part to be placed in the material frame. Specifically, based on the part coordinate parameters, the slot coordinate parameters of the slot to be processed and the coordinate parameters of the limiting structure, it can be determined whether the slot coordinate parameters of the slot to be processed and the limiting structure coordinate parameters of the limiting structure coincide with the part coordinate parameters of the assumed part, and whether the assumed part is located within the range of the slot to be processed and the limiting structure. If the slot coordinate parameters of the slot to be processed and the limiting structure coordinate parameters of the limiting structure do not coincide with the part coordinate parameters of the assumed part, and the assumed part is located within the range of the slot to be processed and the limiting structure, then the slot and the limiting structure can enable the assumed part to be placed in the material frame, and the material frame is confirmed to be qualified. If the slot coordinate parameters of the slot to be processed and the limiting structure coordinate parameters of the limiting structure have overlapping parts with the part coordinate parameters of the assumed part, and the assumed part cannot be located within the range of the slot to be processed and the limiting structure, then the slot and the limiting structure cannot enable the assumed part to be placed in the material frame, and the material frame is confirmed to be unqualified.
[0215] Reference Fig.12, shows a schematic diagram of a process for determining whether a material frame is qualified provided in an embodiment of the present invention. First, the material frame detection system can use the least squares method to process the slot position parameters in the storage location coordinate system to obtain the best fitting model. Based on the best fitting model, the material frame detection system can obtain the residual between the slot and the best fitting model, and sort the different slot position parameters according to the size of the residual. In the sorted data set, the three slots in the middle position are used as target slots. Based on the target slots, the material frame detection system can establish a material frame coordinate system.
[0216] When the assumed material frame is qualified and the preset assumed part has been placed in the material frame, the part coordinate system of the assumed part in the material frame can be established based on the position parameters of the assumed part in the storage location coordinate system. Then, the part coordinate system is fitted to the material frame coordinate system, that is, the part coordinate system is adjusted so that the part coordinate system coincides with the material frame coordinate system. The part coordinate parameters of the assumed part in the material frame coordinate system are obtained, and based on the part coordinate parameters, it is determined whether the assumed part is within the range of the slot.
[0217] If the part is not within the range of the slot, the material frame is determined to be unqualified and the material frame is removed from the production line for repair. If the part is within the range of the slot, the material frame is determined to be qualified and the robot is controlled to load the part into the material frame.
[0218] In an embodiment of the present invention, when it is assumed that the material frame is qualified and a preset assumed part is placed in the material frame, a part coordinate system of the assumed part in the material frame is obtained; the part coordinate system is a coordinate system with any point of the assumed part as the origin; the part coordinate system is adjusted so that the part coordinate system coincides with the material frame coordinate system; the remaining card slots in at least one slot except the target card slot are used as the card slots to be processed; the part coordinate parameters of the assumed part in the material frame coordinate system, the card slot coordinate parameters of the card slot to be processed in the material frame coordinate system and the limiting structure coordinate parameters of the limiting structure in the material frame coordinate system are obtained; based on the part coordinate parameters, the card slot coordinate parameters of the card slot to be processed and the limiting structure coordinate parameters, it is judged whether the card slot and the limiting structure can enable the assumed part to be placed in the material frame; if the card slot and the limiting structure can enable the assumed part to be placed in the material frame, the material frame is confirmed to be qualified. By adjusting the part coordinate system so that the part coordinate system coincides with the material frame coordinate system, whether the material frame is qualified is judged based on the part coordinate parameters of the assumed part in the material frame coordinate system, the slot coordinate parameters of the slot to be processed and the limit structure coordinate parameters, thereby eliminating the process of confirming the limit deviation values of the slot, limit structure and other structures of the material frame, reducing the debugging and optimization time of the material frame, and achieving that the material frame that can be assembled manually can also be assembled by the robot, thereby greatly reducing the accuracy requirements of the material frame and the cost of material frame development. In addition, unqualified material frames can be repaired in a targeted manner, greatly reducing the maintenance cost of the material frame.
[0219] Furthermore, in any of the above embodiments, the method includes:
[0220] Sub-step S61, if the material frame is qualified, obtaining the position information of the material frame in the storage location based on the coordinate information of the feature point in the initial coordinate system;
[0221] Sub-step S62, determining the placement position of the part based on the position information and the slot position parameters;
[0222] Sub-step S63, controlling the robot in the vehicle production workshop to place the part at the placement position.
[0223] In the embodiment of the present invention, if the material frame is qualified, based on the coordinate information of the characteristic points of the material frame in the initial coordinate system, the position information of the reference plate of the material frame in the storage location can be obtained, and then the position information of the material frame in the storage location can be obtained. Since the material frame is placed in the card slot, the placement position of the part can be determined based on the position information of the material frame in the storage location and the position information of the card slot in the material frame coordinate system. The material frame detection system generates a part placement trajectory based on the position information of the material frame in the storage location and the position information of the card slot in the material frame coordinate system, and then controls the robot to place the part at the placement position according to the placement trajectory.
[0224] Reference Fig.13 , showing a schematic diagram of a process of a robot loading parts into a material frame provided in an embodiment of the present invention. If the material frame is qualified, the material frame detection system can obtain the reference plate position parameters of the material frame's reference plate in the storage location coordinate system based on the coordinate information of the feature points of the material frame in the initial coordinate system, and determine the position of the material frame in the storage location. Since the material frame is placed in the card slot, based on the position of the material frame in the storage location and the position information of the card slot in the material frame coordinate system, the material frame detection system can plan the framing trajectory, generate the part placement trajectory, and then control the robot to place the part at the placement position according to the placement trajectory. It should be noted that the position information of the card slot in the material frame coordinate system is the part framing coordinate, and the material frame detection system is also called a control system. The reference plate in the material frame is used to position the material frame.
[0225] In the embodiment of the present invention, if the material frame is qualified, the position information of the material frame in the storage location is obtained based on the coordinate information of the feature point in the initial coordinate system, and the placement position of the part is determined based on the position information and the slot position parameters, and the robot in the vehicle production workshop is controlled to place the part at the placement position, so that the part is smoothly loaded into the qualified material frame. The material frame detection system automatically plans the robot frame loading trajectory, and each material frame only needs to be debugged once, which greatly reduces the debugging workload and debugging time.
[0226] It should be noted that, for the sake of simplicity, the method embodiments are described as a series of action combinations, but those skilled in the art should be aware that the embodiments of the present invention are not limited by the order of the actions described, because according to the embodiments of the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present invention.
[0227] Reference Fig.14 , shows a structural block diagram of a vehicle-related component detection device provided in an embodiment of the present invention, which is applied to a vehicle production detection system. The vehicle production detection system is used to detect vehicle-related components in a preset vehicle production workshop, and specifically may include the following modules:
[0228] A shooting position acquisition module 1401 is used to acquire an initial shooting position and at least one displaced shooting position of the vehicle-related component based on the type of the vehicle-related component;
[0229] The first moving module 1402 is used to control the camera in the vehicle production workshop to move to the initial shooting position and take pictures to obtain an initial shooting image;
[0230] The second moving module 1403 is used to control the camera to move to any of the displaced shooting positions and take pictures to obtain a displaced shooting image if the initial shot image does not meet the preset shooting requirements;
[0231] A coordinate system establishing module 1404 is used to use the displaced photographed image as a target photographed image if the displaced photographed image meets the preset photographing requirement, and establish an initial coordinate system and a displaced coordinate system of the camera; the initial coordinate system is a coordinate system with the initial photographing position as the origin; the displaced coordinate system is a coordinate system with the displaced photographing position corresponding to the target photographed image as the origin;
[0232] The spatial position acquisition module 1405 is used to acquire the spatial position of at least one feature point on the vehicle-related component based on the target captured image, the initial coordinate system and the displaced coordinate system; the feature point is a point on the vehicle-related component with visual features;
[0233] The detection module 1406 is used to detect whether the vehicle-related components are qualified based on the spatial position of the feature points.
[0234] In an optional embodiment of the present invention, the device comprises:
[0235] A stop movement module, used for stopping moving the camera if the displaced captured image meets the preset capture requirement;
[0236] A repetition module is used to repeatedly control the camera to move to any of the displaced shooting positions and take pictures to obtain the displaced shooting image if the displaced shooting image does not meet the preset shooting requirements, until a displaced shooting image that meets the preset shooting requirements is obtained.
[0237] In an optional embodiment of the present invention, the spatial position acquisition module includes:
[0238] A first coordinate information acquisition submodule, used for acquiring coordinate information of the feature point on the vehicle-related component in the displacement coordinate system based on the target captured image;
[0239] The second coordinate information acquisition submodule is used to acquire the coordinate information of the feature point in the initial coordinate system based on the coordinate information of the feature point in the displaced coordinate system.
[0240] In an optional embodiment of the present invention, the detection module includes:
[0241] A vehicle body position acquisition submodule, used for acquiring the position of the vehicle-related component based on the coordinate information of the feature point in the initial coordinate system;
[0242] The moving submodule is used to control the component detection equipment in the vehicle production workshop to move to the vicinity of the position of the vehicle-related component to detect whether the vehicle-related component is qualified.
[0243] In an optional embodiment of the present invention, the vehicle-related components include a material frame; the material frame is located in a storage location of the vehicle production workshop; the material frame includes at least one card slot and at least one limit structure; the vehicle production detection system is used to detect whether the card slot and the limit structure of the material frame can enable the parts of the vehicle to be placed in the material frame; the detection module includes:
[0244] A storage location coordinate system acquisition submodule is used to acquire the storage location coordinate system of the storage location; the storage location coordinate system is a coordinate system with any point of the storage location as the origin;
[0245] A slot position parameter acquisition submodule, used for acquiring a slot position parameter of the at least one slot in the storage location coordinate system based on the coordinate information of the feature point in the initial coordinate system;
[0246] A fitting submodule, used to fit the slot position parameters using a preset least squares method to obtain a target fitting model;
[0247] A predicted position parameter determination submodule, used to determine the predicted position parameters of the card slot based on the target fitting model;
[0248] a target card slot determination submodule, configured to determine a target card slot from the at least one card slot according to a difference between the card slot position parameter and the predicted position parameter;
[0249] The material frame coordinate system establishment submodule is used to establish the material frame coordinate system based on the slot position parameters of the target slot.
[0250] In an optional embodiment of the present invention, the detection module includes:
[0251] A part coordinate system acquisition submodule is used to acquire a part coordinate system of the assumed part in the material frame, assuming that the material frame is qualified and the preset assumed part is placed in the material frame; the part coordinate system is a coordinate system with any point of the assumed part as the origin;
[0252] A part coordinate system adjustment submodule, used for adjusting the part coordinate system so that the part coordinate system coincides with the material frame coordinate system;
[0253] The card slot to be processed is used as a submodule, and is used to use the remaining card slots in the at least one card slot except the target card slot as the card slot to be processed;
[0254] A part coordinate parameter acquisition submodule, used to acquire part coordinate parameters of the assumed part in the material frame coordinate system, slot coordinate parameters of the to-be-processed slot in the material frame coordinate system, and position-limiting structure coordinate parameters of the position-limiting structure in the material frame coordinate system;
[0255] A judgment submodule, configured to judge whether the slot and the limiting structure can enable the assumed part to be placed in the material frame based on the part coordinate parameters, the slot coordinate parameters of the slot to be processed, and the limiting structure coordinate parameters;
[0256] The qualified confirmation submodule is used to confirm that the material frame is qualified if the card slot and the limiting structure can allow the assumed part to be placed in the material frame.
[0257] In an optional embodiment of the present invention, the device comprises:
[0258] A position information acquisition module, configured to acquire position information of the material frame in the storage location based on coordinate information of the feature point in the initial coordinate system if the material frame is qualified;
[0259] A placement position determination module, used to determine the placement position of the part based on the position information and the slot position parameters;
[0260] A placement module is used to control the robot in the vehicle production workshop to place the parts at the placement position.
[0261] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.
[0262] In addition, an embodiment of the present invention further provides an electronic device, such as Fig.15 As shown, it includes a processor 1501, a communication interface 1502, a memory 1503 and a communication bus 1504, wherein the processor 1501, the communication interface 1502, and the memory 1503 communicate with each other through the communication bus 1504.
[0263] Memory 1503, used for storing computer programs;
[0264] The processor 1501 is used to implement the following steps when executing the program stored in the memory 1503:
[0265] Based on the type of the vehicle-related component, acquiring an initial shooting position and at least one displaced shooting position of the vehicle-related component;
[0266] Controlling the camera in the vehicle production workshop to move to the initial shooting position and take a picture to obtain an initial shooting image;
[0267] If the initial captured image does not meet the preset shooting requirements, the camera is controlled to move to any of the displaced shooting positions and take a picture to obtain a displaced captured image;
[0268] If the displaced captured image meets the preset shooting requirements, the displaced captured image is used as the target captured image, and an initial coordinate system and a displaced coordinate system of the camera are established; the initial coordinate system is a coordinate system with the initial shooting position as the origin; the displaced coordinate system is a coordinate system with the displaced shooting position corresponding to the target captured image as the origin;
[0269] Based on the target captured image, the initial coordinate system and the displaced coordinate system, the spatial position of at least one feature point on the vehicle-related component is acquired; the feature point is a point on the vehicle-related component with visual features;
[0270] Based on the spatial position of the feature point, it is detected whether the vehicle-related component is qualified.
[0271] In an optional embodiment of the present invention, if the initial captured image does not meet the preset shooting requirements, the camera is controlled to move to any of the displaced shooting positions and take a picture, and after the step of obtaining the displaced captured image, the method further comprises:
[0272] If the displaced photographed image meets the preset photographing requirement, stopping moving the camera;
[0273] If the displaced image does not meet the preset shooting requirements, the steps of controlling the camera to move to any of the displaced shooting positions and taking pictures to obtain the displaced image are repeated until a displaced image meeting the preset shooting requirements is obtained.
[0274] In an optional embodiment of the present invention, the step of acquiring the spatial position of at least one feature point on the vehicle-related component based on the target captured image, the initial coordinate system and the displaced coordinate system includes:
[0275] Based on the target captured image, obtaining coordinate information of the feature point on the vehicle-related component in the displacement coordinate system;
[0276] Based on the coordinate information of the feature point in the displaced coordinate system, the coordinate information of the feature point in the initial coordinate system is acquired.
[0277] In an optional embodiment of the present invention, the step of detecting whether the vehicle-related components are qualified based on the spatial position of the feature point includes:
[0278] Based on the coordinate information of the feature point in the initial coordinate system, obtaining the position of the vehicle-related component;
[0279] The component inspection equipment in the vehicle production workshop is controlled to move to the vicinity of the position of the vehicle-related component to inspect whether the vehicle-related component is qualified.
[0280] In an optional embodiment of the present invention, the vehicle-related component includes a material frame; the material frame is located in a storage location of the vehicle production workshop; the material frame includes at least one card slot and at least one limit structure; the vehicle production detection system is used to detect whether the card slot and the limit structure of the material frame can enable the parts of the vehicle to be placed in the material frame; the step of detecting whether the vehicle-related component is qualified based on the spatial position of the feature point includes:
[0281] Obtaining a storage location coordinate system of the storage location; the storage location coordinate system is a coordinate system with any point of the storage location as the origin;
[0282] Based on the coordinate information of the feature point in the initial coordinate system, obtaining the slot position parameters of the at least one slot in the storage location coordinate system;
[0283] Using a preset least square method to fit the slot position parameters to obtain a target fitting model;
[0284] Determining predicted position parameters of the card slot based on the target fitting model;
[0285] determining a target card slot from the at least one card slot according to a difference between the card slot position parameter and the predicted position parameter;
[0286] A material frame coordinate system is established based on the slot position parameters of the target slot.
[0287] In an optional embodiment of the present invention, the step of detecting whether the vehicle-related components are qualified based on the spatial position of the feature point includes:
[0288] Under the assumption that the material frame is qualified and the preset assumed part is placed in the material frame, a part coordinate system of the assumed part in the material frame is obtained; the part coordinate system is a coordinate system with any point of the assumed part as the origin;
[0289] Adjusting the part coordinate system so that the part coordinate system coincides with the material frame coordinate system;
[0290] Using the remaining card slots of the at least one card slot except the target card slot as the card slots to be processed;
[0291] Acquire part coordinate parameters of the assumed part in the material frame coordinate system, slot coordinate parameters of the to-be-processed slot in the material frame coordinate system, and position-limiting structure coordinate parameters of the position-limiting structure in the material frame coordinate system;
[0292] Based on the part coordinate parameters, the slot coordinate parameters of the slot to be processed and the limit structure coordinate parameters, determining whether the slot and the limit structure can enable the assumed part to be placed in the material frame;
[0293] If the slot and the limiting structure enable the assumed part to be placed in the material frame, the material frame is confirmed to be qualified.
[0294] In an optional embodiment of the present invention, the method includes:
[0295] If the material frame is qualified, obtaining the position information of the material frame in the storage location based on the coordinate information of the feature point in the initial coordinate system;
[0296] Based on the position information and the slot position parameters, determining the placement position of the part;
[0297] Controlling a robot in the vehicle production workshop to place the part at the placement position.
[0298] The communication bus mentioned in the above terminal can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.
[0299] The communication interface is used for communication between the above terminal and other devices.
[0300] The memory may include a random access memory (RAM) or a non-volatile memory, such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.
[0301] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.
[0302] like Fig.16 As shown, in another embodiment provided by the present invention, a computer-readable storage medium 1601 is also provided, in which instructions are stored. When the computer-readable storage medium is run on a computer, the computer executes a vehicle-related component detection method described in the above embodiment.
[0303] In another embodiment of the present invention, a computer program product including instructions is provided. When the computer program product is run on a computer, the computer executes a method for detecting vehicle-related components described in the above embodiment.
[0304] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented by software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk Solid State Disk (SSD)), etc.
[0305] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.
[0306] Each embodiment in this specification is described in a related manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.
[0307] The above description is only a preferred embodiment of the present invention and is not intended to limit the protection scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention are included in the protection scope of the present invention.
Claims
1. A method for detecting vehicle-related components, characterized in that: Applied to a vehicle production detection system, the vehicle production detection system is used to detect vehicle-related components in a preset vehicle production workshop, and the method includes: Based on the type of the vehicle-related component, acquiring an initial shooting position and at least one displaced shooting position of the vehicle-related component; Controlling the camera in the vehicle production workshop to move to the initial shooting position and take a picture to obtain an initial shooting image; If the initial captured image does not meet the preset shooting requirements, the camera is controlled to move to any of the displaced shooting positions and take a picture to obtain a displaced captured image; If the displaced captured image meets the preset shooting requirements, the displaced captured image is used as the target captured image, and an initial coordinate system and a displaced coordinate system of the camera are established; the initial coordinate system is a coordinate system with the initial shooting position as the origin; the displaced coordinate system is a coordinate system with the displaced shooting position corresponding to the target captured image as the origin; Based on the target captured image, the initial coordinate system and the displaced coordinate system, the spatial position of at least one feature point on the vehicle-related component is acquired; the feature point is a point on the vehicle-related component with visual features; Based on the spatial position of the feature point, it is detected whether the vehicle-related component is qualified.
2. The method according to claim 1, characterized in that If the initial captured image does not meet the preset shooting requirements, the camera is controlled to move to any of the displaced shooting positions and take a picture, and after the step of obtaining the displaced captured image, the method further comprises: If the displaced photographed image meets the preset photographing requirement, stopping moving the camera; If the displaced image does not meet the preset shooting requirements, the steps of controlling the camera to move to any of the displaced shooting positions and taking pictures to obtain the displaced image are repeated until a displaced image meeting the preset shooting requirements is obtained.
3. The method according to claim 1, characterized in that The step of acquiring the spatial position of at least one feature point on the vehicle-related component based on the target captured image, the initial coordinate system and the displaced coordinate system comprises: Based on the target captured image, obtaining coordinate information of the feature point on the vehicle-related component in the displacement coordinate system; Based on the coordinate information of the feature point in the displaced coordinate system, the coordinate information of the feature point in the initial coordinate system is acquired.
4. The method according to claim 3, characterized in that The step of detecting whether the vehicle-related components are qualified based on the spatial position of the feature point comprises: Based on the coordinate information of the feature point in the initial coordinate system, obtaining the position of the vehicle-related component; The component inspection equipment in the vehicle production workshop is controlled to move to the vicinity of the position of the vehicle-related component to inspect whether the vehicle-related component is qualified.
5. The method according to claim 3, characterized in that: The vehicle-related components include a material frame; the material frame is located in a storage location of the vehicle production workshop; the material frame includes at least one card slot and at least one limit structure; the vehicle production detection system is used to detect whether the card slot and the limit structure of the material frame can enable the parts of the vehicle to be placed in the material frame; The step of detecting whether the vehicle-related components are qualified based on the spatial position of the feature point comprises: Obtaining a storage location coordinate system of the storage location; the storage location coordinate system is a coordinate system with any point of the storage location as the origin; Based on the coordinate information of the feature point in the initial coordinate system, obtaining the slot position parameters of the at least one slot in the storage location coordinate system; Using a preset least square method to fit the slot position parameters to obtain a target fitting model; Determining predicted position parameters of the card slot based on the target fitting model; determining a target card slot from the at least one card slot according to a difference between the card slot position parameter and the predicted position parameter; A material frame coordinate system is established based on the slot position parameters of the target slot.
6. The method according to claim 5, characterized in that The step of detecting whether the vehicle-related components are qualified based on the spatial position of the feature point comprises: Under the assumption that the material frame is qualified and the preset assumed part is placed in the material frame, a part coordinate system of the assumed part in the material frame is obtained; the part coordinate system is a coordinate system with any point of the assumed part as the origin; Adjusting the part coordinate system so that the part coordinate system coincides with the material frame coordinate system; Using the remaining card slots of the at least one card slot except the target card slot as the card slots to be processed; Acquire part coordinate parameters of the assumed part in the material frame coordinate system, slot coordinate parameters of the to-be-processed slot in the material frame coordinate system, and position-limiting structure coordinate parameters of the position-limiting structure in the material frame coordinate system; Based on the part coordinate parameters, the slot coordinate parameters of the slot to be processed and the limit structure coordinate parameters, determining whether the slot and the limit structure can enable the assumed part to be placed in the material frame; If the slot and the limiting structure enable the assumed part to be placed in the material frame, the material frame is confirmed to be qualified.
7. The method according to claim 6, characterized in that The method comprises: If the material frame is qualified, obtaining the position information of the material frame in the storage location based on the coordinate information of the feature point in the initial coordinate system; Based on the position information and the slot position parameters, determining the placement position of the part; Controlling a robot in the vehicle production workshop to place the part at the placement position.
8. A detection device for vehicle-related parts, characterized in that: Applied to a vehicle production detection system, the vehicle production detection system is used to detect vehicle-related components in a preset vehicle production workshop, and the device includes: A shooting position acquisition module, used for acquiring an initial shooting position and at least one displaced shooting position of the vehicle-related component based on the type of the vehicle-related component; A first moving module is used to control the camera in the vehicle production workshop to move to the initial shooting position and take pictures to obtain an initial shooting image; A second moving module is used to control the camera to move to any of the displaced shooting positions and take pictures to obtain a displaced shooting image if the initial shot image does not meet the preset shooting requirements; A coordinate system establishment module, for taking the displaced photographed image as a target photographed image if the displaced photographed image meets the preset photographing requirement, and establishing an initial coordinate system and a displaced coordinate system of the camera; the initial coordinate system is a coordinate system with the initial photographing position as the origin; the displaced coordinate system is a coordinate system with the displaced photographing position corresponding to the target photographed image as the origin; A spatial position acquisition module, used to acquire the spatial position of at least one feature point on the vehicle-related component based on the target captured image, the initial coordinate system and the displaced coordinate system; the feature point is a point on the vehicle-related component with visual features; The detection module is used to detect whether the vehicle-related components are qualified based on the spatial positions of the feature points.
9. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; The memory is used to store computer programs; The processor is used to implement the method according to any one of claims 1 to 7 when executing the program stored in the memory.
10. One or more computer-readable media having instructions stored thereon, which when executed by one or more processors cause the processors to perform the method according to any one of claims 1 to 7.
Citation Information
Patent Citations
Automatic shooting control method and device as well as computer readable storage medium
CN109688305A
Automobile chassis detection method and readable storage medium
CN116071358A
Defect detection method, device and equipment and storage medium
CN118111915A
Position detector
JP1999051613A
Detection range adjustment device
JP2006304342A
Cited By
Automobile part defect detection method and system
CN119722678A
