Automatic ball machine pose adjusting method of excavator intelligent quality inspection system

Abstract

The invention belongs to the technical field of excavators, and particularly relates to a method for automatically adjusting the position and posture of a dome camera of an intelligent quality inspection system of an excavator, and the method comprises the following steps: 1, collecting an image; shooting images of the excavator by using the maximum field angle of the camera so as to shoot the complete excavator; 2, detecting the excavator by adopting a YOLOv8 algorithm to obtain an excavator target frame; 3, the positions and size information, relative to the excavator target frame, of various to-be-detected targets of the excavator are collected, and the information can be exported from the excavator 3D model diagram according to the positions, relative to the excavator target frame, of the to-be-detected targets. The automatic dome camera pose adjusting method of the excavator intelligent quality inspection system has the advantages that the pose of the dome camera can be automatically adjusted during detection each time so as to ensure that the target can be accurately shot, the proportion of the target in the image meets a certain requirement, and the effect of subsequent algorithm recognition is facilitated.

Description

Technical Field

[0001] The invention belongs to the technical field of excavators, and in particular relates to a method for automatically adjusting the position and posture of a ball camera of an intelligent quality inspection system of an excavator. Background Art

[0002] like Figures 7 and 8 As shown, during the factory quality inspection of the excavator, it is necessary to check whether there are any missing or wrongly installed configuration items of the vehicle, such as whether the labels on the exterior are missing, whether the rearview mirrors, wipers, work lights, etc. are missing or wrongly installed.

[0003] In the intelligent quality inspection system for excavators based on computer vision technology, a ball camera equipped with an omnidirectional pan / tilt camera is used to take photos of the vehicle, and then a deep learning algorithm is used to identify configuration items, such as Figure 2 As shown, the excavator that needs to be inspected is parked in the inspection area, and the ball camera is deployed on the brackets on both sides of the excavator. The ball camera is used to automatically take pictures and identify the target to be inspected.

[0004] The configuration items that require quality inspection for excavators usually vary greatly in size, for example Figure 1 In the example, there are large size differences in the appearance trademarks on the boom, engine hood, and chassis, and the target detection model based on deep learning usually has difficulty in simultaneously identifying targets of various sizes. Therefore, the method of taking a single picture with a ball camera and then using a deep target detection model to identify all targets has poor detection effect on small targets.

[0005] The excavator is manually driven when docking in the inspection area, and there is a certain docking error. Therefore, in the excavator intelligent quality inspection system, the dome camera preset position method cannot be used to capture high-resolution images of each target separately. Summary of the invention

[0006] The purpose of the present invention is to provide a method for automatically adjusting the posture of a ball camera in an intelligent quality inspection system for an excavator, so as to solve the technical problem that the ball camera pre-position method cannot be used to capture high-resolution images of each target separately in the intelligent quality inspection system for the excavator, so as to achieve the goal of automatically adjusting the posture of the ball camera in each inspection to ensure that the target can be accurately photographed and the proportion of the target in the image meets certain requirements, so as to facilitate the purpose of subsequent algorithm recognition.

[0007] In order to solve the above technical problems, the present invention provides a method for automatically adjusting the posture of a ball camera in an intelligent quality inspection system for an excavator, comprising the following steps:

[0008] Step 1: Capture images; use the camera's maximum field of view to capture the excavator image so as to capture the complete excavator. When installing the camera, i.e., the ball camera, the installation position needs to be adjusted to ensure that at 0 magnification, the excavator can still capture the complete excavator under the maximum docking error.

[0009] Step 2: Use the YOLOv8 algorithm to detect the excavator and obtain the excavator target frame;

[0010] Step 3: Collect the position and size information of various objects to be detected of the excavator relative to the excavator target frame. This information can be derived from the excavator 3D model diagram. According to the position of the object to be detected relative to the excavator target frame, the position frame of the object to be detected can be obtained;

[0011] Step 4: Automatically calculate the position information of the ball camera for the excavator target frame;

[0012] Furthermore, in step 4, the calculation method is: Assuming that the pixel width and height of the collected image are P W , P H , the pixel width and height of the target box are P y , P x , the physical width and height of the target box are D y , D x ;

[0013] Step 1: Calculate the horizontal and pitch rotation angles of the ball camera pan / tilt so that the target frame is located at the center of the captured image after the pan / tilt rotates;

[0014] Step 2: Set the pixel length of the horizontal field of view of the corresponding camera when shooting the entire image of the excavator at 0 magnification to P Q , the pixel length of the target to be detected is P y , the camera’s field of view is 2θ. The line connecting the camera and the black dot is the perpendicular bisector L, and the pixel distance from the center of the target to be detected (i.e., the red dot) to the center of the horizontal field of view (i.e., the black dot) is P c ;

[0015] Step 3: When the center point of the target to be detected is moved to the position of the center point of the horizontal field of view, the target to be detected will be located at the center of the picture, and the corresponding angle is β;

[0016] Assume that the pixel length corresponding to the perpendicular bisector L is P L , then the relevant theorems and calculation formulas of trigonometric functions are:

[0017]

[0018] To calculate the horizontal rotation angle β of the ball camera, the calculation method of the vertical rotation angle is the same as that of the horizontal direction;

[0019] After adjusting the ball camera pan / tilt according to the calculated angle, wait for the 0 detection target to move to the center of the image;

[0020] Since the pixel length of the target box is P y , the physical length is Dy Then the physical length represented by a unit pixel is D y / P y Then, based on the pixel length P of the perpendicular bisector L K the physical length D of the perpendicular bisector L can be obtained L = D y ×P L / P y ;

[0021] Step 4: Adjust the camera magnification so that the pixel size of each target to be detected is consistent;

[0022] Step 5: Set the target to be detected after the camera magnification is adjusted, and its pixel length is P′ y ;

[0023] Step 6: Set the field of view angle after the camera magnification is adjusted to 2θ, and the pixel length corresponding to the perpendicular bisector L is P′ L and the physical length D of the perpendicular bisector L has been calculated in the previous step L From the proportional relationship P′ L / P′ y = D L / D y P can be calculated L = P y ×D L / D y Then, from tanθ′ = P W / (2P L ′), θ′ = tan -1 (P y ×tanθ÷0..8÷P W );

[0024] After adjusting the field of view angle according to 2θ′, an image in which the target length accounts for about 80% in the image can be obtained, and then it is sent to the subsequent target classification or detection network for target recognition.

[0025] 3. The automatic adjustment method for the spherical camera pose of an excavator intelligent quality inspection system according to claim 2, characterized in that

[0026] In step 4, the pose information includes the horizontal and pitch rotation angles of the pan-tilt and the camera magnification value.

[0027] The beneficial effects of the present invention are:

[0028] 1. The present invention proposes a method for automatically adjusting the posture of a ball camera, which can automatically adjust the posture of the ball camera (including the horizontal and pitch rotation angles of the pan / tilt and the camera magnification) during each detection to ensure that the target can be accurately photographed and the proportion of the target in the image (i.e., the target resolution) meets certain requirements to facilitate subsequent algorithm recognition.

[0029] 2. When there is an error in the docking of the excavator, the pan / tilt adjustment angle and camera magnification of the ball camera can be automatically calculated to obtain an image with the target in the center of the picture and a uniform proportion in the image, providing a good basis for subsequent target classification or detection.

[0030] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0032] Figure 1 It is a flowchart of a method for automatically adjusting the position and posture of a ball camera in an intelligent quality inspection system for an excavator of the present invention;

[0033] Figure 2 It is a schematic diagram of the collected images of the method for automatically adjusting the posture of a ball camera of the intelligent quality inspection system for excavators of the present invention;

[0034] Figure 3 It is a schematic diagram of a target frame to be detected in the method for automatically adjusting the position and posture of a ball camera in the intelligent quality inspection system for excavators of the present invention;

[0035] Figure 4 It is a schematic diagram of setting data of a method for automatically adjusting the position and posture of a ball camera in an intelligent quality inspection system for an excavator according to the present invention;

[0036] Figure 5 It is a schematic diagram of data detected by the method for automatically adjusting the position and posture of a ball camera of the intelligent quality inspection system for excavators of the present invention;

[0037] Figure 6 It is a schematic diagram of data detected by the method for automatically adjusting the position and posture of a ball camera of the intelligent quality inspection system for excavators of the present invention;

[0038] Figure 7 It is a structural schematic diagram of the prior art of the present invention;

[0039] Figure 8 It is a structural schematic diagram of the prior art of the present invention. DETAILED DESCRIPTION

[0040] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0041] Example:

[0042] like Figures 1 to 6 As shown, a method for automatically adjusting the posture of a ball camera in an intelligent quality inspection system for an excavator comprises the following steps: Step 1: acquiring an image; capturing an image of the excavator with the maximum field of view of the camera to capture the complete excavator, and adjusting the installation position of the camera, i.e., the ball camera, during installation to ensure that at 0 magnification, the complete excavator can still be captured with the maximum docking error.

[0043] Step 2: Use the YOLOv8 algorithm to detect the excavator and obtain the excavator target frame; Figure 2 shown.

[0044] Step 3: Collect the position and size information of various objects to be detected of the excavator relative to the excavator target frame. This information can be derived from the excavator 3D model diagram. According to the position of the object to be detected relative to the excavator target frame, the position frame of the object to be detected can be obtained; Figure 3 shown.

[0045] Step 4: Automatically calculate the posture information of the ball camera for the excavator target frame; the posture information includes the horizontal and pitch rotation angles of the gimbal, and the camera magnification value.

[0046] In step 4, the calculation method is: Assume that the pixel width and height of the acquired image are P W , P H , the pixel width and height of the target box are P y , P x , the physical width and height of the target box are D y , D x ;like Figure 4 shown.

[0047] Step 1: Calculate the horizontal and pitch rotation angles of the ball camera pan / tilt so that the target frame is located at the center of the captured image after the pan / tilt rotates. Because the calculation principles of the horizontal and pitch rotation angles are the same, the horizontal direction is used as an example for explanation.

[0048] Step 2: Set the pixel length of the horizontal field of view of the corresponding camera when shooting the entire image of the excavator at 0 magnification to P W , the pixel length of the target to be detected is P y , the camera’s field of view is 2θ. The line connecting the camera and the black dot is the perpendicular bisector L, and the pixel distance from the center of the target to be detected (i.e. the bold dot on the right) to the center of the horizontal field of view (i.e. the bold dot in the center) is P c ;like Figure 5 shown.

[0049] Step 3: When the center point of the target to be detected is moved to the position of the center point of the horizontal field of view, the target to be detected will be located at the center of the picture, and the corresponding angle is β;

[0050] Assume that the pixel length corresponding to the perpendicular bisector L is P L , then the relevant theorems and calculation formulas of trigonometric functions are:

[0051]

[0052] To calculate the horizontal rotation angle β of the ball camera, the vertical rotation angle calculation method is the same as the horizontal direction. After adjusting the ball camera pan / tilt according to the calculated angle, wait for the 0 detection target to move to the center of the image; since the pixel length of the target frame is P y , the physical length is D y , then the physical length represented by a unit pixel is D y / P y , and then according to the pixel length P of the perpendicular line L L , we can get the physical length D of the perpendicular bisector L L =D y ×P L / P y ;

[0053] Step 4: Adjust the camera magnification so that the pixel size of each target to be detected remains consistent. Since the pixel sizes of targets to be detected of different sizes vary greatly in the image, and the pixel ratio of some smaller targets to be detected is too small, in order to improve the accuracy of subsequent target recognition, the camera magnification is adjusted here so that the pixel size of each target to be detected remains consistent.

[0054] Step 5: If Figure 6 As shown in the figure, the pixel length of the target to be detected after the camera magnification is adjusted is P′ y In this embodiment, the resolution of the captured image remains unchanged before and after the camera magnification is adjusted, that is, the pixel width P of the captured image W and height P H The present invention sets P' y =P W×0.8.

[0055] Step 6: Set the field of view angle after the camera magnification is adjusted to 2θ′, and the pixel length corresponding to the perpendicular bisector L to P′ L , and the physical length D of the perpendicular bisector L has been calculated in the previous step L , according to the proportional relationship P′ L / P′ y =D L / D y , P′ can be calculated L =P′ y ×D L / D y , and then by tanθ′=P W / (2P L ′), we can get θ′=tan -1 (P y ×tanθ÷0..8÷P W );

[0056] After adjusting the field of view angle according to 2θ′, an image in which the target length accounts for about 80% of the image can be obtained, which is then sent to the subsequent target classification or detection network for target recognition.

[0057] In summary: The present invention proposes a method for automatically adjusting the posture of a ball camera, which can automatically adjust the posture of the ball camera (including the horizontal and pitch rotation angles of the pan / tilt and the camera magnification) during each detection to ensure that the target can be accurately photographed and the proportion of the target in the image (i.e., the target resolution) meets certain requirements for subsequent algorithm recognition. In the case of errors in the docking of the excavator, the pan / tilt adjustment angle and camera magnification of the ball camera can be automatically calculated to obtain an image located in the center of the picture with a uniform proportion of the target in the image, providing a good basis for subsequent target classification or detection.

[0058] The various devices selected in this application are all universal standard parts or parts known to those skilled in the art, and their structures and principles can be known to those skilled in the art through technical manuals or conventional experimental methods.

[0059] In the description of the embodiments of the present invention, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0060] In the description of the present invention, it should be noted that the orientation or positional relationship indicated by the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc. is based on the orientation or positional relationship shown in the drawings. It is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the present invention. In addition, the terms "first", "second", "third" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance.

[0061] Based on the above inspiration from the ideal embodiments of the present invention, through the above description, relevant staff can completely make various changes and modifications without departing from the technical idea of the present invention. The technical scope of the present invention is not limited to the content in the specification, and its technical scope must be determined according to the scope of the claims.

Claims

1. A method for automatically adjusting the position and posture of a ball camera in an intelligent quality inspection system for an excavator, characterized in that: The steps include: Step 1: Capture an image; capture the excavator image with the maximum field of view of the camera to capture the complete excavator; Step 2: Use the YOLOv8 algorithm to detect the excavator and obtain the excavator target frame; Step 3: Collect the position and size information of various objects to be detected of the excavator relative to the excavator target frame. This information can be derived from the excavator 3D model diagram. According to the position of the object to be detected relative to the excavator target frame, the position frame of the object to be detected can be obtained; Step 4: Automatically calculate the position information of the ball camera for the excavator target frame.

2. The method for automatically adjusting the position and posture of a ball camera in an intelligent quality inspection system for an excavator according to claim 1, characterized in that: In step 4, the calculation method is: Assume that the pixel width and height of the acquired image are P W , P H , the pixel width and height of the target box are P y , P x , the physical width and height of the target box are D y , D x ; Step 1: Calculate the horizontal and pitch rotation angles of the ball camera pan / tilt so that the target frame is located at the center of the captured image after the pan / tilt rotates; Step 2: Set the pixel length of the horizontal field of view of the corresponding camera when shooting the entire image of the excavator at 0 magnification to P W , the pixel length of the target to be detected is P y , the camera’s field of view is 2θ. The line connecting the camera and the black dot is the perpendicular bisector L, and the pixel distance from the center of the target to be detected to the center of the horizontal field of view is P c ; Step 3: When the center point of the target to be detected is moved to the position of the center point of the horizontal field of view, the target to be detected will be located at the center of the picture, and the corresponding angle is β; Assume that the pixel length corresponding to the perpendicular bisector L is P L , then the relevant theorems and calculation formulas of trigonometric functions are: To calculate the horizontal rotation angle β of the ball camera, the calculation method of the vertical rotation angle is the same as that of the horizontal direction; After adjusting the ball camera pan / tilt according to the calculated angle, wait for the 0 detection target to move to the center of the image; Since the pixel length of the target box is P y , the physical length is D y , then the physical length represented by a unit pixel is D y / P y , and then according to the pixel length P of the perpendicular line L L , we can get the physical length D of the perpendicular bisector L L =D y ×P L / P y ; Step 4: Adjust the camera magnification so that the pixel size of each target to be detected remains consistent; Step 5: Set the target to be detected after adjusting the camera magnification. ′ Its pixel length is P′ y ; Step 6: Set the field of view angle after the camera magnification is adjusted to 2θ, and the pixel length corresponding to the perpendicular bisector L to P′ L , and the physical length D of the perpendicular bisector L has been calculated in the previous step L , according to the proportional relationship, P′ L / P′ y =D L / D y , P′ can be calculated L =P′ y ×D L / D y , and then by tanθ′=P W / (2P′ L ), we can get θ′=tan -1 (P y ×tanθ÷0.8÷P W ); After adjusting the field of view angle according to 2θ′, an image in which the target length accounts for about 80% of the image can be obtained, which is then sent to the subsequent target classification or detection network for target recognition.

3. The method for automatically adjusting the position and posture of a ball camera in an intelligent quality inspection system for an excavator as claimed in claim 2, characterized in that: In step 4, the pose information includes the gimbal horizontal and pitch rotation angles, and the camera magnification value.

Images