Steering control image recognition method, device and excavator
Through the image recognition method of integrating straight lines and arcs, the riding and walking control of the wheeled excavator remote control system is simplified, the problems of high development difficulty and cost are solved, and the real-time and accuracy of direction control are improved.
Patent Information
- Application Number
- CN202311481324.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-11-08
- Publication Date
- 2024-11-05
- Estimated Expiration
- 2043-11-08
AI Technical Summary
The existing wheeled excavator remote control system has problems such as difficult development, high cost and high structural complexity in the control of getting off the vehicle, resulting in high failure rate and affecting operating accuracy and reliability.
An image recognition method that integrates straight lines and arcs is adopted to initialize and compare the contour features of the remote control steering wheel to generate real-time features, randomly search for fitting combinations, judge the rotation angle, simplify positioning difficulty, and improve real-timeness.
It reduces development difficulty and cost, reduces structural complexity, improves the real-time and accuracy of direction control, and can accurately identify the position and status of the remote control steering wheel, adapt to different environments.
Smart Images

Figure CN117635984B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of engineering machinery, and in particular to an excavator steering control image recognition method, device and excavator. Background Art
[0002] Remote control operation of excavators is an important area of engineering machinery automation at present. For remote control operation of excavators, remote control driving technology plays a key role. Especially for wheeled excavators, their autonomous driving technology is similar to that of automobiles and faces many technical challenges.
[0003] At present, some technical enterprises have been developing the remote control system of wheeled excavators in terms of the working device on the vehicle, but there are some challenges in the walking control of getting off the vehicle, which are mainly reflected in the following problems:
[0004] 1. Difficulty in development and high cost: Traditional remote control systems require the installation of high-cost, high-reliability absolute value encoding sensors on the excavator to measure the steering angle of the steering rod. These sensors are not only expensive, but also need to be precisely installed coaxially with the steering system, which increases the technical difficulty and cost of installation.
[0005] 2. There are many failure points: Due to the need to install a coaxial sensor, the system structure becomes complicated and prone to failure. After long-term use, the rotation of the steering wheel may cause relative sliding between the sensor and the remote control steering wheel shaft, thus affecting the accuracy and reliability of the operation.
[0006] Therefore, there is an urgent need for an excavator steering control image recognition method that can reduce development difficulty, switch cost, and reduce structural complexity. Summary of the invention
[0007] Purpose of the invention: In order to overcome the above shortcomings, the purpose of the present invention is to provide an excavator steering control image recognition method, which identifies and locates the remote control steering wheel by integrating straight lines and arcs, extracts the most effective features, reduces the complexity of recognition, and utilizes similar transmission relationships to simplify the difficulty of positioning, speed up the speed of feature positioning, thereby improving the real-time performance of direction control.
[0008] To solve the above technical problems, the present invention provides an excavator steering control image recognition method, comprising the following steps: according to the collected remote control steering wheel image, initializing the contour recognition of the remote control steering wheel to form a preset contour feature; according to the remote control steering wheel image, generating the real-time contour feature of the remote control steering wheel, and then comparing the real-time contour feature with the preset contour feature to determine whether the image verification is passed; if so, generating an effective search range of the remote control steering wheel image, and shielding all invalid range areas, if not, returning to regenerate the real-time contour feature of the remote control steering wheel; according to the real-time contour feature within the effective search range, randomly searching for a group of symmetrical contour line fitting combinations, and then judging the real-time rotation angle of the excavator remote control steering wheel according to the symmetrical contour line fitting combination.
[0009] By adopting the above technical scheme, the remote control steering wheel can be identified and located by integrating straight lines and arcs, and the most effective features can be extracted from them. This comprehensive analysis reduces the complexity of the recognition process and determines the position and contour of the remote control steering wheel with higher accuracy. At the same time, by utilizing the similar transmission relationship, the difficulty of positioning is simplified, which means that it is easier to establish an accurate correspondence between the remote control steering wheel and the image, further improving the speed of feature positioning. In this way, the real-time tracking and identification of the position and state of the remote control steering wheel can be achieved more quickly. The application of this technical scheme can significantly improve the real-time performance of directional control.
[0010] Preferably, when forming the preset contour feature, the method further comprises the following steps: collecting a remote control steering wheel image, and then generating a remote control steering wheel contour by analyzing the combined features of the remote control steering wheel arc and straight line; and calculating the arc curvature of the pixel point according to each non-zero pixel point of the remote control steering wheel contour:
[0011] ,
[0012] in, is the arc curvature at the pixel point, is the first-order derivative at the pixel point, is the second-order derivative at the pixel point; determine whether there is a pixel point whose arc curvature is greater than the preset curvature, if so, set the matching pixel point as the inflection point, if not, return to recalculate the arc curvature of the pixel point.
[0013] Preferably, after generating the inflection point, the method further comprises the following steps: calculating a scale parameter of the inflection point edge profile:
[0014] ,
[0015] in, is the center angle of a circle; according to the ratio parameter, determine whether the inflection point edge contour is a reasonable arc; if so, arrange all the arcs in a preset order, if not, identify the inflection point edge contour as a straight line; according to the arranged arcs, reorganize the arcs belonging to the same circle to form a preset contour feature.
[0016] Preferably, when reorganizing arcs belonging to the same circle, the method also includes the following steps: comparing the arranged arcs in sequence according to a preset comparison order to determine whether there are arcs belonging to the same circle; if so, reorganizing the arcs belonging to the same circle, recalculating the center and radius, and then comparing the reorganized arcs in sequence according to a preset comparison order; if not, returning to recalculate the arc curvature of the pixel point; repeating the above steps until the final arc and circle fitting results are obtained, and the arc reorganization is completed.
[0017] Preferably, after obtaining the final arc and circle fitting results, the method further includes the following steps: based on the circle fitting results, judging whether the final arc occupies at least half of the circumference of the circle, and the circle fitting error is less than a preset range; if so, it is determined that a circle is obtained, if not, judging the arc of the next length.
[0018] Preferably, after obtaining the circle, the method further comprises the following steps: according to any pixel point of the circle, calculating the image gradient direction of the pixel point:
[0019] ,
[0020] in, , , is the gradient in the horizontal direction, is the gradient in the vertical direction, is the RGB value of the image pixel, is the coordinate of the pixel; if the true direction of the pixel point is within the threshold range where the center of the circle points to the pixel point, the pixel point is judged as a meaningful point, and then a preset contour feature is formed according to the circle; if the true direction of the pixel point is not within the threshold range where the center of the circle points to the pixel point, the pixel point is judged as a meaningless point.
[0021] Preferably, when forming a preset contour feature, the method also includes the following steps: based on generating the remote control steering wheel contour, all reasonable arcs in the remote control steering wheel image are sorted according to the radius, and then whether the reasonable arc is an arc on the remote control steering wheel contour is determined in a preset order; if so, a preset square area is set with the center of the arc as the center, and then a straight line is detected in the preset square area, if not, the next arc is judged; if a preset number of straight lines are detected in the preset square area and match the preset conditions, a preset contour feature is formed.
[0022] By adopting the above technical scheme, through the above excavator steering control image recognition method, the arc on the visual feature mark can be quickly detected, which is suitable for detecting arcs in images. Even if it is a bad arc, the inner and outer edge contours of the arc on the remote control steering wheel visual feature can be accurately detected, and the probability of false detection of the arc is low. Through this image recognition method, even if part of the image is blocked, the remote control steering wheel of the excavator can be accurately detected.
[0023] Preferably, after determining the real-time rotation angle of the excavator remote control steering wheel according to the symmetrical straight line fitting combination, the method further includes the following steps: randomly searching a group of asymmetrical contour line fitting combinations according to the effective search range of the real-time contour feature, and then determining the test rotation angle of the excavator remote control steering wheel according to the asymmetrical contour line fitting combination; comparing the test rotation angle with the real-time rotation angle to determine whether the detection error is within a preset error range; if so, setting the real-time transmission angle to the actual rotation angle, if not, returning to randomly search a group of symmetrical contour line fitting combinations again.
[0024] By adopting the above technical scheme, through random searching of asymmetric contour line fitting combinations, the contour features of the remote control steering wheel can be more comprehensively considered, and the recognition accuracy of the real-time rotation angle can be further improved. At the same time, by comparing the test rotation angle and the real-time rotation angle, it is possible to detect whether there is a detection error, thereby improving fault tolerance and ensuring that even if there is an identification error, it can be corrected in a targeted manner, further improving adaptability and being able to adapt to different situations and specific requirements of the excavator.
[0025] The present invention also provides an excavator steering control image recognition device, which uses the above-mentioned excavator steering control image recognition method to perform steering control image recognition of the remote control steering wheel of the excavator. The excavator steering control image recognition device includes: a camera unit, which is used to collect images of the remote control steering wheel; an image recognition module, which is used to initialize the contour recognition of the remote control steering wheel according to the collected remote control steering wheel image to form a preset contour feature; a contour judgment module, which is used to generate a real-time contour feature of the remote control steering wheel according to the remote control steering wheel image, and then compare the real-time contour feature with the preset contour feature. After judging that the image verification is passed, an effective search range is generated, and all invalid range areas are shielded; an angle judgment module is used to randomly search a group of symmetrical straight line fitting combinations according to the effective search range of the real-time contour feature, and then judge the real-time rotation angle of the excavator remote control steering wheel according to the symmetrical straight line fitting combination.
[0026] The present invention also provides an excavator, comprising the excavator steering control image recognition device, and also comprising: a steering assembly, used to control the movement direction of the excavator; a remote control steering wheel, which is connected to the steering assembly of the excavator and is used to control the movement direction of the steering assembly; a remote control controller, which is connected to the remote control steering wheel and the excavator steering control image recognition device and is used to control the rotation angle of the steering assembly.
[0027] The above technical solution of the present invention has the following advantages compared with the prior art:
[0028] 1. Ability to locate the rotation angle of the remote control steering wheel based on monocular vision recognition, that is, to achieve high real-time positioning with a single camera;
[0029] 2. It can identify and locate the remote control steering wheel by combining straight lines and arcs, extract the most effective features, reduce the complexity of identification, use similar transmission relationships, simplify the difficulty of positioning, speed up feature positioning, and thus improve the real-time performance of direction control;
[0030] 3. It can quickly detect the arc on the visual feature mark, even if it is a bad arc, and can accurately detect the inner and outer edge contours of the arc on the remote control steering wheel visual feature, and the probability of false detection of the arc is low. Even if part of the image is blocked, the remote control steering wheel of the excavator can be accurately detected. BRIEF DESCRIPTION OF THE DRAWINGS
[0031] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.
[0032] Figure 1 is a connection relationship diagram of the excavator of the present invention;
[0033] Figure 2 is a connection relationship diagram of the excavator steering control image recognition device of the present invention;
[0034] Figure 3 is a flow chart of the excavator steering control image recognition method of the present invention;
[0035] Figure 4 is a flow chart of a method for identifying inflection points of pixels of a remote control steering wheel contour according to the present invention;
[0036] Figure 5 is a flow chart of the reasonable arc judgment method of the present invention;
[0037] Figure 6 is a schematic diagram of the tangent distance between the first and last points of the edge profile of the present invention;
[0038] Figure 7 is a flow chart of the reasonable arc reorganization method of the present invention;
[0039] Figure 8 is a flow chart of a circularity determination method of a recombined circular arc of the present invention;
[0040] Fig. 9 is a flow chart of the circle determination method of the present invention;
[0041] Fig.10 It is a schematic diagram of judging the true direction of a pixel point of the present invention;
[0042] Fig.11 is a flow chart of a method for generating preset contour features of the present invention;
[0043] Fig.12 is a schematic diagram of arcs and straight lines of a remote control steering wheel of the present invention;
[0044] Fig.13 is a schematic diagram of a preset square area of a remote control steering wheel of the present invention;
[0045] Fig.14 is a schematic diagram of a symmetrical contour line fitting combination of a remote control steering wheel of the present invention;
[0046] Fig.15 is a schematic diagram of an asymmetric contour line fitting combination of a remote control steering wheel of the present invention;
[0047] Fig.16 It is a flow chart of the rotation angle comparison and judgment method of the present invention. DETAILED DESCRIPTION
[0048] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, and should not be construed as limiting the present invention.
[0049] Remote control of steering of excavator 1 is an important technology. One aspect of the present invention is to provide an image recognition method for steering control of excavator. The steering control image recognition described herein preferably combines computer vision and remote control technology. The steering operation of excavator 1 is realized by image recognition. This technology can improve the intelligence and accuracy of operation and is applicable to various dangerous environments, remote operations or engineering tasks requiring high accuracy, such as but not limited to mining work, construction and earthwork, emergency rescue, and danger elimination. In summary, the basic principle of steering control image recognition involved is: using a camera or other sensor installed on the excavator 1 to capture the image of the surrounding environment, and then using an image recognition algorithm to process and analyze the captured image, identify the control angle value of the excavator 1, and transmit the target angle value to the remote control controller 12 for direction control of the excavator 1.
[0050] refer to Figure 1 , Figure 2 As shown, in some embodiments, the excavator 1 involved includes at least an excavator steering control image recognition device 2, a steering assembly 10, a remote control steering wheel 11 and a remote control controller 12.
[0051] Excavator steering control image recognition device 2
[0052] The excavator steering control image recognition device 2 includes a camera unit 20 , an image recognition module 21 , a contour judgment module 22 , an angle judgment module 23 and a data transceiver module 24 .
[0053] Specifically, the camera unit 20 includes at least one camera, which is installed in the excavator 1, and the camera image acquisition end faces the remote control steering wheel 11 of the excavator 1; wherein, during the operation of the excavator 1, the position distance between the camera and the remote control steering wheel 11 remains unchanged.
[0054] In the specific implementation process, the distance and angle between the camera and the remote control steering wheel 11 of the excavator 1 are set by the operator according to the actual model of the excavator 1.
[0055] Specifically, the image recognition module 21 is used to initialize the contour recognition of the remote control steering wheel 11 according to the collected image of the remote control steering wheel 11 to form a preset contour feature; the contour judgment module 22 is used to generate the real-time contour feature of the remote control steering wheel 11 according to the image of the remote control steering wheel 11, and then compare the real-time contour feature with the preset contour feature. After judging that the image verification is passed, a valid search range is generated, and all invalid range areas are shielded; the angle judgment module 23 is used to randomly search for a group of symmetrical straight line 32 fitting combinations according to the effective search range of the real-time contour feature, and then judge the real-time rotation angle of the remote control steering wheel 11 of the excavator 1 according to the symmetrical straight line 32 fitting combination; the data transceiver module 24 is used to receive and send data.
[0056] Steering assembly 10
[0057] The steering assembly 10 includes a steering motor, a slewing bearing and a transmission mechanism, which are all installed in the excavator 1. The steering operation of the excavator 1 is achieved through the cooperation of the steering motor, the slewing bearing and the transmission mechanism.
[0058] Remote control steering wheel 11
[0059] The remote control steering wheel 11 is installed in the excavator 1 and connected to the steering assembly 10 of the excavator 1 . The angle control of the steering assembly 10 is performed through the remote control steering wheel 11 .
[0060] Remote controller 12
[0061] The remote controller 12 is connected to the camera unit 20 and the steering assembly 10 respectively, and the angle control of the steering assembly 10 is performed through the remote controller 12 .
[0062] Since the remote control steering wheel 11 of the excavator 1 has a large positioning distance range, the image collected by the camera unit 20 at a non-focal distance will be blurred, which will also cause uneven exposure. Therefore, the most obvious feature of the remote control steering wheel 11 is selected as the feature for identifying and positioning the remote control steering wheel 11. Based on the above principles, reference Fig.12 As shown, the most obvious outline of the remote control steering wheel 11 is selected as the basis for identification and positioning. Through analysis, the outline of the remote control steering wheel 11 is mainly composed of an arc 31 and a straight line 32. Therefore, the characteristic information of the remote control steering wheel 11 is determined by identifying the combination of the arc 31 and the straight line 32.
[0063] Therefore, one aspect of the present invention proposes an excavator steering control image recognition method, referring to Figure 3 As shown, the specific working steps of this method are as follows:
[0064] S1: using the camera unit 20 to capture an image of the remote control steering wheel 11 , and then initializing contour recognition of the remote control steering wheel 11 according to the image of the remote control steering wheel 11 to generate a preset contour feature.
[0065] In step S1, refer to Figure 4 As shown, the specific working steps include:
[0066] S10: using the camera unit 20 to capture an image of the remote control steering wheel 11 , and then generating contour features of the remote control steering wheel 11 by analyzing the combined features of the arc 31 and the straight line 32 of the remote control steering wheel 11 .
[0067] S11: Since each non-zero pixel point of the contour feature of the remote control steering wheel 11 is a point on the arc 31, the curvature of the arc 31 at any pixel point is It can be obtained by formula (1):
[0068] (1)
[0069] in, is the curvature of arc 31 at the pixel point, is the first-order derivative at the pixel point, is the second-order derivative at the pixel.
[0070] Step S12: The curvature at any pixel point can be obtained according to the above formula, and it is determined whether there is a pixel point whose curvature of the arc 31 is greater than the preset curvature.
[0071] Step S13: If the curvature of the pixel If it is greater than the curvature range set by the operator, it means that the edge passing through the pixel point has a sudden change in direction at this point, so step S130 is executed to identify the pixel point as an inflection point; otherwise, step S131 is executed and the process returns to step S12 to recalculate the curvature of the arc 31 of other pixel points.
[0072] refer to Figure 5 As shown, step S14: at the inflection point, the edge contour is divided into several parts. In order to obtain an ideal edge contour, it is necessary to determine whether each part of the edge contour is a reasonable arc 31. This determination can be made based on the ratio parameter between the arc length and the chord length. To determine, as shown in formula (2):
[0073] (2)
[0074] like Figure 6 As shown, in formula (2), is the length of the edge contour, and is the tangent distance between the first and last points of the edge contour;
[0075] Step S15: If the edge contour is a reasonable arc 31, then the parameters are as shown in formula (3):
[0076] (3)
[0077] From formula (3), we can see that the central angle changes within a period ,parameter With the central angle The increase of the parameter The value of determines whether the identified partial edge is a straight line 32 or a reasonable arc 31; if it is a reasonable arc 31, step S16 is executed, and if it is a straight line 32, step S151 is executed.
[0078] Step S16: After obtaining the reasonable arcs 31 , in order to avoid repeated calculation of the same circle 30 , all the obtained reasonable arcs 31 are sorted from long to short.
[0079] Step S17: Then, the arc 31 is reorganized, referring to Figure 7 As shown, the process of reorganizing the arc 31 includes:
[0080] Step S170: Sort all the obtained reasonable arcs 31 from long to short, starting from the longest reasonable arc 31, and compare them with other reasonable arcs 31 in sequence to determine whether they belong to the same circle 30. For example: compare the reasonable arc 31a with the reasonable arc 31b, the reasonable arc 31c... the reasonable arc 31z in sequence.
[0081] If two reasonable arcs 31 belong to the same circle 30, execute step S171: reorganize the two reasonable arcs 31 into a longer arc 31, recalculate the center and radius of the reorganized arc 31, and then compare the reorganized arc 31 with other reasonable arcs 31 again. For example: compare the reorganized arc 31ab with the reasonable arc 31c and / or the reasonable arc 31d...the reasonable arc 31z in sequence.
[0082] If the two reasonable arcs 31 do not belong to the same circle 30, or after the longest arc 31 is compared with all other reasonable arcs 31, step S172 is executed: the reasonable arc 31 of the next length is compared with other reasonable arcs 31 according to the above method. For example: the reasonable arc 31c is compared with the reasonable arc 31d, the reasonable arc 31e... the reasonable arc 31z in sequence.
[0083] Step S173: Repeat the above process until all arcs 31 and circle fitting results are found, and the arc 31 reorganization is completed.
[0084] After the longest arc 31 is compared with all other reasonable arcs 31, refer to Figure 8 As shown, the method further comprises the following steps:
[0085] Step S174: obtaining a final long arc 31 and a circle fitting result, and then determining whether the final arc 31 occupies at least half of the circumference of the circle 30, and whether the circle fitting error is less than a preset range.
[0086] Step S175: If this long arc 31 occupies more than half of the circumference of the circle 30, and the circle fitting error is less than the error range set by the operator, it is considered that a circle 30 is found; if this long arc 31 occupies less than half of the circumference of the circle 30, execute step S176: judge the arc 31 of the next length.
[0087] In the specific implementation process, refer to Fig. 9 As shown, including:
[0088] Step S160: Whenever an arc 31 is reorganized, it is necessary to determine whether the reorganized arc 31 is a reasonable arc 31.
[0089] Step S161: If the gradient direction of the pixel point on the reorganized arc 31 is consistent with the direction from the center of the circle to the pixel point, Fig.10 The circle shown, point on arc 31 The ideal direction is point to of , however, the true direction calculated based on the pixel gradient is .
[0090] Specifically, the image gradient direction calculation process of the pixel point is:
[0091] ,
[0092] in:
[0093] ,
[0094] ,
[0095] in, is the gradient in the horizontal direction, is the gradient in the vertical direction, is the RGB value of the image pixel, is the pixel coordinate.
[0096] Step S162: If If it is outside the threshold range, it means that the pixel point is not a meaningful point on the arc 31 .
[0097] By adopting the above technical scheme, through the above excavator steering control image recognition method, the arc 31 on the visual feature mark can be quickly detected, which is suitable for detecting the arc 31 in the image. Even if it is a bad arc, the inner and outer edge contours of the arc 31 on the visual feature of the remote control steering wheel 11 can be accurately detected, and the probability of misdetecting the arc 31 is low; through this image recognition method, even if part of the image is blocked, the remote control steering wheel 11 of the excavator 1 can be accurately detected (in actual work, due to the harsh working environment, some features are often not fully recognized due to the influence of light and dust. This recognition method will have a very good effect in this environment).
[0098] In some embodiments, only recognizing the arc 31 of the remote control steering wheel 11 cannot ensure the accuracy of recognition, so a straight line 32 feature can be added to the visual recognition range, referring to Fig.11 As shown, that is:
[0099] Step 100 : Based on the generated remote control steering wheel 11 contour, all arcs 31 in the remote control steering wheel 11 image are sorted according to radius, and then starting from the largest arc 31 , it is determined whether the arc 31 is an arc 31 on the remote control steering wheel 11 contour.
[0100] Step S101: If yes, set a preset square area 33 with the center of the arc 31 as the center, and then detect the straight line 32 in the preset square area 33; if no, execute step S102: return to step S100 to judge the arc 31 of the next size.
[0101] Step S103: If a preset number of straight lines 32 are detected in the preset square area 33 and match the preset conditions, a preset contour feature is formed, and it is considered that the feature of the remote control steering wheel 11 is found.
[0102] Specifically, the preset square area 33 , the preset number, and the preset conditions are adaptively set by the operator according to actual needs and the actual model of the remote control steering wheel 11 of the excavator 1 .
[0103] refer to Fig.12 , Fig.13 As shown, the dotted line portion represents the recognized straight line 32 feature of the remote control steering wheel 11 , the thick solid line portion represents the recognized arc 31 portion; the straight line 32 rectangular area represents the preset square area 33 .
[0104] Based on the above-mentioned excavator steering control image recognition method, the remote control steering wheel 11 of the excavator 1 can be completely recognized in terms of contour, and the extracted preset contour features can be saved as a template for subsequent recognition, and then by comparing with the real-time recognized contour of the remote control steering wheel 11, the current angle deviation based on the original posture of the remote control steering wheel 11 can be accurately judged; however, if the remote control steering wheel 11 is rotating, real-time image recognition requires the extraction of all features as recognition features, which will inevitably affect the recognition rate, resulting in poor real-time performance of the angle acquisition data.
[0105] Since the relative position of the remote control steering wheel 11 and the camera is fixed, the basic outer contour of the remote control steering wheel 11 does not change based on the axis, that is, the circumference and the long and short diameters of the recognized outer contour do not change. Therefore, based on the above-mentioned excavator steering control image recognition method, a simplified operation is performed (that is, the main features of the recognized multiple arcs 31 and straight lines 32 are taken, and the secondary features are ignored). Figure 3 As shown, specifically including:
[0106] Step S2: Generate a real-time contour feature of the remote control steering wheel 11 according to the image of the remote control steering wheel 11, and then compare the real-time contour feature with the preset contour feature to determine whether the image verification has been passed.
[0107] Among them, the largest arc 31 in the real-time contour feature is firstly identified, and then compared with the preset contour feature in the initialization process. When a match is found, the image verification is passed.
[0108] Step S3: If yes, then generate the effective search range of the remote control steering wheel 11 image, and shield all invalid range areas; if no, then return to regenerate the real-time contour features of the remote control steering wheel 11.
[0109] Among them, the specific effective search range is set by the operator according to the actual distance between the camera unit 20 and the remote control steering wheel 11 and the model of the remote control steering wheel 11. Other areas except the effective search range are invalid range areas. In this way, the search speed is further improved by shielding the invalid range.
[0110] Step S4: randomly searching for a group of symmetrical contour line fitting combinations 35 according to the real-time contour features within the effective search range, and then judging the real-time rotation angle of the remote control steering wheel 11 of the excavator 1 according to the symmetrical contour line fitting combination 35 .
[0111] Among them, reference Fig.13 As shown, a group of symmetrical straight line 32 fitting combinations are randomly searched, and then the real-time rotation angle of the remote control steering wheel 11 of the excavator 1 is determined according to the symmetrical straight line 32 fitting combinations.
[0112] In the actual implementation process, after judging the real-time rotation angle of the remote control steering wheel 11 of the excavator 1 according to the fitting combination of the symmetrical straight lines 32, refer to Fig.16 As shown, the method further comprises the following steps:
[0113] Step S40: According to the effective search range of the real-time contour feature, a group of asymmetric contour line fitting combinations 3635 are randomly searched, and then according to the asymmetric contour line fitting combination 3635, the test rotation angle of the remote control steering wheel 11 of the excavator 1 is determined.
[0114] Among them, reference Fig.15 As shown, a group of asymmetric straight line 32 fitting combinations are randomly searched, and then the test rotation angle of the remote control steering wheel 11 of the excavator 1 is determined according to the asymmetric straight line 32 fitting combinations.
[0115] Step S41: Compare the test rotation angle with the real-time rotation angle to determine whether the detection error is within a preset error range.
[0116] The preset error range is set by the operator according to the actual model of the remote control steering wheel 11 and the actual requirements of the steering control of the excavator 1 .
[0117] Step S42: If yes, then the real-time transmission angle is set as the actual rotation angle; if no, then return to randomly search for a set of symmetrical contour line fitting combinations 35 again.
[0118] After the real-time transmission angle is set as the actual rotation angle, the excavator steering control image recognition device 2 transmits the actual rotation angle to the remote controller 12 , thereby providing the remote controller 12 with the ability to operate the steering assembly 10 to control the movement of the excavator 1 .
[0119] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.
[0120] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present invention. A person skilled in the art may change, modify, replace and vary the above embodiments within the scope of the present invention.
[0121] Description of reference numerals:
[0122] 1 Excavator
[0123] 2 Excavator Steering Control Image Recognition Device
[0124] 10 Steering assembly
[0125] 11 Remote control steering wheel
[0126] 12 Remote Controller
[0127] 20 Camera Unit
[0128] 21 Image Recognition Module
[0129] 22. Contour judgment module
[0130] 23 Angle judgment module
[0131] 24 Data transceiver module
[0132] 30 Round
[0133] 31 Arc
[0134] 32 Straight Line
[0135] 33 Preset Square Area
[0136] 34 Maximum characteristic circle fitting
[0137] 35 Symmetrical contour fitting combination
[0138] 36 Asymmetric contour fitting combination.
Claims
1. An excavator steering control image recognition method, characterized in that: The following steps are involved: Initialize contour recognition of the remote control steering wheel according to the collected remote control steering wheel image to form a preset contour feature; Generating a real-time contour feature of the remote control steering wheel according to the remote control steering wheel image, and then comparing the real-time contour feature with a preset contour feature to determine whether the image verification has been passed; If yes, then generate the effective search range of the remote control steering wheel image and mask all invalid range areas; if no, then return to regenerate the real-time contour features of the remote control steering wheel; According to the real-time contour features within the effective search range, a group of symmetrical contour line fitting combinations are randomly searched, and then according to the symmetrical contour line fitting combinations, the real-time rotation angle of the remote control steering wheel of the excavator is determined; Wherein, the preset contour feature is formed by the inner and outer edge contours of the arc on the visual feature of the remote control steering wheel; When judging the real-time rotation angle, a group of symmetrical straight line fitting combinations are randomly searched according to the real-time contour features within the effective search range, and then the real-time rotation angle of the remote control steering wheel of the excavator is judged according to the symmetrical straight line fitting combinations; The effective search range is defined as: First, the largest arc in the real-time contour feature is identified, and then compared with the preset contour feature in the initialization process. When a match is found, the image verification is passed; then the effective search range is set according to the actual distance between the camera unit and the remote control steering wheel and the model of the remote control steering wheel.
2. The excavator steering control image recognition method according to claim 1, characterized in that: When forming the preset contour feature, the method further comprises the following steps: Collect the remote control steering wheel image, and then generate the remote control steering wheel contour by analyzing the combined features of the arc and straight line of the remote control steering wheel; According to each non-zero pixel point of the remote control steering wheel contour, the arc curvature of the pixel point is calculated: Among them, k is the arc curvature at the pixel point, G x , G y is the first-order derivative at the pixel point, G xx , G xy , G yy is the second-order derivative at the pixel point; Determine whether there is a pixel point whose arc curvature is greater than the preset curvature. If so, set the matching pixel point as the inflection point. If not, return to recalculate the arc curvature of the pixel point.
3. The excavator steering control image recognition method according to claim 2, characterized in that: After generating the inflection point, the method further comprises the following steps: Calculate the scale parameter of the inflection point edge contour: Where θ is the central angle; According to the ratio parameter, determine whether the edge contour of the inflection point is a reasonable arc; If yes, all arcs are arranged in a preset order, if no, the inflection point edge contour is identified as a straight line; According to the arranged arcs, arcs belonging to the same circle are reorganized to form a preset contour feature.
4. The excavator steering control image recognition method according to claim 3, characterized in that: When the arcs belonging to the same circle are reorganized, the method further comprises the following steps: According to the arranged arcs, a comparison is performed in sequence according to a preset comparison order to determine whether there are arcs belonging to the same circle; If yes, the arcs belonging to the same circle are reorganized, the center and radius are recalculated, and then the reorganized arcs are sequentially compared according to the preset comparison order. If no, the arc curvature of the pixel point is recalculated; Repeat the above steps until the final arc and circle fitting results are obtained, completing the arc reconstruction.
5. The excavator steering control image recognition method according to claim 4, characterized in that: After obtaining the final arc and circle fitting results, the method further includes the following steps: According to the circle fitting result, determine whether the final arc occupies at least half of the circle circumference and the circle fitting error is less than a preset range; If so, it is considered that a circle is obtained. If not, the arc of the next length is judged.
6. The excavator steering control image recognition method according to claim 5, characterized in that: After obtaining the circle, the method further comprises the following steps: According to any pixel point of the circle, the image gradient direction of the pixel point is calculated: G(x,y)=dx(i,j)+dy(i,j) Wherein, dx(i, j) = I(i+1, j) + I(i, j), dy(i, j) = I(i, j+1) - I(i, j), dx is the gradient in the horizontal direction, dy is the gradient in the vertical direction, I is the RGB value of the image pixel, and (i, j) is the coordinate of the pixel; If the true direction of the pixel point is within the threshold range where the center of the circle points to the pixel point, the pixel point is determined to be a meaningful point, and then a preset contour feature is formed according to the circle; If the true direction of the pixel is not within the threshold range where the center of the circle points to the pixel, the pixel is judged as a meaningless point.
7. The excavator steering control image recognition method according to claim 6, characterized in that: When forming the preset contour feature, the method further comprises the following steps: According to the generated remote control steering wheel contour, all reasonable arcs in the remote control steering wheel image are sorted according to radius, and then according to a preset order, it is determined whether the reasonable arc is an arc on the remote control steering wheel contour; If yes, a preset square area is set with the center of the arc as the center, and then a straight line is detected in the preset square area. If no, the next arc is judged; If a preset number of straight lines are detected within the preset square area and match the preset conditions, a preset contour feature is formed.
8. The excavator steering control image recognition method according to claim 1, characterized in that: After determining the real-time rotation angle of the excavator remote control steering wheel according to the symmetrical straight line fitting combination, the method further comprises the following steps: According to the effective search range of the real-time contour feature, a group of asymmetric contour line fitting combinations are randomly searched, and then according to the asymmetric contour line fitting combinations, a test rotation angle of the remote control steering wheel of the excavator is determined; Compare the test rotation angle with the real-time rotation angle to determine whether the detection error is within the preset error range; If so, the real-time transmission angle is set to the actual rotation angle. If not, the process returns to randomly search for a set of symmetrical contour line fitting combinations.
9. An excavator steering control image recognition device for executing the method according to any one of claims 1 to 8, characterized in that: include: A camera unit, used to capture images of the remote control steering wheel; An image recognition module is used to initialize the contour recognition of the remote control steering wheel according to the collected remote control steering wheel image to form a preset contour feature; wherein the preset contour feature is formed by the inner and outer edge contours of the arc on the visual feature of the remote control steering wheel; The contour judgment module is used to generate the real-time contour feature of the remote control steering wheel according to the remote control steering wheel image, and then compare the real-time contour feature with the preset contour feature. After judging that the image verification is passed, an effective search range is generated, and all invalid range areas are shielded; the effective search range is defined as follows: firstly, the largest arc in the real-time contour feature is identified, and then compared with the preset contour feature in the initialization process. When a match is found, the image verification is passed; and then the effective search range is set according to the actual distance between the camera unit and the remote control steering wheel and the model of the remote control steering wheel; The angle judgment module is used to randomly search for a group of symmetrical straight line fitting combinations according to the effective search range of the real-time contour feature, and then judge the real-time rotation angle of the excavator remote control steering wheel according to the symmetrical straight line fitting combination; when judging the real-time rotation angle, randomly search for a group of symmetrical straight line fitting combinations according to the real-time contour feature within the effective search range, and then judge the real-time rotation angle of the excavator remote control steering wheel according to the symmetrical straight line fitting combination.
10. An excavator, characterized in that: The excavator steering control image recognition device according to claim 9 further comprises: Steering assembly, used to control the direction of movement of the excavator; A remote control steering wheel, which is connected to the steering assembly of the excavator and is used to control the movement direction of the steering assembly; The remote controller is connected with the remote control steering wheel and the excavator steering control image recognition device, and is used to control the rotation angle of the steering assembly.
