Method, device and equipment for detecting hinged angle of hinged vehicle and medium

Abstract

The invention discloses a hinged vehicle hinging angle detection method, device and equipment and a medium, and relates to the field of auxiliary driving and unmanned driving. The hinged vehicle hinging angle detection method comprises the following steps: identifying pixel coordinates of key feature points in an image collected by a forward camera of a current hinged vehicle; determining a target rotation translation matrix according to the key feature pixel point calibration coordinates and the key feature point pixel coordinates; and determining an included angle between front and rear frames of the hinged vehicle according to the target rotation translation matrix and the angle calculation error. According to the technical scheme, installation of a sensor at a hinge point can be omitted, the cost is reduced, and the detection effect of the hinge angle is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of assisted driving and unmanned driving, and in particular to a method, device, equipment and medium for detecting the articulation angle of an articulated vehicle. Background Art

[0002] Articulated construction vehicles, such as loaders, consist of two frames, front and rear, connected by an articulated joint. Steering is achieved by twisting the front and rear frames at the joint. The joint angle of these construction vehicles is a crucial variable reflecting the vehicle's current state of motion and a crucial control variable for controlling vehicle motion. Therefore, real-time acquisition of the vehicle's precise joint angle is crucial, serving as a crucial input parameter for assisted and autonomous driving systems.

[0003] In the existing technical solutions, the articulation angle of an articulated vehicle is obtained through the following two solutions: installing an angle sensor at the hinge point, which is the more commonly used method. The angle sensor is installed at the hinge point and is connected to the front frame and the rear frame respectively. When the front and rear frames are twisted, the angle sensor can detect the angle difference and transmit the signal to the vehicle's underlying controller through the controller local area network bus to parse out the real-time angle value; or, installing inertial sensors on the front and rear frames respectively (this method is less used and is only used for some articulated vehicles, such as when it is inconvenient to install an angle sensor at the hinge point or there is no obvious hinge point). The articulation angle is obtained by the difference in heading angles of the inertial sensors of the front and rear frames, and then the signal is transmitted to the vehicle's underlying controller through the controller local area network bus or serial port to parse out the real-time angle value.

[0004] Since articulated vehicles are large engineering vehicles, the hinge points are large and subject to high forces, so small precision sensors are difficult to use. The hinge points of articulated vehicles are large mechanical links, which are difficult to be very precise in size, resulting in errors in angle sensor detection. In addition, articulated vehicles have a harsh working environment, and the hinge points are located at the bottom of the vehicle. Therefore, they are often affected by dust, water on the ground, severe vibrations, and stone collisions, resulting in a short lifespan. Or after a period of use, the accuracy and stability will decline, and frequent maintenance and replacement are required. Summary of the Invention

[0005] The present invention provides a method, device, equipment and medium for detecting the articulation angle of an articulated vehicle, so as to solve the problems of insufficient accuracy, low reliability and greatly reduced service life of the sensor in articulated vehicle articulation angle detection, resulting in poor detection effect.

[0006] According to one aspect of the present invention, a method for detecting an articulation angle of an articulated vehicle is provided, comprising:

[0007] Identify the pixel coordinates of key feature points in the image captured by the forward camera of the current articulated vehicle;

[0008] Determine the target rotation and translation matrix based on the calibration coordinates of the key feature pixel points and the pixel coordinates of the key feature points;

[0009] The angle between the front and rear frames of the articulated vehicle is determined based on the target rotation and translation matrix and the angle calculation error.

[0010] According to another aspect of the present invention, there is provided an articulation angle detection device for an articulated vehicle, comprising:

[0011] A coordinate recognition module is used to identify the pixel coordinates of key feature points in the image captured by the forward camera of the current articulated vehicle;

[0012] A rotation and translation matrix determination module is used to determine the target rotation and translation matrix based on the calibration coordinates of the key feature pixel points and the pixel coordinates of the key feature points;

[0013] The angle determination module is used to determine the angle between the front and rear frames of the articulated vehicle based on the target rotation and translation matrix and the angle calculation error.

[0014] According to another aspect of the present invention, an electronic device is provided, comprising:

[0015] at least one processor; and

[0016] a memory communicatively connected to the at least one processor; wherein,

[0017] The memory stores a computer program executable by the at least one processor. The computer program is executed by the at least one processor so that the at least one processor can perform the method for detecting the articulation angle of an articulated vehicle according to any embodiment of the present invention.

[0018] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the articulation angle detection method of an articulated vehicle according to any embodiment of the present invention when executed.

[0019] According to another aspect of the present invention, a computer program product is provided. The computer program product comprises a computer program. When the computer program is executed by a processor, the method for detecting the articulation angle of an articulated vehicle according to any embodiment of the present invention is implemented.

[0020] The technical solution of the embodiment of the present invention identifies the pixel coordinates of key feature points in the image captured by the forward-facing camera of the current articulated vehicle, thereby determining the target rotation and translation matrix based on the calibration coordinates of the key feature pixel points and the pixel coordinates of the key feature points, and then determining the angle between the front and rear frames of the articulated vehicle based on the target rotation and translation matrix and the angle calculation error. In this solution, the collection of key data relies on the forward-facing camera commonly installed on articulated vehicles, eliminating the need to install sensors at the hinge point, simplifying the wiring harness and reducing costs. There is no need to consider the problem of the performance reliability of the sensor when used in harsh environments, thereby improving the overall reliability of the system. In addition, the angle calculation error is taken into account when calculating the angle between the front and rear frames of the articulated vehicle, so that the accuracy and stability of the articulation angle detection are improved. This solves the problems of insufficient accuracy, low reliability and greatly reduced sensor service life in the articulation angle detection of articulated vehicles, resulting in poor detection results. It can eliminate the need to install sensors at the hinge point to reduce costs while improving the detection effect of the articulation angle.

[0021] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present invention, nor is it intended to limit the scope of the present invention. Other features of the present invention will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0023] Figure 1 This is a flow chart of a method for detecting the articulation angle of an articulated vehicle provided in the first embodiment of the present invention;

[0024] Figure 2 This is a flow chart of a method for detecting the articulation angle of an articulated vehicle provided in the second embodiment of the present invention;

[0025] Figure 3 A schematic diagram of the placement of markers on a bucket according to a second embodiment of the present invention;

[0026] Figure 4 A schematic structural diagram of an articulation angle detection device for an articulated vehicle provided in a third embodiment of the present invention;

[0027] Figure 5 A schematic structural diagram of an electronic device that can be used to implement an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0028] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0029] It should be noted that the terms "current", "target", etc. in the description and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0030] Example 1

[0031] Figure 1 This is a flow chart of a method for detecting the articulation angle of an articulated vehicle provided in the first embodiment of the present invention. This embodiment is applicable to the case of accurately detecting the articulation angle of an articulated vehicle. The method can be executed by an articulated vehicle articulation angle detection device, which can be implemented in the form of hardware and / or software. The articulated vehicle articulation angle detection device can be configured in an electronic device, which can be a vehicle bottom-layer controller and can be configured in an articulated vehicle. Figure 1 As shown, the method includes:

[0032] Step 110: Identify pixel coordinates of key feature points in an image captured by a forward-facing camera of the current articulated vehicle.

[0033] The current articulated vehicle may be a vehicle currently performing front and rear frame articulation angle detection. The image captured by the forward camera may be an image of the front frame captured in real time by the forward camera of the current articulated vehicle. The forward camera may be a camera installed on the rear frame of the current articulated vehicle for capturing images of the vehicle's forward perspective. The pixel coordinates of the key feature points may be the pixel coordinates of preset points on the front frame in the image captured by the forward camera. The preset points on the front frame may be boundary points of components of the front frame of the current articulated vehicle. Exemplarily, the preset points on the front frame may include, but are not limited to, boundary points of the boom and / or boundary points of the bucket.

[0034] In an embodiment of the present invention, the image of the front frame can be captured in real time by the forward camera of the current articulated vehicle, that is, the forward camera captures the image, and the pixel coordinates of key feature points can be identified from the image captured by the forward camera based on image recognition technology.

[0035] Step 120: Determine the target rotation and translation matrix based on the key feature pixel calibration coordinates and the key feature point pixel coordinates.

[0036] The calibrated coordinates of the key feature pixel points may be the coordinates of a preset point on the front frame in a three-dimensional coordinate system, and the target rotation and translation matrix may be a linear transformation matrix describing the position and orientation transformation of the front frame of the current articulated vehicle.

[0037] In an embodiment of the present invention, the calibrated coordinates of the key feature pixel points can be unified into the camera coordinate system so that the unified coordinates and the key feature point pixel coordinates are in the same coordinate system, thereby determining the target rotation and translation matrix based on the computer vision and machine learning software library and the difference between the two sets of coordinates.

[0038] Step 130: Determine the angle between the front and rear frames of the articulated vehicle based on the target rotation and translation matrix and the angle calculation error.

[0039] The angle calculation error may be an angle error caused by a gap when the hinge point of the current articulated vehicle is twisted. The angle between the front and rear frames of the articulated vehicle may be the articulation angle of the current articulated vehicle, that is, the angle between the front and rear frames of the articulated vehicle.

[0040] In an embodiment of the present invention, the target rotation and translation matrix may be converted into an angle, and the converted angle may be compensated by angle calculation error to obtain the angle between the front and rear frames of the articulated vehicle.

[0041] The technical solution of the embodiment of the present invention identifies the pixel coordinates of key feature points in the image captured by the forward-facing camera of the current articulated vehicle, thereby determining the target rotation and translation matrix based on the calibration coordinates of the key feature pixel points and the pixel coordinates of the key feature points, and then determining the angle between the front and rear frames of the articulated vehicle based on the target rotation and translation matrix and the angle calculation error. In this solution, the collection of key data relies on the forward-facing camera commonly installed on articulated vehicles, eliminating the need to install sensors at the hinge point, simplifying the wiring harness and reducing costs. There is no need to consider the problem of the performance reliability of the sensor when used in harsh environments, thereby improving the overall reliability of the system. In addition, the angle calculation error is taken into account when calculating the angle between the front and rear frames of the articulated vehicle, so that the accuracy and stability of the articulation angle detection are improved. This solves the problems of insufficient accuracy, low reliability and greatly reduced sensor service life in the articulation angle detection of articulated vehicles, resulting in poor detection results. It can eliminate the need to install sensors at the hinge point to reduce costs while improving the detection effect of the articulation angle.

[0042] Example 2

[0043] Figure 2 This is a flowchart of a method for detecting the articulation angle of an articulated vehicle provided in the second embodiment of the present invention. This embodiment is specific based on the above embodiment and provides a specific optional implementation method for identifying the pixel coordinates of key feature points in the image captured by the forward camera of the current articulated vehicle. Figure 2 As shown, the method includes:

[0044] Step 210: Identify the pixel coordinates of key feature points in the image captured by the forward camera of the current articulated vehicle based on the key feature point pixel coordinate recognition model; or, identify the pixel coordinates of key feature points in the image captured by the forward camera of the current articulated vehicle based on the markers on the front frame object to be identified of the current articulated vehicle.

[0045] The key feature point pixel coordinate recognition model can be used to identify the pixel coordinates of key feature points in images captured by the forward-facing camera. The key feature point pixel coordinate recognition model can be a deep learning artificial intelligence model. The front frame object to be identified can be a detection target of the front frame of an articulated vehicle. The marker can be a marking object pre-placed on the front frame object to be identified. Exemplary markers include, but are not limited to, stickers such as circles or triangles filled with a color distinct from the front frame.

[0046] In an embodiment of the present invention, an image captured by a forward-facing camera of the current articulated vehicle can be input into a key feature point pixel coordinate recognition model to identify and locate the key feature point pixel coordinates. If a marker is pre-placed on the front frame of the current articulated vehicle to be identified, the marker can be used as a detection target in the image captured by the forward-facing camera of the current articulated vehicle, and the coordinates of the marker's geometric center in the camera coordinate system can be used as the key feature point pixel coordinates.

[0047] In an optional embodiment of the present invention, the markers on the front frame object to be identified of the current articulated vehicle are symmetrically arranged at the boundary of the front frame object to be identified; the front frame object to be identified may include a boom and / or a bucket.

[0048] In an embodiment of the present invention, before calculating the angle between the front and rear frames of the articulated vehicle, markers may be symmetrically arranged on the left and right boundaries of the front frame object to be identified (such as a boom and / or bucket).

[0049] In an optional embodiment of the present invention, before identifying the pixel coordinates of key feature points in the image captured by the forward camera of the current articulated vehicle based on the key feature point pixel coordinate recognition model, the method may also include: obtaining a key feature recognition training sample set of the front frame object to be identified; training a preset deep learning model based on the key feature recognition training sample set to obtain a key feature point pixel coordinate recognition model.

[0050] The key feature recognition training sample set may be a set of image samples of the front frame captured by a forward-facing camera at different rotation angles. The image samples in the key feature recognition training sample set may be annotated samples, with the annotated content including, but not limited to, the category and location of the front frame object to be identified. The preset deep learning model may be a preselected deep learning model.

[0051] In an embodiment of the present invention, historical images captured by the forward camera of the current articulated vehicle can be cleaned, and the processed historical images can be annotated with relevant data of the front frame object to be identified to obtain a key feature recognition training sample set of the front frame object to be identified. Then, based on the key feature recognition training sample set, a preset deep learning model is trained to obtain a key feature point pixel coordinate recognition model that meets the model accuracy requirements.

[0052] In an optional embodiment of the present invention, based on the marker on the front frame object to be identified of the current articulated vehicle, identifying the pixel coordinates of the key feature points in the image captured by the forward camera of the current articulated vehicle may include: using the marker on the front frame object to be identified of the current articulated vehicle as the key feature point, and using image recognition technology to determine the pixel coordinates of the key feature points in the image captured by the forward camera of the current articulated vehicle.

[0053] Among them, the key feature points can be understood as the recognition targets of the images captured by the forward camera.

[0054] In an embodiment of the present invention, the marker on the front frame object to be identified of the current articulated vehicle is used as the key feature point. The image recognition technology can be used to identify the coordinates of the key feature point in the image captured by the forward camera of the current articulated vehicle to obtain the pixel coordinates of the key feature point.

[0055] Step 220: Determine the target rotation and translation matrix based on the key feature pixel calibration coordinates and the key feature point pixel coordinates.

[0056] In an optional embodiment of the present invention, determining the target rotation and translation matrix based on the calibrated coordinates of the key feature pixel points and the pixel coordinates of the key feature points can include: projecting the calibrated coordinates of the key feature pixel points onto the target two-dimensional plane to obtain the projection coordinates of the key feature pixel points; and creating the target rotation and translation matrix based on the coordinate errors of the projection coordinates of the key feature pixel points and the pixel coordinates of the key feature points.

[0057] The target two-dimensional plane may be a two-dimensional plane corresponding to the camera coordinate system, and the projection coordinates of the key feature pixel points may be the coordinates of the key feature pixel point calibration coordinates after being projected onto the target two-dimensional plane.

[0058] In an embodiment of the present invention, the calibration coordinates of the key feature pixel points can be determined first, and then the calibration coordinates of the key feature pixel points can be projected onto the target two-dimensional plane, and the coordinates projected onto the target two-dimensional plane can be used as the projection coordinates of the key feature pixel points. The coordinate errors of the projection coordinates of the key feature pixel points and the key feature point pixel coordinates can be further calculated, and the coordinate errors can be converted based on the software in the computer vision and machine learning software library to obtain the target rotation and translation matrix.

[0059] Step 230: Determine the angle between the front and rear frames of the articulated vehicle based on the target rotation and translation matrix and the angle calculation error.

[0060] In an optional embodiment of the present invention, determining the angle between the front and rear frames of an articulated vehicle based on a target rotation and translation matrix and an angle calculation error may include: calculating the Euler angle to be corrected based on the target rotation and translation matrix; and using the sum of the Euler angle to be corrected and the angle calculation error as the angle between the front and rear frames of the articulated vehicle.

[0061] The Euler angle to be corrected may be the Euler angle converted from the target rotation and translation matrix.

[0062] In an embodiment of the present invention, the existing visual processing function can be used to convert the target rotation and translation matrix into the Euler angle to be corrected, and then the sum of the Euler angle to be corrected and the angle calculation error is calculated, and the calculated sum is used as the angle between the front and rear frames of the articulated vehicle.

[0063] In a specific example, the pixel coordinates of key feature points on the boom and bucket are identified through the key feature point recognition model from the image captured by the forward camera, or when markers are attached to the boom or bucket, the pixel coordinates of key feature points are located with the help of the markers. Since the relative relationship between the key feature points has been calibrated in advance, the coordinate error between the projection coordinates of the key feature pixel points and the pixel coordinates of the key feature points can be further calculated, and the coordinate error can be converted into the target rotation and translation matrix, that is, the rotation and translation matrix of the boom coordinate system or the bucket coordinate system, through the software in the computer vision and machine learning software library. The target rotation and translation matrix is ​​then converted into Euler angles, and compensation is performed based on the angle calculation error to obtain the angle between the front and rear frames. The layout of the markers on the bucket can be found in Figure 3 , the marker is Figure 3 The circular part in the.

[0064] It should be noted that there is no need to synchronize the timestamps of the front and rear frame angles of the articulated vehicle with the forward-facing camera.

[0065] The technical solution of the embodiment of the present invention is to identify the pixel coordinates of key feature points in the image captured by the forward camera of the current articulated vehicle based on the key feature point pixel coordinate recognition model; or, based on the markers on the front frame object to be identified of the current articulated vehicle, to identify the pixel coordinates of key feature points in the image captured by the forward camera of the current articulated vehicle, and then determine the target rotation and translation matrix based on the calibration coordinates of the key feature pixel points and the pixel coordinates of the key feature points, so as to determine the angle between the front and rear frames of the articulated vehicle based on the target rotation and translation matrix and the angle calculation error. In this solution, the collection of key data relies on the forward camera commonly installed on articulated vehicles, eliminating the need to install sensors at the hinge points, simplifying the wiring harness and reducing costs. There is no need to consider the problem that the performance reliability of the sensors cannot be guaranteed when used in harsh environments, thereby improving the overall reliability of the system. When calculating the angle between the front and rear frames of an articulated vehicle, the angle calculation error is taken into account, so that the accuracy and stability of the articulation angle detection can be improved. This solves the problems of insufficient accuracy, low reliability and greatly reduced sensor service life in the articulation angle detection of articulated vehicles, resulting in poor detection results. It can eliminate the need to install sensors at the hinge points to reduce costs while improving the detection effect of the articulation angle.

[0066] Example 3

[0067] Figure 4 This is a schematic diagram of the structure of an articulated vehicle articulation angle detection device provided by the third embodiment of the present invention. Figure 4 As shown, the device includes:

[0068] A coordinate recognition module 310 is used to identify pixel coordinates of key feature points in an image captured by a forward-facing camera of the current articulated vehicle;

[0069] A rotation and translation matrix determination module 320 is configured to determine a target rotation and translation matrix based on the key feature pixel calibration coordinates and the key feature point pixel coordinates;

[0070] The angle determination module 330 is used to determine the angle between the front and rear frames of the articulated vehicle based on the target rotation and translation matrix and the angle calculation error.

[0071] The technical solution of the embodiment of the present invention identifies the pixel coordinates of key feature points in the image captured by the forward-facing camera of the current articulated vehicle, thereby determining the target rotation and translation matrix based on the calibration coordinates of the key feature pixel points and the pixel coordinates of the key feature points, and then determining the angle between the front and rear frames of the articulated vehicle based on the target rotation and translation matrix and the angle calculation error. In this solution, the collection of key data relies on the forward-facing camera commonly installed on articulated vehicles, eliminating the need to install sensors at the hinge point, simplifying the wiring harness and reducing costs. There is no need to consider the problem of the performance reliability of the sensor when used in harsh environments, thereby improving the overall reliability of the system. In addition, the angle calculation error is taken into account when calculating the angle between the front and rear frames of the articulated vehicle, so that the accuracy and stability of the articulation angle detection are improved. This solves the problems of insufficient accuracy, low reliability and greatly reduced sensor service life in the articulation angle detection of articulated vehicles, resulting in poor detection results. It can eliminate the need to install sensors at the hinge point to reduce costs while improving the detection effect of the articulation angle.

[0072] Optionally, the coordinate recognition module 310 is specifically used to identify the pixel coordinates of key feature points in the image captured by the forward camera of the current articulated vehicle based on a key feature point pixel coordinate recognition model; or, based on a marker on the front frame object to be identified of the current articulated vehicle, to identify the pixel coordinates of key feature points in the image captured by the forward camera of the current articulated vehicle.

[0073] Optionally, the articulation angle detection device for an articulated vehicle further includes a model training module for obtaining a key feature recognition training sample set of the front frame object to be identified; based on the key feature recognition training sample set, a preset deep learning model is trained to obtain the key feature point pixel coordinate recognition model.

[0074] Optionally, the rotation and translation matrix determination module 320 is specifically used to calibrate the coordinates of the key feature pixel points and project them onto the target two-dimensional plane to obtain the projection coordinates of the key feature pixel points; and create the target rotation and translation matrix based on the projection coordinates of the key feature pixel points and the coordinate error of the key feature point pixel coordinates.

[0075] Optionally, the angle determination module 330 is used to calculate the Euler angle to be corrected according to the target rotation and translation matrix; and use the sum of the Euler angle to be corrected and the angle calculation error as the angle between the front and rear frames of the articulated vehicle.

[0076] Optionally, the markers on the front frame object to be identified of the current articulated vehicle are symmetrically arranged at the boundary of the front frame object to be identified; the front frame object to be identified includes a boom and / or a bucket.

[0077] Optionally, the coordinate recognition module 310 is specifically used to use the markers on the front frame object to be identified of the current articulated vehicle as key feature points, and use image recognition technology to determine the pixel coordinates of the key feature points in the image captured by the forward camera of the current articulated vehicle.

[0078] The articulated vehicle articulation angle detection device provided in the embodiment of the present invention can execute the articulated vehicle articulation angle detection method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0079] Example 4

[0080] Figure 5 A schematic diagram of an electronic device that can be used to implement embodiments of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as workstations, personal digital assistants, blade servers, and other suitable computers. The components shown herein, their connections and relationships, and their functions are provided for example only and are not intended to limit the implementation of the present inventions described and / or claimed herein.

[0081] like Figure 5 As shown, electronic device 10 includes at least one processor 11 and memory, such as ROM 12 and RAM 13, communicatively connected to at least one processor 11. The memory stores computer programs executable by the at least one processor, and processor 11 can perform various appropriate actions and processes based on the computer programs stored in ROM 12 or loaded from storage unit 18 into RAM 13. RAM 13 can also store various programs and data required for the operation of electronic device 10. Processor 11, ROM 12, and RAM 13 are interconnected via bus 14. An I / O interface 15 is also connected to bus 14. ROM 12 is a read-only memory, RAM 13 is a random access memory, and I / O interface 15 is an input / output interface.

[0082] Multiple components in the electronic device 10 are connected to the I / O interface 15, including an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a magnetic disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0083] The processor 11 can be any general-purpose and / or specialized processing component with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any other suitable processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as the articulation angle detection method for an articulated vehicle.

[0084] In some embodiments, the articulated vehicle articulation angle detection method may be implemented as a computer program tangibly embodied in a computer-readable storage medium, such as the storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the articulated vehicle articulation angle detection method described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to execute the articulated vehicle articulation angle detection method in any other suitable manner (e.g., via firmware).

[0085] Various embodiments of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chip systems (SOCs), programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0086] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when the computer program is executed by the processor, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The computer program may be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0087] In the context of the present invention, computer-readable storage medium can be a tangible medium that can contain or store a computer program for use with an instruction execution system, device or equipment or used in combination with an instruction execution system, device or equipment. Computer-readable storage medium can include but is not limited to electronic, magnetic, optical, electromagnetic, infrared or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, computer-readable storage medium can be a machine-readable signal medium. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, RAM, ROM, an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device or any suitable combination of the foregoing.

[0088] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0089] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0090] A computing system may include clients and servers. The clients and servers are generally remote from each other and typically interact via a communication network. This client-server relationship arises through computer programs running on the respective computers, creating a client-server relationship. The server may be a cloud server, also known as a cloud computing server or cloud host. This server is a hosting product within the cloud computing service ecosystem that addresses the management difficulties and limited scalability of traditional physical hosts and VPS servers.

[0091] The present application also discloses a computer program product comprising a computer program that, when executed by a processor, implements the articulation angle detection method for an articulated vehicle provided in any of the embodiments of the present application. This program product shares the same inventive concept as the articulation angle detection method for an articulated vehicle disclosed in each embodiment of the present application and is therefore not further described here.

[0092] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in the present invention can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solution of the present invention can be achieved. This is not limited herein.

[0093] The above specific embodiments do not limit the scope of protection of the present invention. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention are intended to be included within the scope of protection of the present invention.

Claims

1. A method for detecting the articulation angle of an articulated vehicle, characterized in that: include: Identify the pixel coordinates of key feature points in the image captured by the forward camera of the current articulated vehicle; Determine the target rotation and translation matrix based on the key feature pixel calibration coordinates and the key feature point pixel coordinates; The angle between the front and rear frames of the articulated vehicle is determined based on the target rotation and translation matrix and the angle calculation error.

2. The method according to claim 1, characterized in that Identify the pixel coordinates of key feature points in the image captured by the forward camera of the current articulated vehicle, including: Identifying the pixel coordinates of key feature points in the image captured by the forward camera of the current articulated vehicle based on a key feature point pixel coordinate recognition model; or Based on the marker on the to-be-identified front frame object of the current articulated vehicle, pixel coordinates of key feature points in an image captured by a forward camera of the current articulated vehicle are identified.

3. The method according to claim 2, characterized in that Before identifying the pixel coordinates of the key feature points in the image captured by the forward camera of the current articulated vehicle based on the key feature point pixel coordinate recognition model, the method further includes: Obtaining a key feature recognition training sample set of the front frame object to be identified; Based on the key feature recognition training sample set, a preset deep learning model is trained to obtain the key feature point pixel coordinate recognition model.

4. The method according to claim 1, wherein Determining the target rotation and translation matrix based on the key feature pixel calibration coordinates and the key feature point pixel coordinates includes: Calibrate the coordinates of the key feature pixel points and project them onto the target two-dimensional plane to obtain the projection coordinates of the key feature pixel points; The target rotation and translation matrix is ​​created according to the projection coordinates of the key feature pixel points and the coordinate errors of the key feature point pixel coordinates.

5. The method according to claim 1, wherein Determining the angle between the front and rear frames of the articulated vehicle according to the target rotation and translation matrix and the angle calculation error includes: Calculating the Euler angles to be corrected according to the target rotation and translation matrix; The sum of the Euler angle to be corrected and the angle calculation error is used as the angle between the front and rear frames of the articulated vehicle.

6. The method according to claim 2, characterized in that The markers on the front frame object to be identified of the current articulated vehicle are symmetrically arranged at the boundary of the front frame object to be identified; The front frame object to be identified includes a boom and / or a bucket.

7. The method according to claim 2, characterized in that Identifying pixel coordinates of key feature points in an image captured by a forward camera of the current articulated vehicle based on a marker on a front frame object to be identified of the current articulated vehicle includes: The markers on the front frame object to be identified of the current articulated vehicle are used as key feature points, and the pixel coordinates of the key feature points in the image captured by the front camera of the current articulated vehicle are determined by using image recognition technology.

8. An articulated vehicle articulation angle detection device, characterized in that: include: A coordinate recognition module is used to identify the pixel coordinates of key feature points in the image captured by the forward camera of the current articulated vehicle; A rotation and translation matrix determination module is used to determine a target rotation and translation matrix based on the calibration coordinates of the key feature pixel points and the pixel coordinates of the key feature points; The angle determination module is used to determine the angle between the front and rear frames of the articulated vehicle based on the target rotation and translation matrix and the angle calculation error.

9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor. The computer program is executed by the at least one processor to enable the at least one processor to perform the articulation angle detection method for an articulated vehicle according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the articulation angle detection method for an articulated vehicle according to any one of claims 1 to 7 when executed.

Images