Articulated roller panoramic view system and calibration method, articulated roller

Abstract

The present disclosure relates to a panoramic surround view system and calibration method for an articulated roller, and an articulated roller, wherein the articulated roller comprises a first articulated structure section and a second articulated structure section connected by an articulated frame, the first articulated structure section comprises a first body and a cab arranged on the first body, the first body is one of a front body and a rear body, the second articulated structure section comprises a second body, and the second body is the other of the front body and the rear body, and the panoramic surround view system comprises: a plurality of shooting components, installed on the first articulated structure section, and configured to shoot the environment around the articulated roller; an image processing component, configured to receive images shot by the plurality of shooting components, and splice them into a surround view image of the entire circumference around the first articulated structure section; and a human-computer interaction component, configured to display the surround view image spliced ​​by the image processing components.

Description

Technical Field

[0001] The present disclosure relates to the technical field of assisted driving of road rollers, and in particular to a panoramic surround view system and calibration method for an articulated road roller, and an articulated road roller. Background Art

[0002] The development of modern science and technology has greatly promoted the intelligence of road rollers. More and more high-end technologies have been introduced into the field of road rollers, making road rollers move towards the goal of being more efficient and safer. Usually, during the construction process, road rollers can only rely on rearview mirrors or reversing images to assist in observing the surrounding environment. However, due to the presence of obstructions such as the front of the vehicle, windows, and pillars, and the generally high and limited field of vision of the road roller cab, the driver has a blind spot in his field of vision and cannot judge whether there are obstacles in the obstructed area and the distance between obstacles.

[0003] Moreover, in order to improve the steering and operating flexibility of the roller, a roller in which the cab is connected to the rear frame by an articulated mechanism has appeared. During the steering process of the roller, since the angle of the rear frame relative to the cab is constantly changing, it is more difficult for the driver to judge the environment around the roller, which increases the difficulty of operation and makes it difficult to ensure the safety of the driving process. Summary of the invention

[0004] Embodiments of the present disclosure provide a panoramic surround view system and calibration method for an articulated roller, and an articulated roller, which can improve the safety of the roller during driving.

[0005] A first aspect of the present disclosure provides a panoramic surround view system for an articulated roller, the articulated roller comprising a first articulated structure section and a second articulated structure section connected by an articulated frame, the first articulated structure section comprising a first body and a cab provided on the first body, the first body being one of a front body and a rear body, the second articulated structure section comprising a second body, the second body being the other of the front body and the rear body, the panoramic surround view system comprising:

[0006] A plurality of photographing components, mounted on the first articulated structure section, and configured to photograph the environment around the articulated roller;

[0007] An image processing component configured to receive images captured by a plurality of capturing components and stitch them together into a surround image of the entire circumference of the first articulated structure segment; and

[0008] The human-computer interaction component is configured to display the surround image formed by the splicing of the image processing components.

[0009] In some embodiments, the plurality of photographing components are all located on the same side of the articulated frame along the length direction of the articulated roller.

[0010] In some embodiments, a plurality of photographing components are disposed around the outer circumference of the cab or the outer circumference of the first articulated structure segment.

[0011] In some embodiments, the plurality of photographing components include four photographing components, which are respectively disposed at the front, rear, left and right sides of the cab.

[0012] In some embodiments, a plurality of photographing components are disposed on the top of the cab, and photographing surfaces of the photographing components face the outside of the cab and are slanted downward.

[0013] In some embodiments, the plurality of photographing components include four photographing components, which are respectively arranged on a side of the cab close to the articulated frame, a left side of the cab, a right side of the cab, and a side of the first vehicle body away from the articulated frame.

[0014] In some embodiments, it also includes:

[0015] A travel control component is electrically connected to the image processing component and is configured to control the travel direction and travel speed of the articulated roller and transmit the travel direction and travel speed information to the image processing component;

[0016] Among them, the image processing component is configured to move the surround view image displayed in the human-computer interaction component up and down according to the traveling direction, move the surround view image downward when the traveling direction is forward, and move the surround view image upward when the traveling direction is backward, and determine the speed of movement of the surround view image and the position where the final image stops according to the traveling speed.

[0017] In some embodiments, multiple calibration devices are also included, which are arranged on a preset reference plane outside the cab and around the periphery of the cab or the first articulated structure segment. The top surface of each of the multiple calibration devices is lower than the installation height of the shooting component at the corresponding position. The multiple calibration devices are configured to calibrate the captured images before the multiple shooting components are officially used.

[0018] In some embodiments, the human-computer interaction component is configured to receive a calibration instruction from an operator to move a calibration cursor to a feature recognition object in a calibration device, and send the calibration instruction to the image processing component; the image processing component is configured to receive the calibration instruction and correct the distortion in the images captured by multiple shooting components according to actual parameters of multiple calibration devices.

[0019] In some embodiments, actual parameters of the multiple calibration devices include: the distance between two adjacent calibration devices and the outer dimensions of a feature identifier in a single calibration device, and the feature identifier is configured for an operator to perform calibration through a calibration cursor in a human-computer interaction component.

[0020] In some embodiments, a plurality of calibration devices are arranged outside a preset reference frame, and the preset reference frame is offset outward by a preset distance as a whole relative to a rectangular area formed by the outer contour of the cab or the first articulated structure segment;

[0021] In conjunction with the display of the human-computer interaction component, the shooting angles of the multiple shooting components are configured to be adjusted to be able to capture a range between the body of the articulated roller and a preset reference frame.

[0022] In some embodiments, a plurality of calibration devices are disposed outside a preset reference frame, and the preset reference frame is offset outward by a preset distance as a whole relative to the area where the outer contour of the cab is located;

[0023] A plurality of calibration devices are respectively arranged at the four corners of the preset reference frame.

[0024] In some embodiments, the height of the calibration device is adjustable.

[0025] In some embodiments, the human-computer interaction component is configured to receive externally input image revision parameters of at least some of the plurality of photographing components, so as to make the surround view images at the same height.

[0026] In some embodiments, the calibration device includes:

[0027] Base;

[0028] A first support frame, wherein a first end of the first support frame is fixed on the base;

[0029] A second support frame, located above the first support frame, wherein a first end of the second support frame is connected to a second end of the first support frame and the height of the second support frame is adjustable; and

[0030] The frame is connected to the second end of the second support frame, and the frame serves as a feature identification object when an operator marks in the human-machine interaction component.

[0031] A second aspect of the present disclosure provides an articulated roller, comprising: a first articulated structure segment and a second articulated structure segment connected by an articulated frame and a panoramic view system for an articulated roller of the above embodiment;

[0032] The first articulated structure section includes a first body and a cab arranged on the first body, the first body is one of the front body and the rear body, and the second articulated structure section includes a second body, the second body is the other of the front body and the rear body.

[0033] A third aspect of the present disclosure provides a calibration method for a panoramic view system of an articulated roller based on the above embodiment, comprising:

[0034] In combination with the display of the human-machine interaction component, the shooting angles of the multiple shooting components are adjusted to be able to shoot the range between the body of the articulated roller and a preset reference frame, and the preset reference frame is offset outward by a preset distance as a whole relative to the area where the cab or the outer contour of the first articulated structure section is located;

[0035] Arrange a plurality of calibration devices on a reference plane around a preset reference frame;

[0036] The calibration cursor in the human-computer interaction component is moved to the feature recognition object of the calibration device to calibrate the captured images before the multiple shooting components are officially used.

[0037] In some embodiments, it also includes:

[0038] After the surround view image is formed by stitching, image revision parameters of at least some of the multiple shooting components are set through the human-computer interaction component according to the height matching degree of each part of the surround view image, so that the surround view image is at the same height.

[0039] In some embodiments, arranging a plurality of calibration devices on a reference plane outside a preset reference frame includes:

[0040] Obtain an overall top-view model of the articulated roller;

[0041] Only the partial model of the area where the cab or the outer contour of the first articulated structure section is located is retained in the overall top view model;

[0042] Find an actual area corresponding to the partial model in the actual vehicle body, offset the actual area outward by a preset distance, and mark a preset reference frame on the reference plane;

[0043] A plurality of calibration devices are arranged on a reference plane around a preset reference frame.

[0044] In some embodiments, before calibrating the captured image, the calibration method further includes:

[0045] Inputting actual parameters of multiple calibration devices through the human-computer interaction component so that the image processing component can correct the distortion in the images captured by the multiple capturing components to achieve calibration;

[0046] Among them, the actual parameters of the multiple calibration devices include the distance between two adjacent calibration devices and the outer dimensions of the feature identification object in a single calibration device, and the feature identification object is configured for the operator to calibrate in the human-machine interaction component.

[0047] In some embodiments, after arranging the plurality of calibration devices on a reference plane outside a preset reference frame, the calibration method further includes:

[0048] The heights of the multiple calibration devices are adjusted to adjust the proportion of the articulated roller body in the surround view image after calibration.

[0049] In some embodiments, the plurality of photographing components include four photographing components, which are respectively arranged at the front, rear, left and right sides of the cab, and a calibration device is respectively provided at each of the four corners of the preset reference frame; wherein,

[0050] With the articulated frame located in front of the cab, adjusting the height of multiple calibration devices includes:

[0051] Adjust the height of the calibration devices at the left front and right front positions to be close to the height of the vehicle body structure and lower than the installation height of the front shooting components;

[0052] Adjust the height of the calibration device at the left rear and right rear positions to be lower than the installation height of the rear shooting component, and the height difference with the calibration device at the left front and right front positions does not exceed the preset height; or

[0053] With the articulated frame located behind the cab, adjusting the height of multiple calibration devices includes:

[0054] Adjust the height of the calibration devices at the left rear and right rear positions to be close to the vehicle body structure and lower than the installation height of the rear shooting components;

[0055] The height of the calibration device at the left front and right front positions is adjusted to be lower than the installation height of the front shooting component, and the height difference with the calibration device at the left rear and right rear positions does not exceed the preset height.

[0056] Based on the above technical scheme, in the panoramic view system of the articulated roller of the disclosed embodiment, multiple shooting components are installed on the first articulated structure section. When the second articulated structure section swings relative to the first articulated structure section including the cab during the movement of the roller, the shooting components can also clearly capture the position change of the second articulated structure section and the situation outside the cab, making image stitching easier, optimizing the image stitching effect, and preventing visual blind spots. There is also no need to install an angle sensor to detect the swing angle of the vehicle body structure to achieve image stitching, making the structure of the panoramic view system simpler and easier to implement. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] The drawings described herein are used to provide a further understanding of the present disclosure and constitute a part of the present application. The illustrative embodiments of the present disclosure and their descriptions are used to explain the present disclosure and do not constitute an improper limitation on the present disclosure. In the drawings:

[0058] Figure 1 is a side view of some embodiments of the articulated roller disclosed herein;

[0059] Figure 2It is a top view of some embodiments of the articulated roller disclosed in the present invention;

[0060] Figure 3A It is an overall top view model of the articulated roller disclosed in the present invention;

[0061] Figure 3B for Figure 3A Only the model of the hinged end section where the cab is located is retained in the overall top-view model;

[0062] Figure 3C is a schematic diagram of a determined preset reference frame;

[0063] Figure 3D A schematic diagram of the structure of arranging the calibration devices at four corners outside the preset reference frame;

[0064] Figure 4 A schematic diagram of actual parameters of multiple calibration devices;

[0065] Figure 5 It is a schematic diagram of the structure of some embodiments of the calibration device;

[0066] Figure 6 The present invention is a schematic diagram of the module composition of some embodiments of the panoramic view system for an articulated roller.

[0067] Description of Reference Numerals

[0068] 1. First articulated structure section; 11. First vehicle body; 12. Driver's cab; 13. Elastic coupling; 2. Second articulated structure section; 21. Second vehicle body; 3. Articulated frame; 4. Front wheel; 5. Rear wheel; 6. Photographing component; 7. Calibration device; 71. Base; 711. Crossbeam; 72. First support frame; 73. Second support frame; 74. Frame; 741. Beam; 742. Reinforcement beam; X. Calibration cursor; 8. Image processing component; 9. Human-computer interaction component; 10. Travel control component. DETAILED DESCRIPTION

[0069] The present disclosure is described in detail below. In the following paragraphs, different aspects of the embodiments are defined in more detail. Each aspect so defined can be combined with any other aspect or aspects, unless explicitly stated not to be combined. In particular, any feature that is considered to be preferred or advantageous can be combined with one or more other features that are considered to be preferred or advantageous.

[0070] The terms "first", "second", etc. that appear in the present disclosure are only for the convenience of description to distinguish different components with the same name, and do not indicate a sequence or a primary and secondary relationship.

[0071] In the description of the present disclosure, directions or positional relationships indicated by "up", "down", "left", "right", "front", "back", "inside" and "outside" are all defined based on the position of the operator sitting in the cab. They are only for the convenience of describing the present disclosure and do not indicate or imply that the device referred to must have a specific direction, be constructed and operated in a specific direction. Therefore, they should not be understood as limiting the scope of protection of the present disclosure.

[0072] like Figures 1 to 6 As shown, the present disclosure provides a panoramic surround view system for an articulated roller. In some embodiments, the articulated roller includes a first articulated structure segment 1 and a second articulated structure segment 2 connected by an articulated frame 3. The first articulated structure segment 1 includes a first body 11 and a cab 12 provided on the first body 11. The cab 12 can be installed on the first body 11 through an elastic coupling 13. The first body 11 is one of the front body and the rear body. The second articulated structure segment 2 includes a second body 21. The second body 21 is the other of the front body and the rear body.

[0073] When the roller is running, the second articulated structure section 2 can swing in the horizontal plane relative to the first articulated structure section 1 where the cab 12 is located. If the first vehicle body 11 is the front vehicle body, the cab 12 is installed on the front vehicle body, and when the roller is running, the rear vehicle body can swing in the horizontal plane relative to the cab 12; or the cab 12 can also be installed on the rear vehicle body. Figure 1 As shown, a front wheel 4 is provided at the front side of the cab 12, and the front wheel 4 may be a steel wheel. A rear wheel 5 is provided at the rear side of the second vehicle body 21, and the rear wheel 5 may be a steel wheel or a tire.

[0074] The panoramic surround view system may include: a plurality of shooting components 6 , an image processing component 8 and a human-computer interaction component 9 .

[0075] Among them, multiple shooting components 6 are installed on the first articulated structure section 1. For example, the multiple shooting components 6 can all be installed on the cab 12, or can be installed on the cab 12 and the first vehicle body 11 at the same time. The multiple shooting components 6 are configured to shoot the environment around the articulated roller, including the vehicle body, the ground and the surrounding conditions. For example, the shooting component 6 can be installed on the outside of the cab 12 or the first vehicle body 11. The shooting component 6 can be a camera, such as a panoramic camera, a wide-angle camera or a fisheye camera.

[0076] The image processing component 8 is electrically connected to the plurality of shooting components 6, and is configured to receive images shot by the plurality of shooting components 6, which may be pictures or videos, and stitch them into a 360° surround image around the first articulated structure segment 1. Thus, the image processing component 8 performs distortion correction, perspective transformation, and panoramic stitching and fusion processing on the images shot by the plurality of shooting components to generate a surround image, thereby obtaining a real-time image of the periphery of the first articulated structure segment 1. For example, the image processing component 8 may be a DSP, a PLC controller, etc.

[0077] The human-machine interaction component 9 is configured to display the surround image formed by the image processing component 8, so that the driver can see the surrounding environment of the roller more clearly, providing a basis for controlling the movement and operation of the roller. For example, the human-machine interaction component 9 can be a display or a touch screen.

[0078] In this embodiment, multiple shooting components 6 are installed on the first articulated structure segment 1. When the second articulated structure segment 2 swings relative to the first articulated structure segment 1 during the movement of the roller, the shooting component 6 can clearly capture the position change of the second articulated structure segment 2 and the situation outside the cab 12, making image stitching easier and optimizing the image stitching effect, so that the surround view image can more clearly reflect the situation around the vehicle body and prevent visual blind spots. This can provide the operator with a more intuitive and clear basis for judgment when the roller is in various complex working conditions. When the roller is compacting against the roadside, the operator can also clearly observe the situation on the side in the cab, which can prevent pressure leakage problems. In addition, there is no need to install an angle sensor to detect the swing angle of the vehicle structure to achieve image stitching, making the structure of the panoramic surround view system simpler and easier to implement.

[0079] If multiple shooting components 6 are placed around the entire vehicle body, the shooting components 6 will swing significantly when the vehicle body structure swings, causing a split effect between the images shot by the multiple shooting components 6. It is also necessary to install an angle sensor to detect the swing angle in order to continuously correct the stored vehicle model according to the actual situation. The implementation is complicated, and the storage capacity of vehicle models at various angles is also large. The final implementation process is complicated and costly. If the angle sensor is damaged, the switching of the vehicle model and the image stitching during articulation cannot be completed. Even if other compensation algorithms such as visual recognition are used, the camera itself will shift with the vehicle body, causing the camera's view of the vehicle to shift, and ultimately a panoramic view close to the real thing cannot be obtained.

[0080] like Figure 1As shown, the plurality of photographing components 6 are all located on the same side of the articulated frame 3 along the length direction of the articulated roller. For example, if the cab 12 is located on the front body, the plurality of photographing components 6 are all located on the front side of the articulated frame 3. This arrangement enables the entire rear body to be completely within the photographing range of the photographing component 6 close to the rear body, and is not easily blocked, so that the panoramic image can more clearly reflect the environment around the first articulated structure section 1 and prevent visual blind spots.

[0081] In some embodiments, a plurality of photographing components 6 are disposed around the outer circumference of the cab 12 or the outer circumference of the first articulated structure segment 1 .

[0082] If the length of the first vehicle body 11 is short, since the cab 12 is high, a wider field of view can be obtained by disposing a plurality of photographing components 6 around the periphery of the cab 12 .

[0083] If the length of the first vehicle body 11 is relatively large, for example, the first vehicle body 11 is the rear vehicle body, and two tires are installed at the bottom of the rear vehicle body, after the cab 12 of this type of roller is installed on the side of the rear vehicle body close to the articulated frame 3, there is still a long distance behind the cab 12. If all the shooting components 6 are arranged on the cab 12, the blind area of ​​the rear shooting components 6 blocked by the rear vehicle body will be too large. If the rear shooting components 6 are installed at the end of the first vehicle body 11 away from the articulated frame 3, the blind area blocked by the vehicle body can be avoided.

[0084] like Figure 2 As shown, the plurality of photographing components 6 are all disposed on the cab 12, and the plurality of photographing components 6 include four photographing components 6, which are respectively disposed at the front, rear, left and right sides of the cab 12 to respectively capture images of the environments in front, rear, left and right sides of the cab 12. For example, the four photographing components 6 may be respectively disposed at the middle positions of the front, rear, left and right sides of the cab 12. This arrangement enables the conditions of the cab 12 in all directions to be clearly reflected in the surround view image.

[0085] In some embodiments, a plurality of photographing components 6 are disposed on the top of the cab 12, and the photographing surface of the photographing components 6 faces the outside of the cab 12 and is obliquely downward, so that the photographing components 6 can capture both partial images of the vehicle body and images around the cab. For example, when a fisheye camera is used, a range of 190° can be captured in the horizontal direction and a range of 140° can be captured in the vertical direction.

[0086] In some embodiments, with respect to the manner in which multiple shooting components 6 are arranged around the periphery of the first articulated structure segment 1, the multiple shooting components 6 include four shooting components 6, which are respectively arranged on a side of the cab 12 close to the articulated frame 3, a left side of the cab 12, a right side of the cab 12, and a side of the first vehicle body 11 away from the articulated frame 3.

[0087] In some embodiments, Figure 6 As shown, the panoramic view system may further include: a travel control component 10, which is electrically connected to the image processing component 8 and is configured to control the travel direction and travel speed of the articulated roller and transmit the travel direction and travel speed information to the image processing component 8. For example, the travel control component 10 may be a DSP, a PLC controller, etc.

[0088] The image processing component 8 is configured to move the surround image displayed in the human-machine interaction component 9 up and down according to the traveling direction of the road roller, and the up and down movement here refers to moving up and down in the display area of ​​the human-machine interaction component 9. When the traveling direction is forward, the surround image is moved downward to increase the display range in front of the road roller; when the traveling direction is backward, the surround image is moved upward to increase the display range behind the road roller, and the speed of the surround image movement and the position where it stops in the display area of ​​the human-machine interaction component 9 are determined according to the traveling speed, so that the display change of the surround image in the human-machine interaction component 9 follows the traveling process of the road roller and the displayed range meets the requirements of the safe braking distance range at the current speed. The running speed of the vehicle is proportional to the final braking safety distance of the vehicle, and the area displayed in the picture is proportional to the actual space in front of or behind the actual vehicle, that is, the faster the speed, the more safety areas should be displayed in the picture, for example, from 3m to 10m linearly with the speed.

[0089] This embodiment can make the vehicle model closer to the direction opposite to the travel direction, so that the upward image is more presented in the human-computer interaction component 9, so that the operator can observe the environmental conditions in front of the roller's travel direction in a wider range, so as to make advance judgments based on the roller's travel conditions and take measures in advance when special circumstances arise.

[0090] Moreover, since the spliced ​​surround view image itself is larger than the image displayed in the human-computer interaction component 9, when the entire picture moves up and down, only the pre-spliced ​​image is moved into the display area for display, and the new image is not completely re-spliced ​​and then displayed, so it will not affect the coordination of the overall picture, and the spliced ​​image will not be deformed.

[0091] Based on the above embodiments, FIG. 3A to FIG. 3D The panoramic surround view system further includes a plurality of calibration devices 7, which are arranged on a preset reference plane outside the cab 12 and around the periphery of the cab 12 or the first articulated structure segment 1. The top surfaces of the plurality of calibration devices 7 are all lower than the installation height of the photographing components 6 at the corresponding positions, so that the top surfaces of the calibration devices 7 are in the mid-air above the reference plane and below the photographing components 6. The plurality of calibration devices 7 are configured to calibrate the photographed images before the plurality of photographing components 6 are formally used. For example, the reference plane may be the ground or a platform.

[0092] For example, the height of the calibration device 7 can be close to the height of the front wheel 4 so that the proportion of the vehicle body in the displayed image is appropriate. If the height of the calibration device 7 is too high, the proportion of the vehicle body in the displayed image is very small, and obstacles around the vehicle cannot be observed clearly and intuitively; if the height of the calibration device is too low, the proportion of the vehicle body in the displayed image is very large, resulting in a very small visible area around the vehicle body, which cannot meet the safety distance requirements required by the system.

[0093] When the panoramic surround view system is formally used in this embodiment, the image captured by the camera component 6 is first calibrated by the calibration device 7, so that the vehicle body and the surrounding scene can be adjusted to a more appropriate proportion in the surround view image, so as to be closer to the real God's perspective, so that the displayed image seen by the driver is more realistic and easier to identify the situation around the vehicle. This embodiment makes the top surface of the calibration device 7 at a suitable height in the air, so that the camera component 6 can obtain a clearer and more proportional image, thereby making the picture ratio of the surround view image more coordinated. In addition, when a person passes through the image splicing area, the blind spot for human recognition is minimized.

[0094] Specifically, FIG. 3A to FIG. 3D As shown, four photographing components 6 are all arranged on the cab 12, and four calibration devices 7 are arranged on a reference plane outside a preset reference frame K. The specific method is as follows:

[0095] 1. If Figure 3A , obtain the overall top-view model of the articulated roller;

[0096] 2. If Figure 3B , only the partial model of the area where the outer contour of the cab 12 is located is retained in the overall top-view model, where the outer contour of the cab 12 refers to the actual outer edge of the cab 12;

[0097] 3. If Figure 3C , find the actual area corresponding to the partial model in the actual vehicle body, and offset the actual area outward by a preset distance L, and mark a preset reference frame K on the reference plane, where the preset reference frame K is a rectangle;

[0098] 4. If Figure 3D , four calibration devices 7 are placed around the preset reference frame K on the reference plane, and are respectively set at the four corners of the preset reference frame K, and the projection of the inner vertex of the feature recognition object of the calibration device 7 on the reference plane coincides with the corner of the preset reference frame K.

[0099] In the captured image, the partial model of the area where the outer contour of the cab 12 is located does not change much, and the real vehicle body part actually captured by the shooting component 6 will be highly affected by the calibration device 7 because the calibration device 7 is directly observed by the shooting component 6.

[0100] In some embodiments, Figure 4 As shown, the human-computer interaction component 9 is configured to receive a calibration instruction from an operator to move the calibration cursor X to a feature recognition object in the calibration device 7, and send the calibration instruction to the image processing component 8; the image processing component 8 is configured to receive the calibration instruction and correct the distortion in the images captured by the multiple shooting components 6 according to the actual parameters of the multiple calibration devices 7.

[0101] For some shooting components 6 such as fisheye cameras, in order to capture a larger range, objects in the captured image will be distorted according to the principle of larger objects appearing closer and smaller objects appearing farther away. That is, some lines of the objects will change from straight lines to curves. This embodiment can correct the captured image through the characteristic markers in the calibration device 7, so that a more realistic image can be displayed in the human-computer interaction component 9 for the operator to observe.

[0102] In some embodiments, the actual parameters of the multiple calibration devices 7 include: the distance between two adjacent calibration devices 7 and the outer dimensions of the feature identifier in a single calibration device 7, and the feature identifier is configured for the operator to perform calibration through the calibration cursor X in the human-computer interaction component 9.

[0103] For example, a calibration device 7 is arranged at each of the four corners of the preset reference frame K. The outer contour of the characteristic marker of the calibration device 7 is a rectangle. The actual parameters of the multiple calibration devices 7 include: the distance L1 between the left and right calibration devices 7 in the front or rear, the distance L2 between the front and rear calibration devices 7 on the left or right side, the length dimension L3 of the characteristic marker, and the width dimension L4 of the characteristic marker.

[0104] After the calibration device 7 is placed, actual parameters are input through the human-machine interaction component 9 to adapt to different vehicle models. The actual parameters of multiple calibration devices can be obtained by manual measurement or by arranging visual recognition components, such as arranging cameras, and the focal length, installation position and distance from the calibration device 7 and the shooting component 6 of the visual recognition component are known.

[0105] In some embodiments, a plurality of calibration devices 7 are arranged around the outside of the preset reference frame K, the calibration devices 7 are located outside the preset reference frame K as a whole, and the projection of the characteristic markers of the calibration devices 7 on the reference plane falls on the preset reference frame K, and the preset reference frame K is offset outward by a preset distance L as a whole relative to the area where the outer contour of the cab 12 or the first articulated structure segment 1 is located, and the preset distance L can be 1m, or can also be set in combination with the requirements of safe work. For example, the area where the outer contour of the cab 12 is located is a rectangular area. In combination with the display of the human-machine interaction component 9, the shooting angles of the plurality of shooting components 6 are configured to be adjusted to be able to shoot the range between the body of the articulated roller and the preset reference frame K.

[0106] This embodiment enables the photographing component 6 to fully photograph the area between the vehicle body and the preset reference frame K, thereby improving the safety of the roller during travel and operation. Moreover, the proportion of the vehicle body in the display image can be made appropriate.

[0107] In some embodiments, a plurality of calibration devices 7 are disposed outside a preset reference frame K, and the preset reference frame K is offset outward by a preset distance L as a whole relative to a rectangular area where the cab 12 or the outer contour of the first articulated structure segment 1 is located.

[0108] like Figure 3D As shown, four calibration devices 7 are provided, which are respectively arranged at the four corners of the preset reference frame K.

[0109] In some embodiments, the height of the calibration device 7 is adjustable. By adjusting the height of the height calibration device 7, the plane distortion at the height of the frame 74 in the calibration device 7 can be minimized. If the calibration device 7 is too close to the shooting component 6, more scenes outside the vehicle body can be seen after calibration, but the proportion of the vehicle body in the displayed image is very small, and obstacles around the vehicle cannot be clearly and intuitively observed; if the calibration device 7 is too far from the shooting component 6, the scenes outside the vehicle body can be seen after calibration. The proportion of the vehicle body in the displayed image is large, resulting in a very small visible area around the vehicle body, which cannot meet the safety distance requirements required by the system. Therefore, adjusting the calibration device 7 to a suitable height can make the proportion of the vehicle body and the environment outside the vehicle body in the image appropriate, and minimize the distortion of the plane where the vehicle body focus area is located after calibration.

[0110] In some embodiments, the human-machine interaction component 9 is configured to receive image revision parameters of at least some of the plurality of shooting components 6 input from outside, so that the surround images are at the same height. Thus, the operator can calibrate the shooting components 6 and set key parameters.

[0111] Since it is difficult to achieve completely consistent installation heights for the various shooting components 6, by observing the surround image, the operator can input image revision parameters to change the depth of image display for the shooting components 6 with mismatched image heights, so that the transition between adjacent images is smooth, the surround image is at the same height, and the image quality is optimized.

[0112] In some embodiments, Figure 5 As shown, the calibration device 7 includes: a base 71, a first support frame 72, a second support frame 73 and a frame. Among them, the first end of the first support frame 72 is fixed on the base 71; the second support frame 73 is located above the first support frame 72, and the first end of the second support frame 73 is connected to the second end of the first support frame 72 and the height is adjustable; the frame 74 is connected to the second end of the second support frame 73, and the frame 74 is used as a feature identification object when the operator calibrates in the human-machine interaction component 9, for example, the frame 74 can be a rectangular frame.

[0113] Specifically, Figure 5 As shown, the base 71 includes two cross beams 711 arranged in a cross shape; the first support frame 72 adopts a vertical pole, and the first end of the vertical pole is connected to the intersection of the two cross beams 711; the second support frame 73 also adopts a vertical pole, and the first end of the second support frame 73 can be inserted into the second end of the first support frame 72, or be mounted outside the second end of the first support frame 72, and the height is adjustable; the frame 74 includes a rectangular frame formed by four beams 741, and the reinforcing beam 742 is connected between the two opposite beams 741 and is located in the middle position of the beam 741 along the length direction, and the second end of the second support frame 73 is connected to the reinforcing beam 742.

[0114] The size of the frame 74 is known, and as a characteristic marker, it is convenient for the operator to calibrate the shooting component 6 by moving the marking cursor X to the outer frame of the frame 74. Furthermore, the size of the base 71 is larger than the size of the frame 74, which can prevent the calibration device 7 from tipping over during the calibration process, thereby improving the safety and efficiency of the calibration.

[0115] Secondly, the present disclosure also provides an articulated roller, comprising: a first articulated structure section 1 and a second articulated structure section 2 connected by an articulated frame 3, and a panoramic view system for an articulated roller of the above embodiment. The first articulated structure section 1 comprises a first vehicle body 11 and a cab 12 provided on the first vehicle body 11, the first vehicle body 11 being one of the front vehicle body and the rear vehicle body, and the second articulated structure section 2 comprises a second vehicle body 21, the second vehicle body 21 being the other of the front vehicle body and the rear vehicle body.

[0116] Secondly, the present disclosure also provides a calibration method for a panoramic view system of an articulated roller based on the above embodiment, which in some embodiments includes:

[0117] Step 101, in combination with the display of the human-computer interaction component 9, the shooting angles of the multiple shooting components 6 are adjusted to be able to capture the range between the body of the articulated roller and the preset reference frame K, and the preset reference frame K is offset outward by a preset distance L relative to the area where the outer contour of the cab 12 or the first articulated structure segment 1 is located; for example, the preset reference frame K is a rectangle.

[0118] Step 102, arrange multiple calibration devices 7 around the preset reference frame K on the reference plane; wherein, when the calibration device 7 is arranged at the corner of the preset reference frame K, the projection of the inner side of the vertex of the characteristic marker in the calibration device 7 in the reference plane may coincide with the corner of the preset reference frame K; when the calibration device 7 is arranged at the edge of the preset reference frame K, the projection of the inner side of the characteristic marker in the calibration device 7 in the reference plane may coincide with the edge of the preset reference frame K.

[0119] Step 103 : Move the calibration cursor X in the human-machine interaction component 9 to the feature recognition object of the calibration device 7 , so as to calibrate the captured images before the multiple shooting components 6 are officially used.

[0120] Among them, steps 101-103 can be performed sequentially. This embodiment can adjust the shooting angles of multiple shooting components 6 and calibrate the captured images, so that the shooting components 6 can fully capture the area between the vehicle body and the preset reference frame K, thereby improving the safety of the roller during travel and operation. Moreover, the vehicle body and the surrounding scene can be adjusted to a more appropriate proportion in the surround view image, so as to be closer to the real God's perspective, so that the displayed image seen by the driver is more realistic and easier to identify the situation around the vehicle.

[0121] In some embodiments, the calibration method of the present disclosure further includes:

[0122] Step 104: after the surround view image is formed by stitching, image revision parameters of at least some of the plurality of shooting components 6 are set through the human-computer interaction component 9 according to the degree of height matching of each part of the surround view image, so that the surround view image is at the same height.

[0123] Step 104 may be performed after step 103. Since it is difficult to achieve completely consistent installation heights of the various photographing components 6, by observing the surround image, the operator may input image revision parameters to change the image height of the photographing components 6 whose image heights do not match, so that the transition between adjacent images is smooth, the surround images are at the same height, and the image quality is optimized.

[0124] In some embodiments, step 102 of arranging a plurality of calibration devices 7 on the reference plane around a preset reference frame K includes:

[0125] Step 102A: Obtain an overall top-view model of the articulated roller, such as Figure 3A As shown;

[0126] Step 102B: Only the partial model of the area where the cab 12 or the outer contour of the first articulated structure segment 1 is located is retained from the overall top view model. When multiple shooting components 6 are all provided on the cab 1, only the partial model of the area where the outer contour of the cab 12 is located is retained; when multiple shooting components 6 are provided on both the cab 1 and the first vehicle body 11, only the partial model of the area where the outer contour of an articulated structure segment 1 is located is retained.

[0127] Step 102C: Find the actual area corresponding to the partial model in the actual vehicle body, and offset the actual area outward by a preset distance L as a whole, and mark a preset reference frame K on the reference plane, for example, mark the preset reference frame K on the ground or on a platform;

[0128] Step 102D, arrange multiple calibration devices 7 around the preset reference frame K on the reference plane, the characteristic markers of the calibration devices 7 are located outside the preset reference frame K as a whole, and the projection parts on the reference plane fall on the preset reference frame K, for example, the vertices or inner sides of the characteristic markers may fall on the preset reference frame K.

[0129] In some embodiments, step 102 of arranging a plurality of calibration devices 7 on the reference plane around a preset reference frame K includes:

[0130] There are four calibration devices 7, which are respectively arranged at the four corners of the preset reference frame K;

[0131] In some embodiments, before calibrating the captured image in step 103, the calibration method of the present disclosure further includes:

[0132] Step 105: inputting actual parameters of multiple calibration devices 7 through the human-computer interaction component 9, so that the image processing component 8 can correct the distortion in the images captured by the multiple capturing components 6 to achieve calibration;

[0133] Among them, the actual parameters of the multiple calibration devices 7 include the distance between two adjacent calibration devices 7 and the outer dimensions of the feature identification object in a single calibration device 7 , and the feature identification object is configured for the operator to calibrate in the human-machine interaction component 9 .

[0134] This embodiment inputs the actual parameters of multiple calibration devices 7, and can use the characteristic identifiers of the calibration devices 7 as targets in the actual captured images, and correct the distortion of the actual images according to their actual parameters, so that the human-computer interaction component 9 presents a more realistic view, which is convenient for the operator to observe and improves the safety of the roller during travel and operation.

[0135] In some embodiments, after arranging multiple calibration devices 7 around the preset reference frame K on the reference plane in step 102 and before step 103, the calibration method of the present disclosure further includes:

[0136] Step 106: Adjust the heights of the plurality of calibration devices 7 to adjust the proportion of the body of the articulated roller in the surround view image after calibration. For example, the height of the calibration device 7 can be adjusted automatically.

[0137] By adjusting the height of the height calibration device 7, the plane distortion at the height of the calibration device 7 can be minimized. If the calibration device 7 is too close to the shooting component 6, more scenes outside the vehicle body can be seen after calibration; if the calibration device 7 is too far from the shooting component 6, fewer scenes outside the vehicle body can be seen after calibration. Therefore, by adjusting the calibration device 7 to a suitable height, the proportion of the vehicle body and the environment outside the vehicle body in the image can be appropriate.

[0138] In some embodiments, the plurality of photographing components 6 include four photographing components 6, which are respectively disposed at the front, rear, left and right sides of the cab 12, and a calibration device 7 is provided at each of the four corners of the preset reference frame K.

[0139] Based on this arrangement, when the articulated frame 3 is located in front of the cab 12, adjusting the heights of the plurality of calibration devices 7 in step 106 includes:

[0140] Step 106A, adjusting the height of the calibration device 7 at the left front and right front positions to a height difference with the vehicle body structure that does not exceed a first preset height, that is, the height is relatively close and lower than the installation height of the front shooting component 6;

[0141] Step 106B, adjust the height of the calibration device 7 at the left rear and right rear positions to be lower than the installation height of the rear shooting component 6, and the height difference with the calibration device 7 at the left front and right front positions does not exceed a second preset height.

[0142] Or in the case where the articulated frame 3 is located at the rear side of the cab 12, adjusting the heights of the plurality of calibration devices 7 in step 106 includes:

[0143] Step 106C, adjusting the height of the calibration device 7 at the left rear and right rear positions to a height difference with the vehicle body structure that does not exceed a third preset height and is lower than the installation height of the rear shooting component 6;

[0144] Step 106D, adjust the height of the calibration device 7 at the left front and right front positions to be lower than the installation height of the front shooting component 6, and the height difference with the calibration device 7 at the left rear and right rear positions does not exceed a fourth preset height.

[0145] This embodiment enables the shooting component 6 close to the articulated frame 3 to clearly and comprehensively shoot the vehicle body structure, and makes the entire vehicle body occupy an appropriate proportion in the captured image. If the calibration device 7 is too close to the shooting component 6, the vehicle body occupies a small proportion in the displayed image, and it is impossible to clearly and intuitively observe the obstacles around the vehicle; if the calibration device 7 is too far from the shooting component 6, the vehicle body occupies a large proportion in the displayed image, resulting in a very small visible area around the vehicle body, which cannot meet the safety distance requirements required by the system. Therefore, adjusting the calibration device 7 to a suitable height can make the vehicle body and the environment outside the vehicle body occupy an appropriate proportion in the image, and minimize the distortion of the plane where the vehicle body is located after calibration.

[0146] The above-mentioned embodiment of the present disclosure collects the original image of the vehicle's surroundings through the shooting component 6 installed on the roller cab, the operator completes the calibration through the human-machine interaction component 9 in combination with the calibration device 7, and the image processing component obtains a complete surround image of the vehicle based on the calibration information. The shooting components 6 are all installed in the articulated section where the cab 12 is located. The special calibration device 7 can be used to adjust the height of the characteristic marker from the ground according to the needs of the vehicle. The articulated frame 3 and the vehicle body structure will be converted into a bird's-eye view and presented in the surround image, ultimately realizing the God's perspective of the panoramic system of the articulated roller with no blind spots, and the surround area displayed in the human-machine interaction component 9 is controlled to move up and down according to the walking direction and walking speed information provided by the walking control component 10.

[0147] The embodiments provided by the present disclosure are described in detail above. Specific embodiments are used herein to illustrate the principles and implementation methods of the present disclosure, and the description of the above embodiments is only used to help understand the methods and core ideas of the present disclosure. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of the present disclosure, several improvements and modifications can be made to the present disclosure, and these improvements and modifications also fall within the scope of protection of the claims of the present disclosure.

Claims

1. A panoramic view system for an articulated roller, characterized in that: The articulated roller comprises a first articulated structure section (1) and a second articulated structure section (2) connected by an articulated frame (3), the first articulated structure section (1) comprising a first body (11) and a cab (12) arranged on the first body (11), the first body (11) being one of a front body and a rear body, the second articulated structure section (2) comprising a second body (21), the second body (21) being the other of the front body and the rear body, the panoramic surround view system comprising: a plurality of photographing components (6) mounted on the first articulated structure section (1) and configured to photograph the environment around the articulated roller, and the plurality of photographing components (6) are all located on the same side of the articulated frame (3) along the length direction of the articulated roller, so that when the second articulated structure section (2) swings relative to the first articulated structure section (1) during the movement of the roller, the position change of the second articulated structure section (2) and the situation outside the cab (12) can be clearly photographed; An image processing component (8) is configured to receive the images captured by the plurality of capturing components (6) and stitch them together into a panoramic image of the entire circumference of the first articulated structure segment (1); and The human-computer interaction component (9) is configured to display the surround image spliced ​​by the image processing component (8).

2. The panoramic view system for an articulated roller according to claim 1, characterized in that: The plurality of photographing components (6) are arranged around the outer circumference of the cab (12) or the outer circumference of the first articulated structure section (1).

3. The panoramic view system for an articulated roller according to claim 1, characterized in that: The plurality of photographing components (6) include four photographing components (6), which are respectively arranged at the front, rear, left and right sides of the cab (12).

4. The panoramic view system for an articulated roller according to claim 3, characterized in that: The plurality of photographing components (6) are arranged on the top of the cab (12), and the photographing surfaces of the photographing components (6) face the outside of the cab (12) and are obliquely downward.

5. The panoramic view system for an articulated roller according to claim 1, characterized in that: The plurality of shooting components (6) include four shooting components (6), which are respectively arranged on a side of the cab (12) close to the articulated frame (3), a left side of the cab (12), a right side of the cab (12) and a side of the first vehicle body (11) away from the articulated frame (3).

6. The panoramic view system for an articulated roller according to claim 1, characterized in that: Also includes: a travel control component (10), electrically connected to the image processing component (8), configured to control the travel direction and travel speed of the articulated roller, and transmit the travel direction and travel speed information to the image processing component (8); The image processing component (8) is configured to move the surround image displayed in the human-computer interaction component (9) up and down according to the traveling direction, to move the surround image downward when the traveling direction is forward, and to move the surround image upward when the traveling direction is backward, and to determine the speed of movement of the surround image and the position where the image finally stops according to the traveling speed.

7. The panoramic view system for an articulated roller according to any one of claims 1 to 6, characterized in that: It also includes a plurality of calibration devices (7), which are arranged on a preset reference plane outside the cab (12) and around the periphery of the cab (12) or the first hinged structure section (1), wherein the top surface of each of the plurality of calibration devices (7) is lower than the installation height of the shooting component (6) at the corresponding position, and the plurality of calibration devices (7) are configured to calibrate the captured images before the plurality of shooting components (6) are officially used.

8. The panoramic view system for an articulated roller according to claim 7, characterized in that: The human-machine interaction component (9) is configured to receive a calibration instruction from an operator to move a calibration cursor (X) to a feature recognition object in the calibration device (7), and to send the calibration instruction to the image processing component (8); the image processing component (8) is configured to receive the calibration instruction and to correct the distortion in the images captured by the multiple capturing components (6) according to actual parameters of the multiple calibration devices (7).

9. The panoramic view system for an articulated roller according to claim 8, characterized in that: The actual parameters of the multiple calibration devices (7) include: the distance between two adjacent calibration devices (7) and the outer dimensions of a feature identifier in a single calibration device (7), wherein the feature identifier is configured for an operator to perform calibration in the human-machine interaction component (9) through the calibration cursor (X).

10. The panoramic view system for an articulated roller according to claim 7, characterized in that: The plurality of calibration devices (7) are arranged outside a preset reference frame (K), and the preset reference frame (K) is offset outward by a preset distance (L) relative to a rectangular area formed by an outer contour of the cab (12) or the first articulated structure section (1); In combination with the display of the human-machine interaction component (9), the shooting angles of the multiple shooting components (6) are configured to be adjusted to be able to shoot a range between the body of the articulated roller and the preset reference frame (K).

11. The panoramic view system for an articulated roller according to claim 7, characterized in that: The plurality of calibration devices (7) are arranged outside a preset reference frame (K), and the preset reference frame (K) is offset outward by a preset distance (L) relative to the area where the outer contour of the cab (12) is located; The multiple calibration devices (7) are respectively arranged at the four corners of the preset reference frame (K).

12. The panoramic view system for an articulated roller according to claim 7, characterized in that: The height of the calibration device (7) is adjustable.

13. The panoramic view system for an articulated roller according to claim 7, characterized in that: The human-machine interaction component (9) is configured to receive externally input image revision parameters of at least some of the plurality of photographing components (6), so as to place the surround view images at the same height.

14. The panoramic view system for an articulated roller according to claim 7, characterized in that: The calibration device (7) comprises: Base (71); A first support frame (72), wherein a first end of the first support frame (72) is fixed on the base (71); A second support frame (73) is located above the first support frame (72), wherein a first end of the second support frame (73) is connected to a second end of the first support frame (72) and the height of the second support frame (73) is adjustable; and A frame (74) is connected to the second end of the second support frame (73), and the frame (74) serves as a feature identification object when an operator marks the human-machine interaction component (9).

15. An articulated roller, characterized in that: include: A first articulated structure section (1) and a second articulated structure section (2) connected by an articulated frame (3) and a panoramic view system for an articulated roller according to any one of claims 1 to 14; The first articulated structure section (1) comprises a first vehicle body (11) and a cab (12) arranged on the first vehicle body (11), the first vehicle body (11) being one of a front vehicle body and a rear vehicle body, and the second articulated structure section (2) comprises a second vehicle body (21), the second vehicle body (21) being the other of the front vehicle body and the rear vehicle body.

16. A calibration method for a panoramic view system of an articulated roller based on any one of claims 1 to 14, characterized in that: include: In combination with the display of the human-machine interaction component (9), the shooting angles of the multiple shooting components (6) are adjusted to a range between the body of the articulated roller and a preset reference frame (K) capable of being photographed, wherein the preset reference frame (K) is offset outwardly by a preset distance (L) relative to the area where the cab (12) or the outer contour of the first articulated structure section (1) is located; Arranging a plurality of calibration devices (7) on a reference plane around the preset reference frame (K); The calibration cursor (X) in the human-machine interaction component (9) is moved onto the feature identification object of the calibration device (7) to calibrate the captured images before the multiple shooting components (6) are officially used.

17. The calibration method according to claim 16, characterized in that: Also includes: After the surround view image is formed by stitching, image revision parameters of at least some of the plurality of shooting components (6) are set through the human-computer interaction component (9) according to the degree of height matching of the partial images in the surround view image, so that the surround view image is at the same height.

18. The calibration method according to claim 16, characterized in that: Arranging a plurality of calibration devices (7) on a reference plane outside the preset reference frame (K) comprises: Obtaining an overall top-view model of the articulated roller; Only a partial model of the area where the outer contour of the cab (12) or the first articulated structure section (1) is located is retained in the overall top-view model; Finding an actual area corresponding to the partial model in the actual vehicle body, offsetting the actual area outward by a preset distance (L), and marking the preset reference frame (K) on the reference plane; A plurality of calibration devices (7) are arranged on the reference plane around the preset reference frame (K).

19. The calibration method according to claim 16, characterized in that: Before calibrating the captured image, the calibration method further includes: Inputting actual parameters of the plurality of calibration devices (7) through the human-machine interaction component (9) so that the image processing component (8) can correct the distortion in the images captured by the plurality of capturing components (6) to achieve calibration; The actual parameters of the plurality of calibration devices (7) include the distance between two adjacent calibration devices (7) and the outer dimensions of a feature identifier in a single calibration device (7), and the feature identifier is configured for an operator to perform calibration in the human-machine interaction component (9).

20. The calibration method according to claim 16, characterized in that: After arranging a plurality of calibration devices (7) on a reference plane outside the preset reference frame (K), the calibration method further comprises: The heights of the multiple calibration devices (7) are adjusted to adjust the proportion of the body of the articulated roller in the surround view image after calibration.

21. The calibration method according to claim 20, characterized in that: The plurality of photographing components (6) include four photographing components (6), which are respectively arranged at the front, rear, left and right sides of the cab (12), and each of the four corners of the preset reference frame (K) is provided with a calibration device (7); wherein, When the articulated frame (3) is located at the front side of the cab (12), adjusting the height of the plurality of calibration devices (7) comprises: The height of the calibration device (7) at the left front and right front positions is adjusted to be close to the height of the vehicle body structure and lower than the installation height of the front shooting component (6); The height of the calibration device (7) at the left rear and right rear positions is adjusted to be lower than the installation height of the rear shooting component (6), and the height difference with the calibration device (7) at the left front and right front positions does not exceed a preset height; or When the articulated frame (3) is located at the rear side of the cab (12), adjusting the height of the plurality of calibration devices (7) comprises: The height of the calibration device (7) at the left rear and right rear positions is adjusted to be close to the vehicle body structure and lower than the installation height of the rear shooting component (6); The height of the calibration device (7) at the left front and right front positions is adjusted to be lower than the installation height of the front shooting component (6), and the height difference with the calibration device (7) at the left rear and right rear positions does not exceed a preset height.

Images