Image generation method and device

Abstract

The embodiment of the present application provides an image generation method and device for providing a more comprehensive and accurate driving image of a trailer to reduce driving risks. The method comprises: acquiring at least two images; wherein the at least two images are respectively acquired by at least two cameras, the at least two cameras are arranged on the trailer, and at least one of the at least two cameras is arranged on the trailer of the trailer; according to the articulation angle of the trailer, the at least two images are spliced ​​to obtain a target image.

Description

Technical Field

[0001] The present application relates to the field of intelligent driving, and specifically to an image generation method and device. Background Art

[0002] Since the relative angle between the tractor and the trailer (i.e. the articulation angle) will change constantly during the driving of the trailer, currently, when a camera is installed on the tractor, the trailer can only obtain a 270° image at most, and the driver cannot obtain the rear view image of the trailer, which poses a driving hazard. Summary of the invention

[0003] The embodiment of the present application provides an image generation method for providing a more comprehensive and accurate driving image of a trailer and reducing driving risks.

[0004] In a first aspect, an embodiment of the present application provides an image generation method, comprising: acquiring at least two images; wherein the at least two images are respectively captured by at least two cameras, the at least two cameras are arranged on a trailer, and at least one of the at least two cameras is arranged on a trailer of the trailer; according to the articulation angle of the trailer, splicing the at least two images to obtain a target image.

[0005] In this solution, by setting at least one camera on the trailer of the tow vehicle, the rear view image of the trailer can be obtained, thereby providing a more comprehensive driving image of the tow vehicle; by splicing the images collected by different cameras on the tow vehicle according to the articulation angle of the tow vehicle, the accuracy of the splicing can be improved, thereby improving the accuracy of the driving image of the tow vehicle. In summary, this solution can provide a more comprehensive and accurate driving image of the tow vehicle, thereby helping users reduce driving risks.

[0006] Optionally, the at least two cameras include a first camera, a second camera and a third camera; wherein the first camera is arranged at the rear of the trailer, and the second camera and the third camera are relatively arranged on both sides of the trailer.

[0007] Through this method, a first camera is set at the rear of the trailer to obtain a rear view image of the trailer, and a second camera and a third camera are also set on the towed vehicle to obtain images on both sides of the trailer, thereby obtaining a reliable driving image of the trailer and reducing driving risks.

[0008] Optionally, the second camera is disposed inside the articulation angle of the trailer. Accordingly, stitching the at least two images according to the articulation angle of the trailer includes: calculating a first angle according to the articulation angle of the trailer, the length of the trailer, the width of the trailer, and a first distance; the first angle is the angle between the field of view of the first camera and the field of view of the second camera, and the first distance is the distance from the midpoint of the connection line between the second camera and the third camera to the midpoint of the front of the trailer; stitching the image captured by the first camera and the image captured by the second camera according to the first angle. The stitched image may be a target image or a part of a target image, which is not limited in this application.

[0009] Through this method, the image captured by the first camera and the image captured by the second camera can be spliced, so that the spliced ​​image is coherent and there is no image in an overlapping area, thereby improving the accuracy of the spliced ​​image.

[0010] Optionally, the first angle satisfies the following relationship: Wherein, α1 is the first angle, G is the first distance, L is the length of the trailer, K is the width of the trailer, and β is the articulation angle of the towing vehicle.

[0011] It can be understood that the above formula is only an example and not a limitation, and can be modified or deformed according to needs in actual applications.

[0012] Optionally, the third camera is arranged outside the articulation angle of the trailer. Accordingly, stitching the at least two images according to the articulation angle of the trailer includes: calculating a second angle according to the articulation angle of the trailer; wherein the second angle is the angle between the field of view of the first camera and the field of view of the third camera; stitching the image captured by the first camera and the image captured by the third camera according to the second angle; wherein the second angle satisfies the following relationship: α2=90°-β; wherein α2 is the second angle, and β is the articulation angle of the trailer. The stitched image may be a target image or a part of a target image, which is not limited in the present application.

[0013] Through this method, the image captured by the first camera and the image captured by the third camera can be spliced, so that the spliced ​​image is coherent and there is no image in an overlapping area, thereby improving the accuracy of the spliced ​​image.

[0014] Optionally, the at least two cameras further include a fourth camera, and the fourth camera is arranged at the front of the tractor of the trailer. The spliced ​​image may be a target image or a part of a target image, which is not limited in the present application.

[0015] By this method, a fourth camera is set at the front of the tractor, and the images captured by the first, second, third and fourth cameras can be stitched together to obtain a more complete view around the trailer, thereby improving the practicality and reliability of the stitched images.

[0016] Optionally, the method further includes: calculating a blind spot and a vehicle model filling area according to the articulation angle of the tow vehicle, the length of the trailer, and the width of the trailer; and generating a panoramic image according to the target image, the blind spot, and the vehicle model filling area.

[0017] Through this method, the blind spot of the trailer and the filling area of ​​the vehicle model are taken into consideration when stitching images, so that the panoramic image can provide a relatively complete 360° scene centered on the trailer, which is convenient for the driver to drive according to the panoramic image, reduces driving risks and improves driving safety.

[0018] Optionally, the blind area satisfies the following relationship: The car model filling area satisfies the following relationship: Among them, S1 is the blind area, S2 is the filling area of ​​the vehicle model, L is the length of the trailer, K is the width of the trailer, and β is the articulation angle of the towing trailer.

[0019] It can be understood that the above formula is only an example and not a limitation, and can be modified or deformed according to needs in actual applications.

[0020] Optionally, the at least two cameras include a fifth camera and a sixth camera; wherein the fifth camera is arranged on the first side of the tractor, and the sixth camera is arranged on the first side of the trailer, and the first side of the tractor and the first side of the trailer are located on the first side of the trailer; and stitching the at least two images according to the articulation angle of the trailer comprises: calculating the offset angle of the stitching line according to the articulation angle of the trailer; and stitching the image captured by the fifth camera and the image captured by the sixth camera according to the offset angle of the stitching line.

[0021] In this way, two cameras are set on the trailer, the offset angle of the stitching line is calculated, and the images captured by the two cameras are stitched according to the offset angle of the stitching line, so that the stitched image is coherent and there is no image in the overlapping area, and the images are stitched according to the offset angle of the stitching line, so that the stitched image can be clear, and the usability of the stitched image is improved. Among them, the stitched image can be a target image, or a part of a target image, which is not limited in this application.

[0022] Optionally, the at least two cameras also include a seventh camera, an eighth camera, a ninth camera, and a tenth camera; wherein the seventh camera is arranged at the front of the tractor, the eighth camera is arranged at the second side of the tractor, and the ninth camera is arranged at the second side of the trailer, the second side of the tractor and the second side of the trailer are located on the second side of the towed vehicle, and the first side and the second side are opposite; and the tenth camera is arranged at the rear of the trailer.

[0023] Through this method, the images collected by the six cameras on the trailer are stitched together, and the stitched image is a 360° target image centered on the trailer, which can fully and clearly reflect the scene around the trailer, reducing driving risks and ensuring the safety of the driving process.

[0024] Optionally, the method further comprises: acquiring the articulation angle of the trailer by means of an angle sensor; wherein the angle sensor is disposed on the tractor.

[0025] Through this method, the articulation angle of the trailer is obtained through the angle sensor, so that the obtained articulation angle is accurate, thereby improving the availability of the target image.

[0026] Optionally, the method further includes: acquiring the length and the width of the trailer through a human-computer interaction interface.

[0027] In this way, the length and width of the trailer are obtained through the human-computer interaction interface, and when the towing vehicle replaces the trailer, the target image of the towing vehicle corresponding to the different trailers can be obtained, thereby improving the flexibility of the method.

[0028] In a second aspect, an embodiment of the present application provides an image generating device, which includes a module / unit / technical means for executing the method in the above-mentioned first aspect or any optional implementation manner of the first aspect.

[0029] Exemplarily, the device may include:

[0030] An acquisition module, configured to acquire at least two images; wherein the at least two images are respectively acquired by at least two cameras, the at least two cameras are arranged on a trailer, and at least one of the at least two cameras is arranged on a trailer of the trailer;

[0031] The processing module is used to stitch the at least two images according to the articulation angle of the trailer to obtain a target image.

[0032] Optionally, the at least two cameras include a first camera, a second camera and a third camera; wherein the first camera is arranged at the rear of the trailer, and the second camera and the third camera are relatively arranged on both sides of the trailer.

[0033] Optionally, the second camera is arranged on the inner side of the articulation angle of the trailer; the processing module is used to: calculate the first angle according to the articulation angle of the trailer, the length of the trailer, the width of the trailer, and the first distance; the first angle is the angle between the field of view of the first camera and the field of view of the second camera, and the first distance is the distance from the midpoint of the connecting line between the second camera and the third camera to the midpoint of the front of the trailer; splice the image captured by the first camera and the image captured by the second camera according to the first angle.

[0034] Optionally, the first angle satisfies the following relationship: Wherein, α1 is the first angle, G is the first distance, L is the length of the trailer, K is the width of the trailer, and β is the articulation angle of the towing vehicle.

[0035] Optionally, the third camera is arranged outside the articulation angle of the trailer; the processing module is used to: calculate a second angle according to the articulation angle of the trailer; wherein the second angle is the angle between the field of view of the first camera and the field of view of the third camera; stitching the image captured by the first camera and the image captured by the third camera according to the second angle; wherein the second angle satisfies the following relationship: α2=90°-β; wherein α2 is the second angle, and β is the articulation angle of the trailer.

[0036] Optionally, the at least two cameras further include a fourth camera, and the fourth camera is disposed at the front of the tractor of the trailer.

[0037] Optionally, the processing module is also used to: calculate the blind spot and the vehicle model filling area according to the articulation angle of the trailer, the length of the trailer, and the width of the trailer; and generate a panoramic image according to the target image, the blind spot, and the vehicle model filling area.

[0038] Optionally, the blind area satisfies the following relationship: The car model filling area satisfies the following relationship: Among them, S1 is the blind area, S2 is the filling area of ​​the vehicle model, L is the length of the trailer, K is the width of the trailer, and β is the articulation angle of the towing trailer.

[0039] Optionally, the at least two cameras include a fifth camera and a sixth camera; wherein the fifth camera is arranged on the first side of the tractor, and the sixth camera is arranged on the first side of the trailer, and the first side of the tractor and the first side of the trailer are located on the first side of the trailer; the processing module is used to: calculate the offset angle of the stitching line according to the articulation angle of the trailer; and stitch the image captured by the fifth camera and the image captured by the sixth camera according to the offset angle of the stitching line.

[0040] Optionally, the at least two cameras also include a seventh camera, an eighth camera, a ninth camera, and a tenth camera; wherein the seventh camera is arranged at the front of the tractor, the eighth camera is arranged at the second side of the tractor, and the ninth camera is arranged at the second side of the trailer, the second side of the tractor and the second side of the trailer are located on the second side of the towed vehicle, and the first side and the second side are opposite; and the tenth camera is arranged at the rear of the trailer.

[0041] Optionally, the acquisition module is further used to: acquire the articulation angle of the trailer through an angle sensor; wherein the angle sensor is arranged on the tractor.

[0042] Optionally, the acquisition module is further used to: acquire the length and the width of the trailer through a human-computer interaction interface.

[0043] According to a third aspect, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the at least one processor executes the instructions stored in the memory, thereby enabling the at least one processor to execute the method described in the first aspect or any optional embodiment of the first aspect.

[0044] Optionally, the electronic device further includes: at least two cameras for capturing at least two images.

[0045] In a fourth aspect, the present application provides a trailer, comprising the image generating device of the second aspect, or comprising the electronic device of the third aspect.

[0046] In a fifth aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by a computer, the computer executes any one of the methods described in the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 A schematic diagram of a trailer;

[0048] Figure 2 Schematic diagrams of several trailers of different sizes;

[0049] Figure 3 A schematic diagram of a trailer provided in an embodiment of the present application;

[0050] Figure 4 A flowchart of an image generation method provided in an embodiment of the present application;

[0051] Figure 5 A schematic diagram of another trailer provided in an embodiment of the present application;

[0052] Figure 6 A schematic diagram of another trailer provided in an embodiment of the present application;

[0053] Figure 7 A schematic diagram of another trailer provided in an embodiment of the present application;

[0054] Figure 8 A schematic diagram of another trailer provided in an embodiment of the present application;

[0055] Fig. 9 A schematic diagram of a vehicle-mounted display screen provided in an embodiment of the present application;

[0056] Fig.10 An image generating device 1000 is provided for an embodiment of the present application;

[0057] Fig.11 An electronic device 1100 is also provided for the embodiment of the present application. DETAILED DESCRIPTION

[0058] In order to make the purpose, technical scheme and advantages of the present application clearer, the technical scheme in the embodiment of the present application will be clearly and completely described below in conjunction with the drawings in the embodiment of the present application. Obviously, the described embodiment is only a part of the embodiment of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application. In the absence of conflict, the embodiments in the present application and the features in the embodiments can be arbitrarily combined with each other. In addition, although the logical order is shown in the flow chart, in some cases, the steps shown or described can be performed in an order different from that here.

[0059] The terms "first" and "second" in the specification and claims of the present application and the above-mentioned drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the term "comprising" and any of their variations are intended to cover non-exclusive protection. For example, a process, method, system, product or device comprising a series of steps or units is not limited to the listed steps or units, but optionally also includes steps or units that are not listed, or optionally also includes other steps or units inherent to these processes, methods, products or devices. "Multiple" in the present application can mean at least two, for example, two, three or more, and the embodiments of the present application are not limited.

[0060] The term "and / or" in the embodiments of the present application is only a description of the association relationship of the associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0061] In order to facilitate understanding of the solutions of the embodiments of the present application, possible application scenarios of the embodiments of the present application are first introduced below.

[0062] See also Figure 1 , is a schematic diagram of a trailer; the trailer includes a tractor and a trailer, the difference between the heading angle of the tractor and the heading angle of the trailer is the articulation angle, and the articulation angle is an acute angle. When the trailer is driving, there is an articulation angle. A camera is set at the front, left front, and right front of the tractor, respectively, that is, the trailer can only obtain a 270° image, and cannot obtain the rear view image of the trailer, which makes the trailer have certain driving risks.

[0063] Furthermore, in practical applications, the trailers of tow trucks are frequently replaced and the size of the trailers varies greatly, such as Figure 2 As shown, it is difficult to stitch the rear view image of the trailer with other images under different trailers.

[0064] In view of this, a technical solution of an embodiment of the present application is provided to provide a more comprehensive and accurate driving image of a trailer and reduce driving risks.

[0065] See also Figure 3 , is a schematic diagram of a trailer provided in an embodiment of the present application, wherein at least one camera ( Figure 3 Only one camera at the rear of the trailer is shown, but the invention is not limited thereto). Figure 1 The trailer shown can obtain images from more perspectives and reduce driving risks.

[0066] Reference Figure 4, is a flowchart of an image generation method provided in an embodiment of the present application, the method is applied to Figure 3 Take the scenario shown as an example.

[0067] The method can be executed by various devices with computing capabilities, which can be set on the trailer, such as an intelligent driving controller, or an intelligent cockpit controller, or a controller integrating multiple functions such as driving control, cockpit control, or body control; or, the device can be set outside the trailer, such as a background server of a vehicle-mounted application.

[0068] Taking the method executed by an intelligent driving controller as an example, the method includes:

[0069] S401: Acquire at least two images.

[0070] The at least two images are collected by at least two cameras respectively, the at least two cameras are arranged on the trailer, and at least one of the at least two cameras is arranged on the trailer of the trailer. The intelligent driving controller receives at least two images from the at least two cameras.

[0071] In one possible embodiment, at least two cameras include a first camera, a second camera and a third camera; wherein the first camera is arranged at the rear of the trailer, the second camera and the third camera are arranged on both sides of the trailer relatively, the second camera is arranged on the inner side of the articulation angle of the trailer, and the third camera is arranged on the outer side of the articulation angle of the trailer.

[0072] For a specific example, see Figure 5 , the first camera is A1, the second camera is A2, and the third camera is A3.

[0073] Optionally, a fourth camera may be provided at the front of the tractor. Figure 5 , a fourth camera A4 is arranged at the front of the tractor.

[0074] In one possible embodiment, at least two cameras include a fifth camera and a sixth camera; wherein the fifth camera is arranged on the first side of the tractor, and the sixth camera is arranged on the first side of the trailer, and the first side of the tractor and the first side of the trailer are located on the first side of the towed vehicle.

[0075] For a specific example, see Figure 6 When the first side is the left side of the trailer, the fifth camera is B1 and the sixth camera is B2; when the first side is the right side of the trailer, the fifth camera is B4 and the sixth camera is B5.

[0076] Optionally, a seventh camera may be arranged at the front of the tractor, an eighth camera may be arranged at the second side of the tractor, and a ninth camera may be arranged at the second side of the trailer. The second side of the tractor and the second side of the trailer are located on the second side of the towed vehicle, the first side and the second side are opposite to each other, and a tenth camera may be arranged at the rear of the trailer.

[0077] Exemplarily, when the first side is the left side of the trailer, the second side is the right side of the trailer, the fifth camera is B1, the sixth camera is B2, the seventh camera is B3, the eighth camera is B4, the ninth camera is B5, and the tenth camera is B6; when the first side is the right side of the trailer, the second side is the left side of the trailer, the fifth camera is B4, the sixth camera is B5, the seventh camera is B3, the eighth camera is B1, the ninth camera is B2, and the tenth camera is B6.

[0078] Through this method, at least two images respectively captured by at least two cameras are obtained to provide support for the target image, and at least one of the at least two cameras is set on the trailer of the tow vehicle, so that the driver can obtain a rear view image of the trailer, reducing the driving risk of the trailer.

[0079] S402: stitching at least two images according to the articulation angle of the trailer to obtain a target image.

[0080] Below, with Figure 5 Taking the four camera settings shown in the figure (i.e., A1, A2, A3, and A4) as an example, the specific implementation method of stitching at least two images is introduced:

[0081] (1) Image stitching solution for the camera on the inside of the articulation angle and the camera on the rear of the trailer:

[0082] In one possible manner, the image captured by the first camera A1 and the image captured by the second camera A2 are spliced ​​according to a first angle α1, wherein the first angle α1 is the angle between the viewing area of ​​the first camera A1 and the viewing area of ​​the second camera A2.

[0083] The following describes how to calculate the first angle α1.

[0084] The first angle α1 is calculated according to the articulation angle of the towing vehicle, the length of the trailer, the width of the trailer, and the first distance; wherein the first distance is the distance from the midpoint of the connecting line between the second camera A2 and the third camera A3 to the midpoint of the front of the trailer.

[0085] For a specific example, see Figure 7, the first angle is α1, the articulation angle of the trailer is β, the length of the trailer is L, the width of the trailer is K, the first distance is G, and the distance between the field of view of the second camera A2 and the third camera A3 is equal to the width of the trailer K, then α1 satisfies the following relationship:

[0086]

[0087] The following describes how to stitch the image captured by the first camera A1 and the image captured by the second camera A2 according to the first angle α1.

[0088] According to the bisector of the first angle α1, the image captured by the first camera A1 and the image captured by the second camera A2 are spliced. Figure 7 , the overlapping area between the viewing area of ​​the first camera A1 and the viewing area of ​​the second camera A2 is Figure 7 the lower right corner area; for the image captured by the first camera A1, crop it along the angular bisector of the first angle α1, and discard the image in the overlapping area that is close to the second camera A2 and far away from the first camera A1; for the image captured by the second camera A2, crop it along the angular bisector of the first angle α1, and discard the image in the overlapping area that is close to the first camera A1 and far away from the second camera A2; splice the remaining images captured by the first camera A1 with the remaining images captured by the second camera A2.

[0089] It can be understood that the spliced ​​image can be the target image or a part of the target image. The user can make a selection based on actual needs, and this application does not impose any limitation.

[0090] Through this method, the image captured by the first camera A1 and the image captured by the second camera A2 can be spliced ​​so that the spliced ​​image is coherent and there are no images in overlapping areas, thereby improving the practicality of the spliced ​​image. The spliced ​​image includes the rear view image of the trailer, reducing the driving risk of the trailer.

[0091] (2) Image stitching solution for the camera outside the articulation angle and the camera at the rear of the trailer:

[0092] In one possible manner, the image captured by the first camera A1 and the image captured by the third camera A3 are spliced ​​according to the second angle α2, wherein the second angle α2 is the angle between the viewing area of ​​the first camera A1 and the viewing area of ​​the third camera A3.

[0093] The following describes how to calculate the second angle α2.

[0094] According to the articulation angle of the towed vehicle, the second angle α2 is calculated.

[0095] For a specific example, see Figure 6 , the second angle is α2, the articulation angle of the trailer is β, then α2 satisfies the following relationship: α2=90°-β.

[0096] The following describes how to stitch the image captured by the first camera A1 and the image captured by the third camera A3 according to the second angle α2.

[0097] According to the angle bisector of the second angle α2, the image captured by the first camera A1 and the image captured by the third camera A3 are spliced. Figure 7 , the overlapping area between the viewing area of ​​the first camera A1 and the viewing area of ​​the third camera A3 is Figure 7 The upper right corner area of ​​the image captured by the first camera A1 is cropped along the bisector of the second angle α2, and the image in the overlapping area close to the third camera A3 and away from the first camera A1 is discarded; the image captured by the third camera A3 is cropped along the bisector of the second angle α2, and the image in the overlapping area close to the first camera A1 and away from the third camera A3 is discarded; the remaining images captured by the first camera A1 are spliced ​​with the remaining images captured by the third camera A3.

[0098] It can be understood that the spliced ​​image can be the target image or a part of the target image. The user can make a selection based on actual needs, and this application does not impose any limitation.

[0099] Through this method, the image captured by the first camera A1 and the image captured by the third camera A3 can be spliced, so that the spliced ​​image is coherent and there are no images in overlapping areas, thereby improving the practicality of the spliced ​​image. The spliced ​​image includes the rear view image of the trailer, reducing the driving risk of the trailer.

[0100] (3) Image stitching solution for the camera on the inside of the articulation angle, the camera on the outside of the articulation angle, and the camera on the rear side of the trailer:

[0101] In a possible manner, the image captured by the first camera A1, the image captured by the second camera A2, and the image captured by the third camera A3 are spliced ​​according to the first angle α1 and the second angle α2.

[0102] For a specific example, see Figure 7, stitch the image captured by the first camera A1, the image captured by the second camera A2, and the image captured by the third camera A3 according to the angular bisector of the first angle α1 and the angular bisector of the second angle α2. Among them, the method of trimming and discarding the image captured by the first camera A1 and the image captured by the second camera A2 according to the angular bisector of the first angle α1 and the method of trimming and discarding the image captured by the first camera A1 and the image captured by the third camera A3 according to the angular bisector of the second angle α2 are the same as the above method, and will not be repeated here. Stitch the remaining images captured by the first camera A1, the remaining images captured by the second camera A2, and the remaining images captured by the third camera A3.

[0103] It can be understood that the spliced ​​image can be the target image or a part of the target image. The user can make a selection based on actual needs, and this application does not impose any limitation.

[0104] Through this method, the image captured by the first camera A1, the image captured by the second camera A2 and the image captured by the third camera A3 can be spliced, so that the spliced ​​image is coherent and there are no images in overlapping areas, thereby improving the practicality of the spliced ​​image. The spliced ​​image includes the rear view image of the trailer, reducing the driving risk of the trailer.

[0105] (4) Image stitching scheme of the camera at the front of the tractor, the camera inside the articulation angle, the camera outside the articulation angle, and the camera at the rear of the trailer:

[0106] In a possible manner, an image captured by at least one of the second camera A2 and the third camera A3 and an image captured by the fourth camera A4 are spliced.

[0107] For a specific example, see Figure 7 The angle between the viewing area of ​​the second camera A2 and the viewing area of ​​the fourth camera A4 is 90°. Along the bisector of the angle, the images captured by the second camera A2 that are close to the fourth camera A4 and far away from the second camera A2 are discarded, and the images captured by the fourth camera A4 that are close to the second camera A2 and far away from the fourth camera A4 are discarded, and / or the angle between the viewing area of ​​the third camera A3 and the viewing area of ​​the fourth camera A4 is 90°. Along the bisector of the angle, the images captured by the third camera A3 that are close to the fourth camera A4 and far away from the third camera A3 are discarded, and the images captured by the fourth camera A4 that are close to the third camera A3 and far away from the fourth camera A4 are discarded; the remaining images are spliced.

[0108] It is understandable that the images captured by at least two of the first camera A1, the second camera A2, and the third camera A3 can also be spliced ​​with the image captured by the fourth camera A4. The method for splicing the images captured by at least two of the first camera A1, the second camera A2, and the third camera A3 is the same as the above method and will not be repeated here.

[0109] It can be understood that the spliced ​​image can be the target image or a part of the target image. The user can make a selection based on actual needs, and this application does not impose any limitation.

[0110] By this method, a fourth camera A4 is arranged at the front of the tractor, and the images captured by the first, second, third and fourth cameras can be stitched together to obtain a more complete view around the trailer, thereby improving the practicality and reliability of the stitched images.

[0111] Optionally, a panoramic image may be generated based on the target image, blind spots, and vehicle model filling areas.

[0112] The following introduces the method of calculating the blind area and the vehicle model filling area.

[0113] Calculate the blind spot and vehicle model filling area based on the articulation angle of the towing vehicle, the length of the trailer, and the width of the trailer.

[0114] For a specific example, see Figure 7 , the articulation angle of the trailer is β, the length of the trailer is L, and the width of the trailer is K.

[0115] The blind area S1 satisfies the following relationship:

[0116] The car model filling area S2 satisfies the following relationship:

[0117]

[0118] The target image, blind area, and vehicle model filling area are stitched together to obtain a panoramic image.

[0119] It can be understood that the target image can be the image obtained by stitching the image captured by the first camera A1 and the image captured by the second camera A2 according to the first angle α1 as mentioned above, or it can be the image obtained by stitching the image captured by the first camera A1 and the image captured by the third camera A3 according to the second angle α2, or it can be the image obtained by stitching the image captured by the first camera A1, the image captured by the second camera A2 and the image captured by the third camera A3 according to the first angle α1 and the second angle α2, and this application does not impose any restrictions.

[0120] Through this method, a panoramic image is generated based on the target image, blind spots, and vehicle model filling areas, so that the panoramic image can provide a relatively complete 360° scene centered on the trailer, making it easier for the driver to drive according to the panoramic image, reducing driving risks and improving driving safety.

[0121] Below, with Figure 6 Taking the arrangement of the six cameras (i.e., B1, B2, B3, B4, B5, and B6) shown in the figure as an example, taking the first side as the left side of the trailer as an example (i.e., the fifth camera is B1, and the sixth camera is B2), a specific implementation method for stitching at least two images is introduced:

[0122] (1) Image stitching solution of the camera on the first side of the tractor and the camera on the first side of the trailer:

[0123] In a possible manner, the image captured by the fifth camera and the image captured by the sixth camera are spliced ​​according to the offset angle of the splicing line.

[0124] See also Figure 6 , Figure 8 , when the trailer status changes from Figure 6 becomes Figure 8 When the articulation angle of the trailer changes from 0° to β, the offset angle of the splicing line is That is, the splicing line is Figure 6 The center is perpendicular to the axis of the trailer and offset to Figure 8 The angle between the center and the axis of the tractor is The stitching line is an image captured by the fifth camera B1 and an image captured by the sixth camera B2.

[0125] According to the offset angle of the stitching line, the image captured by the fifth camera B1 and the image captured by the sixth camera B2 are stitched. Figure 8 , the fifth camera is B1, the sixth camera is B2, the image captured by the fifth camera B1 is cropped along the stitching line, and the image within the angle is discarded; the image captured by the sixth camera B2 is cropped along the stitching line, and the image outside the angle is discarded; the remaining images captured by the fifth camera B1 are stitched with the remaining images captured by the sixth camera B2.

[0126] When the fifth camera is B4 and the sixth camera is B5, the stitching method is the same as that in the above text where the fifth camera is B1 and the sixth camera is B2, and will not be repeated here.

[0127] It can be understood that the spliced ​​image can be the target image or a part of the target image. The user can make a selection based on actual needs, and this application does not impose any limitation.

[0128] Through this method, the images captured by two cameras are stitched according to the offset angle of the stitching line, so that the stitched image is coherent and there is no image in the overlapping area. In addition, by stitching the images according to the offset angle of the stitching line, the stitched image can be clear, thereby improving the usability of the stitched image.

[0129] (2) Image stitching solution for the camera at the front of the tractor, the cameras on both sides of the tractor, the cameras on both sides of the trailer, and the camera at the rear of the trailer:

[0130] In one possible manner, images captured by the fifth camera, the fifth camera is B1, the sixth camera is B2, the seventh camera is B3, the eighth camera is B4, the ninth camera is B5, and the tenth camera is B6 are stitched.

[0131] For a specific example, see Figure 8 , the fifth camera is B1, the sixth camera is B2, the seventh camera is B3, the eighth camera is B4, the ninth camera is B5, and the tenth camera is B6. The images captured by two adjacent cameras are cropped according to the bisector of the angle between the viewing areas of the two cameras. For example, the angle between the viewing area of ​​the sixth camera B2 and the viewing area of ​​the tenth camera B6 is 90°. Along the bisector of the angle, the images captured by the sixth camera B2 that are close to the tenth camera B6 and far away from the sixth camera B2 are discarded, and the images captured by the tenth camera B6 that are close to the sixth camera B2 and far away from the tenth camera B6 are discarded. The remaining images are spliced.

[0132] Through this method, four cameras can also be set at other positions of the trailer, and the images collected by the six cameras on the trailer can be stitched together. The stitched image is a 360° target image centered on the trailer, which can fully and clearly reflect the scene around the trailer, reducing driving risks and ensuring the safety of the driving process.

[0133] In a possible implementation, the articulation angle of the trailer can be obtained by an angle sensor, wherein the angle sensor is disposed on the tractor. For example, the intelligent driving controller receives the angle information collected by the sensor, and then determines the size of the articulation angle of the trailer.

[0134] In a possible implementation, a human-computer interaction interface (such as Fig. 9 ) and other human-machine interaction interfaces to obtain the length and width of the trailer. Fig. 9 As shown, the user inputs the length and width of the trailer on the vehicle display screen, and the intelligent driving controller obtains the input information collected by the display screen to determine the length and width of the trailer.

[0135] Through this method, the articulation angle of the trailer is obtained by an angle sensor, so that the obtained articulation angle is accurate, thereby improving the availability of the target image, and the length and width of the trailer are obtained through the human-computer interaction interface. When the trailer is replaced by a trailer, the target image of the trailer corresponding to the trailer under different trailers can be obtained, thereby improving the flexibility of the method.

[0136] See also Fig.10 An embodiment of the present application provides an image generating device 1000, which includes a module / unit / technical means for executing the method executed by the SOC in the above method embodiment.

[0137] Exemplarily, the device 1000 includes:

[0138] An acquisition module 1001 is used to acquire at least two images; wherein the at least two images are respectively acquired by at least two cameras, the at least two cameras are arranged on a trailer, and at least one of the at least two cameras is arranged on a trailer of the trailer;

[0139] The processing module 1002 is used to stitch the at least two images according to the articulation angle of the trailer to obtain a target image.

[0140] It should be understood that all relevant contents of each step involved in the above method embodiment can be referred to the functional description of the corresponding functional module and will not be repeated here.

[0141] As a possible product form of the above device, see Fig.11 The embodiment of the present application further provides an electronic device 1100, including:

[0142] At least one processor 112; the at least one processor 112 executes instructions stored in the memory 113, so that the electronic device 1100 executes the method steps executed by any device in the above method embodiments.

[0143] Optionally, the memory 113 is located outside the electronic device 1100 .

[0144] Optionally, the electronic device 1100 includes the memory 113, the memory 113 is connected to the at least one processor 112, and the memory 113 stores instructions that can be executed by the at least one processor 112. Fig.11 The dashed lines indicate that the memory 113 is optional for the electronic device 1100 .

[0145] The processor 112 and the memory 113 may be coupled via an interface circuit or may be integrated together, which is not limited here.

[0146] Optionally, the electronic device further includes at least two cameras 111 for capturing at least two images.

[0147] The specific connection medium between the processor 112 and the memory 113 is not limited in the embodiment of the present application. Fig.11 In the embodiment, at least two cameras 111, a processor 112, and a memory 113 are connected via a bus 114. Fig.11 The connections between the other components are shown in bold lines, which are only for illustration and are not intended to be limiting. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Fig.11 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0148] It should be understood that the processor mentioned in the embodiments of the present application can be implemented by hardware or by software. When implemented by hardware, the processor can be a logic circuit, an integrated circuit, etc. When implemented by software, the processor can be a general-purpose processor implemented by reading software code stored in a memory.

[0149] Exemplarily, the processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0150] It should be understood that the memory mentioned in the embodiments of the present application may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM) and direct memory bus random access memory (DirectRambus RAM, DR RAM).

[0151] It should be noted that when the processor is a general-purpose processor, DSP, ASIC, FPGA or other programmable logic device, discrete gate or transistor logic device, discrete hardware component, the memory (storage module) can be integrated into the processor.

[0152] It should be noted that the memory described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0153] As another possible product form, an embodiment of the present application also provides a computer-readable storage medium, which is used to store instructions. When the instructions are executed, the computer executes the method steps executed by any device in the above method example.

[0154] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0155] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0156] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0157] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0158] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. An image generation method, characterized in that: include: Get at least two images; The at least two images are respectively captured by at least two cameras, the at least two cameras are arranged on the trailer, and at least one of the at least two cameras is arranged on the trailer of the trailer; splicing the at least two images according to the articulation angle of the trailer to obtain a target image; The at least two cameras include a first camera, a second camera and a third camera, the first camera is arranged at the rear of the trailer, and the second camera and the third camera are arranged on both sides of the trailer opposite to each other; The step of stitching the at least two images according to the articulation angle of the trailer to obtain a target image includes: A first angle is calculated according to the articulation angle of the towing vehicle, the length of the trailer, the width of the trailer, and the first distance, and the first angle satisfies the following relationship: Wherein, α1 is the first angle, G is the first distance, L is the length of the trailer, K is the width of the trailer, and β is the articulation angle of the towed vehicle; the first angle is the angle between the viewing area of ​​the first camera and the viewing area of ​​the second camera, and the first distance is the distance from the midpoint of the connecting line between the second camera and the third camera to the midpoint of the front of the trailer; The second camera is arranged inside the articulation angle of the trailer; and the stitching of the at least two images according to the articulation angle of the trailer further includes: The image captured by the first camera and the image captured by the second camera are stitched together according to the first angle.

2. The method according to claim 1, characterized in that The third camera is arranged outside the articulation angle of the trailer; The step of stitching the at least two images according to the articulation angle of the trailer includes: Calculating a second angle according to the articulation angle of the trailer; wherein the second angle is the angle between the viewing area of ​​the first camera and the viewing area of ​​the third camera; The image captured by the first camera and the image captured by the third camera are stitched according to the second angle; wherein the second angle satisfies the following relationship: α2=90°-β; wherein α2 is the second angle, and β is the articulation angle of the trailer.

3. The method according to claim 1 or 2, characterized in that The at least two cameras also include a fourth camera, which is arranged at the front of the tractor of the trailer.

4. The method according to claim 1 or 2, characterized in that: Also includes: Calculate the blind area and the vehicle model filling area according to the articulation angle of the towing vehicle, the length of the trailer, and the width of the trailer; A panoramic image is generated according to the target image, the blind area, and the vehicle model filling area.

5. The method according to claim 4, characterized in that The blind area satisfies the following relationship: The car model filling area satisfies the following relationship: Among them, S1 is the blind area, S2 is the filling area of ​​the vehicle model, L is the length of the trailer, K is the width of the trailer, and β is the articulation angle of the towing trailer.

6. The method according to claim 1, characterized in that The at least two cameras include a fifth camera and a sixth camera; wherein the fifth camera is arranged on a first side of the tractor of the trailer, and the sixth camera is arranged on a first side of the trailer, and the first side of the tractor and the first side of the trailer are located on a first side of the trailer; The step of stitching the at least two images according to the articulation angle of the trailer includes: Calculating the offset angle of the splicing line according to the articulation angle of the trailer; The image captured by the fifth camera and the image captured by the sixth camera are stitched together according to the offset angle of the stitching line.

7. The method according to claim 6, characterized in that The at least two cameras also include a seventh camera, an eighth camera, a ninth camera, and a tenth camera; wherein the seventh camera is arranged at the front of the tractor, the eighth camera is arranged at the second side of the tractor, and the ninth camera is arranged at the second side of the trailer, the second side of the tractor and the second side of the trailer are located on the second side of the towed vehicle, and the first side and the second side are opposite; the tenth camera is arranged at the rear of the trailer.

8. The method according to claim 1, characterized in that Also includes: Acquiring the articulation angle of the trailer by means of an angle sensor; The angle sensor is arranged on the tractor of the trailer.

9. The method according to claim 1, characterized in that Also includes: The length and width of the trailer are obtained through a human-computer interaction interface.

10. An image generating device, characterized in that: include: An acquisition module, used for acquiring at least two images; The at least two images are respectively captured by at least two cameras, the at least two cameras are arranged on the trailer, and at least one of the at least two cameras is arranged on the trailer of the trailer; A processing module, configured to stitch the at least two images together according to the articulation angle of the trailer to obtain a target image; The at least two cameras include a first camera, a second camera and a third camera, the first camera is arranged at the rear of the trailer, and the second camera and the third camera are arranged on both sides of the trailer relatively; when the processing module splices the at least two images according to the articulation angle of the trailer to obtain the target image, it is specifically used to: A first angle is calculated according to the articulation angle of the towing vehicle, the length of the trailer, the width of the trailer, and the first distance, and the first angle satisfies the following relationship: Wherein, α1 is the first angle, G is the first distance, L is the length of the trailer, K is the width of the trailer, and β is the articulation angle of the towed vehicle; the first angle is the angle between the viewing area of ​​the first camera and the viewing area of ​​the second camera, and the first distance is the distance from the midpoint of the connecting line between the second camera and the third camera to the midpoint of the front of the trailer; The second camera is arranged inside the articulation angle of the trailer; when the processing module stitches the at least two images according to the articulation angle of the trailer, it is further used to: The image captured by the first camera and the image captured by the second camera are stitched together according to the first angle.

11. An electronic device, characterized in that: include: at least one processor; and a memory communicatively coupled to the at least one processor; The memory stores instructions executable by the at least one processor, and the at least one processor executes the instructions stored in the memory, so that the image generating device according to claim 10 executes the method according to any one of claims 1 to 9.

12. The electronic device according to claim 11, characterized in that: Also includes: At least two cameras are used to capture at least two images.

13. A trailer, characterized in that: Includes the image generating device as claimed in claim 10, or includes the electronic device as claimed in claim 11 or 12.

14. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and when the computer instructions are executed on a computer, the computer is enabled to execute the method according to any one of claims 1 to 9.

Images