Communication method and apparatus, and system

Abstract

A communication method and apparatus, and a system, relating to the field of communications, and capable of accurately obtaining a spliced image in a vehicle body replacement scenario. The method comprises: acquiring intrinsic parameters of cameras of a first vehicle and intrinsic parameters of cameras of a second vehicle; and according to the intrinsic parameters of the cameras of the first vehicle and the intrinsic parameters of the cameras of the second vehicle, obtaining a spliced image, the spliced image being a panoramic image after the second vehicle is hitched to the first vehicle. Intrinsic parameters at least comprise a video encoding and decoding parameter, and the video encoding and decoding parameter is used for parsing images acquired by the cameras. Intrinsic parameters further comprise one or more of the following: a first parameter, a camera internal parameter, and a camera external parameter. The first parameter is used for indicating a mapping relationship between each pixel in a spliced image and a pixel of the image acquired by the cameras. The solution of the present application can be widely applied to the technical fields of communication, artificial intelligence, vehicle networking, intelligent home networking, and the like.

Description

A communication method, apparatus and system

[0001] This application claims priority to Chinese Patent Application No. 202411527964.1, filed on October 29, 2024, entitled "A Communication Method, Apparatus and System", the entire contents of which are incorporated herein by reference. Technical Field

[0002] This application relates to the field of communications, and more particularly to a communication method, apparatus, and system. Background Technology

[0003] In the field of modern transportation, especially in improving the driving safety and efficiency of large vehicles such as trucks, trailers, and container trucks, the panoramic surround view system (hereinafter referred to as the surround view system) plays a crucial role. For example, the surround view system of a vehicle with a trailer integrates multi-angle cameras. Through the multi-angle cameras, it identifies obstacles such as pedestrians and other vehicles around the vehicle, providing the driver with a panoramic image (or stitched image) of the surrounding environment and alerting the driver when necessary. At the same time, it uses camera and sensor information to estimate the turning angle of the vehicle in real time, assisting the driver in making precise turning operations, effectively reducing the driver's blind spots and improving driving safety.

[0004] However, for detachable vehicle combinations such as semi-trailers, tractor units, and container trucks, when the vehicle body is changed, such as by changing to a different trailer or container, the changes in the vehicle combination lead to the reconfiguration and adjustment of the vehicle's surround view system cameras. In this case, it is necessary to provide the driver with a panoramic image (or stitched image) of the vehicle's surrounding environment in a timely and accurate manner based on the reconfigured and adjusted cameras. Summary of the Invention

[0005] This application provides a communication method, device, and system that can accurately obtain stitched images in a vehicle body replacement scenario.

[0006] To achieve the above objectives, this application adopts the following technical solution:

[0007] Firstly, this application provides a communication method, which can be executed by a first device, such as a central control module. Unless otherwise specified, the term "central control module" in this application can refer to the central control module itself, components within the central control module (e.g., processor, radio frequency unit, chip, or chip system), or logic modules or software capable of implementing all or part of the functions of the central control module. The method includes: acquiring at least inherent parameters, including video encoding and decoding, of a camera of a first vehicle, and acquiring at least inherent parameters, including video encoding and decoding, of a camera of a second vehicle; obtaining a stitched image based on the inherent parameters of the cameras of the first and second vehicles, wherein the stitched image is a panoramic image of the second vehicle mounted on the first vehicle. The video encoding and decoding parameters are used to parse the images captured by the cameras; the inherent parameters also include one or more of the following: a first parameter, camera intrinsic parameters, and camera extrinsic parameters; the first parameter is used to indicate the mapping relationship between each pixel in the stitched image and the pixels in the images captured by the cameras.

[0008] Based on the method described in the first aspect, compared with transmitting image or video stream data acquired by the camera, the transmission of key parameters that have a positive benefit in obtaining stitched images is added, namely the transmission of the inherent parameters of the camera of the first vehicle and the inherent parameters of the camera of the second vehicle. This facilitates the rapid and accurate acquisition of stitched images after dynamic changes in the vehicle body, such as stitched images of the second vehicle attached to the first vehicle, based on the inherent parameters of the camera of the first vehicle and the inherent parameters of the camera of the second vehicle.

[0009] In one possible design, the inherent parameters also include panoramic image parameters, which include one or more of the following: the resolution of the stitched image, the pixel coordinates of the traction axis in the stitched image, and the actual ground area corresponding to the stitched image.

[0010] Based on this possible design, the first device can obtain the panoramic image parameters of the camera of the first vehicle and the panoramic image parameters of the camera of the second vehicle, so that the inherent parameters can include different parameters in different scenarios, thereby improving the flexibility of the embodiments of this application.

[0011] In one possible design, the inherent parameters also include the communication protocol version, which includes the supported application layer protocol versions.

[0012] Based on this possible design, the first device can obtain the application layer protocol version supported by the camera of the first vehicle and the application layer protocol version supported by the camera of the second vehicle, so that the inherent parameters can include different parameters in different scenarios, thereby improving the flexibility of the embodiments of this application.

[0013] In one possible design, obtaining the stitched image may include: obtaining a first pixel mapping table and a second pixel mapping table; normalizing the first pixel mapping table and the second pixel mapping table to obtain a global pixel mapping table; and obtaining the stitched image based on the global pixel mapping table. The first pixel mapping table is determined by the first parameters of the camera on the first vehicle, and the second pixel mapping table is determined by the first parameters of the camera on the second vehicle.

[0014] Based on this possible design, the first device can obtain the stitched image according to the first parameter in the inherent parameters of the cameras of each vehicle.

[0015] In one possible design, obtaining the stitched image includes: calculating the correspondence between the resolution of each camera and the resolution of the stitched image based on camera intrinsic and extrinsic parameters; generating a global pixel mapping table based on the correspondence between the resolution of each camera and the resolution of the stitched image; and obtaining the stitched image based on the global pixel mapping table.

[0016] Based on this possible design, the first device can obtain stitched images according to the camera intrinsic and extrinsic parameters in the inherent parameters of the cameras of each vehicle.

[0017] In one possible design, the method described in the first aspect may further include obtaining the heading angle between the first vehicle and the second vehicle; adjusting the global pixel mapping table based on the heading angle; and adjusting the stitched image based on the adjusted global pixel mapping table.

[0018] Based on this possible design, the first device can adjust the stitched image according to the heading angle between the first vehicle and the second vehicle, so that the first device can still obtain the stitched image in real time and accurately when the relative pose between the first vehicle and the second vehicle changes.

[0019] In one possible design, the method described in the first aspect may further include: sending video stream data and sensor data to the vehicle attitude estimation module, wherein the video stream data includes video stream data collected by cameras in the cameras of the first vehicle and the second vehicle, and the sensor data includes one or more of the following: angle sensor data, vehicle control data, and vehicle controller area network (CAN) data; obtaining the heading angle between the first vehicle and the second vehicle in the method described in the first aspect includes: receiving the heading angle from the vehicle attitude estimation module.

[0020] Based on this possible design, the first device can obtain the heading angle between the first vehicle and the second vehicle through the vehicle attitude estimation module.

[0021] In one possible design, the method described in the first aspect may further include: in response to the fact that both the camera of the first vehicle and the camera of the second vehicle support hot-plugging, discovering the camera of the first vehicle and discovering the camera of the second vehicle; and authenticating the camera of the first vehicle and the camera of the second vehicle.

[0022] Based on this possible design, if both the camera in the first vehicle and the camera in the second vehicle support hot-swapping, the first device can ensure secure and efficient communication between the first device, the camera in the first vehicle, and the camera in the second vehicle through three operations: discovering the camera in the first vehicle, discovering the camera in the second vehicle, and authenticating the camera in the first vehicle and the camera in the second vehicle.

[0023] Secondly, this application provides a communication method that can be executed by a second device, such as a camera subsystem of a first vehicle. Unless otherwise specified, "camera subsystem of the first vehicle" in this application can refer to the camera subsystem of the first vehicle itself, components within the camera subsystem (e.g., processors, radio frequency units, chips, or chip systems), or logic modules or software capable of implementing all or part of the functions of the camera subsystem of the first vehicle. The method includes: transmitting inherent parameters from the camera of the first vehicle, including at least video encoding and decoding parameters, used to determine a stitched image, the stitched image being a panoramic image of the second vehicle mounted on the first vehicle, wherein the video encoding and decoding parameters are used to parse the image captured by the camera; the inherent parameters also include one or more of the following: a first parameter, camera intrinsic parameters, and camera extrinsic parameters; the first parameter is used to indicate the mapping relationship between each pixel in the stitched image and the pixels in the image captured by the camera.

[0024] Based on the method described in the second aspect, the first device can obtain at least the inherent parameters of the camera of the first vehicle, including video encoding and decoding, through the second device. Compared with transmitting the image or video stream data captured by the camera of the first vehicle to the first device, the transmission of key parameters that have a positive benefit to obtaining the stitched image is added, namely the inherent parameters of the camera of the first vehicle. This makes it easier for the first device to quickly and accurately obtain the stitched image after the dynamic changes of the vehicle body, such as the stitched image of the second vehicle attached to the first vehicle, based on the inherent parameters of the camera of the first vehicle.

[0025] In one possible design, the inherent parameters also include panoramic image parameters, which include one or more of the following: the resolution of the stitched image, the pixel coordinates of the traction axis in the stitched image, and the actual ground area corresponding to the stitched image.

[0026] Based on this possible design, the first device can obtain the panoramic image parameters of the camera of the first vehicle, so that the inherent parameters can include different parameters in different scenarios, thereby improving the flexibility of the embodiments of this application.

[0027] In one possible design, the inherent parameters also include the communication protocol version, which includes the supported application layer protocol versions.

[0028] Based on this possible design, the first device can obtain the application layer protocol version supported by the camera of the first vehicle, so that the inherent parameters can include different parameters in different scenarios, thereby improving the flexibility of the embodiments of this application.

[0029] Thirdly, this application provides a communication method that can be executed by a third device, such as a camera subsystem of a second vehicle. Unless otherwise specified, "camera subsystem of the second vehicle" in this application can refer to the camera subsystem of the second vehicle itself, components within the camera subsystem (e.g., processors, radio frequency units, chips, or chip systems), or logic modules or software capable of implementing all or part of the functions of the camera subsystem of the second vehicle. The method includes: transmitting inherent parameters from the camera of the second vehicle, including at least video encoding and decoding parameters, used to determine a stitched image, which is a panoramic image of the second vehicle mounted on a first vehicle, wherein the video encoding and decoding parameters are used to parse the image captured by the camera; the inherent parameters also include one or more of the following: a first parameter, camera intrinsic parameters, and camera extrinsic parameters; the first parameter indicates the mapping relationship between each pixel in the stitched image and the pixels in the image captured by the camera.

[0030] Based on the method described in the third aspect, the first device can obtain at least the inherent parameters of the second vehicle's camera, including video encoding and decoding, through the third device. Compared to transmitting the image or video stream data captured by the second vehicle's camera to the first device, this method adds the transmission of key parameters that have a positive benefit in obtaining the stitched image, namely the inherent parameters of the second vehicle's camera. This makes it easier for the first device to quickly and accurately obtain the stitched image after the vehicle body has undergone dynamic changes, such as the stitched image of the second vehicle attached to the first vehicle, based on the inherent parameters of the second vehicle's camera.

[0031] In one possible design, the inherent parameters also include panoramic image parameters, which include one or more of the following: the resolution of the stitched image, the pixel coordinates of the traction axis in the stitched image, and the actual ground area corresponding to the stitched image.

[0032] Based on this possible design, the first device can obtain the panoramic image parameters of the second vehicle's camera, so that the inherent parameters can include different parameters in different scenarios, thereby improving the flexibility of the embodiments of this application.

[0033] In one possible design, the inherent parameters also include the communication protocol version, which includes the supported application layer protocol versions.

[0034] Based on this possible design, the first device can obtain the application layer protocol version supported by the camera of the second vehicle, so that the inherent parameters can include different parameters in different scenarios, thereby improving the flexibility of the embodiments of this application.

[0035] Fourthly, this application provides a communication device that can be applied to the first device mentioned in the first aspect, such as a central control module, to realize the functions performed by the first device. The communication device can be the first device itself, or it can be a chip, chip system, or system-on-a-chip of the first device. The communication device can execute the functions performed by the first device through hardware, or it can execute corresponding software through hardware. The hardware or software includes one or more modules corresponding to the above functions. For example: a module for acquiring the inherent parameters of the camera of the first vehicle and the inherent parameters of the camera of the second vehicle; a module for obtaining a stitched image based on the inherent parameters of the camera of the first vehicle and the camera of the second vehicle. The inherent parameters include at least video encoding / decoding parameters used to parse the images captured by the camera; the inherent parameters also include one or more of the following: a first parameter, camera intrinsic parameters, and camera extrinsic parameters; the first parameter indicates the mapping relationship between each pixel in the stitched image and the pixels in the image captured by the camera; the stitched image is a panoramic image of the second vehicle attached to the first vehicle.

[0036] Fifthly, this application provides a communication device that can be applied to the second device described in the second aspect above, such as the camera subsystem of a first vehicle, to realize the functions performed by the second device. The communication device can be the second device, or it can be a chip, chip system, or system-on-a-chip of the second device. The communication device can execute the functions performed by the second device through hardware, or it can execute corresponding software through hardware. The hardware or software includes one or more modules corresponding to the above functions. For example, a module for transmitting inherent parameters of the camera of the first vehicle, wherein the inherent parameters are used to determine the stitched image, and the stitched image is a panoramic image of the second vehicle attached to the first vehicle; the inherent parameters include at least video encoding / decoding parameters used to parse the image captured by the camera; the inherent parameters also include one or more of the following: a first parameter, camera intrinsic parameters, and camera extrinsic parameters; the first parameter is used to indicate the mapping relationship between each pixel in the stitched image and the pixels in the image captured by the camera.

[0037] Sixthly, this application provides a communication device that can be applied to the third device described in the third aspect above, such as the camera subsystem of a second vehicle, to realize the functions performed by the third device. The communication device can be the third device, or it can be a chip, chip system, or system-on-a-chip of the third device. The communication device can execute the functions performed by the third device through hardware, or it can execute corresponding software through hardware. The hardware or software includes one or more modules corresponding to the above functions. For example, a module for transmitting the inherent parameters of the camera of the second vehicle, wherein the inherent parameters are used to determine the stitched image, and the stitched image is a panoramic image of the second vehicle attached to the first vehicle; the inherent parameters include at least video encoding and decoding parameters used to parse the image captured by the camera; the inherent parameters also include one or more of the following: a first parameter, camera intrinsic parameters, and camera extrinsic parameters; the first parameter is used to indicate the mapping relationship between each pixel in the stitched image and the pixels in the image captured by the camera.

[0038] In a seventh aspect, embodiments of this application provide a communication device, the communication device including one or more processors; the one or more processors are configured to run computer programs or instructions, such that when the one or more processors execute the computer instructions or instructions, the communication method described in any one of the first to third aspects is executed.

[0039] In one possible design, the communication device further includes one or more memories coupled to one or more processors, the memories used to store the aforementioned computer programs or instructions. In one possible implementation, the memories are located outside the communication device. In another possible implementation, the memories are located inside the communication device. In embodiments of this application, the processor and memory may also be integrated into a single device, i.e., the processor and memory may be integrated together. In one possible implementation, the communication device further includes a transceiver for receiving and / or transmitting information.

[0040] In one possible design, the communication device further includes one or more communication interfaces coupled to one or more processors, and the communication interfaces are used to communicate with other modules outside the communication device.

[0041] Eighthly, embodiments of this application provide a communication device, which includes an interface circuit and a logic circuit; the interface circuit is used to input and / or output information; the logic circuit is used to perform the communication method as described in any one of the first to third aspects, and to process and / or generate information based on the information.

[0042] Ninthly, embodiments of this application provide a computer-readable storage medium storing computer instructions or programs that, when executed on a computer, cause the communication method described in any one of the first to third aspects to be performed.

[0043] In a tenth aspect, embodiments of this application provide a computer program product containing computer instructions that, when run on a computer, causes the communication method described in any one of the first to third aspects to be executed.

[0044] Eleventhly, embodiments of this application provide a computer program that, when run on a computer, causes the communication method described in any one of the first to third aspects to be executed.

[0045] In a twelfth aspect, embodiments of this application provide a chip, including: a processor coupled to a memory, the memory being used to store programs or instructions, wherein when the program or instructions are executed by the processor, a communication method as described in any one of the first to third aspects is executed.

[0046] The technical effects of any of the design methods in aspects four through twelfth are similar to those in aspects one through three, and will not be elaborated upon further.

[0047] In a thirteenth aspect, embodiments of this application provide a communication system that may include a communication device for performing the method described in the first aspect or any possible design of the first aspect, a communication device for performing the method described in the second aspect or any possible design of the second aspect, and a communication device for performing the method described in the third aspect or any possible design of the third aspect. Attached Figure Description

[0048] Figure 1 is a schematic diagram of the architecture of a communication system provided in an embodiment of this application;

[0049] Figure 2 is a flowchart illustrating a communication method provided in an embodiment of this application;

[0050] Figure 3 is a schematic diagram of an inherent parameter transmission process provided in an embodiment of this application;

[0051] Figure 4 is a schematic diagram of a stitched image initialization process provided in an embodiment of this application;

[0052] Figure 5 is a schematic diagram of another image stitching initialization process provided in an embodiment of this application;

[0053] Figure 6 is a schematic diagram of a stitched image adjustment process provided in an embodiment of this application;

[0054] Figure 7 is a schematic diagram of the structure of a communication device provided in an embodiment of this application;

[0055] Figure 8 is a schematic diagram of another communication device provided in an embodiment of this application;

[0056] Figure 9 is a schematic diagram of another communication device provided in an embodiment of this application;

[0057] Figure 10 is a schematic diagram of the structure of another communication device provided in an embodiment of this application. Detailed Implementation

[0058] Before introducing the embodiments of this application, some technical terms involved in the embodiments of this application will be explained. It should be noted that the following explanations are for the purpose of making the embodiments of this application easier to understand, and should not be regarded as a limitation on the scope of protection claimed by the embodiments of this application.

[0059] In the field of modern transportation, especially in improving the driving safety and efficiency of large vehicles such as trucks, trailers, and container trucks, the panoramic surround view system (hereinafter referred to as the surround view system) can provide more intuitive assisted driving image information for car drivers, and has a very good application prospect in car assisted driving and car safety.

[0060] However, as mentioned in the background technology, surround view systems face some technical challenges in practical applications. When the vehicle body is changed, such as by changing to different trailers or containers, the changes in vehicle combination lead to the reconfiguration and adjustment of the cameras in the vehicle's surround view system. At this time, it is necessary to provide the driver with a panoramic image (or stitched image) of the vehicle's surrounding environment in a timely and accurate manner based on the reconfigured and adjusted cameras.

[0061] In one implementation, the surround view system can use either statically calibrated image stitching or dynamically calibrated image stitching to obtain the stitched image. Staticly calibrated image stitching is suitable for scenarios where the camera position is fixed and the shooting environment is relatively stable; while dynamically calibrated image stitching is suitable for scenarios where the camera's motion changes significantly and parameters need to be updated in real time.

[0062] The core of statically calibrated image stitching technology lies in obtaining the camera's intrinsic and extrinsic parameters through a pre-set calibration pattern, further generating a lookup table (LUT), and then obtaining the stitched image based on the lookup table. The lookup table describes how each pixel in the camera or acquired image is mapped to a pixel position in the stitched image. The statically calibrated image stitching technology can include the following steps (1)-(5):

[0063] Step (1) Placement of calibration pattern: Place the preset calibration pattern in a specific calibration environment.

[0064] Calibration patterns are typically a series of points or patterns with known positions and dimensions, such as a checkerboard pattern, used to assist in camera positioning and parameter measurement.

[0065] Step (2) Image acquisition: Use each camera to capture images of the calibration pattern from different angles.

[0066] Step (3) Calculation of intrinsic and extrinsic parameters: Process the images acquired by each camera and calculate the intrinsic and extrinsic parameters of the camera.

[0067] In this context, camera intrinsic and extrinsic parameters refer to the camera's internal parameters (or camera intrinsic parameters) and external parameters (or camera extrinsic parameters). Common camera intrinsic parameters include focal length, image center, and distortion coefficients, while a common camera extrinsic parameter is camera attitude, which refers to the camera's position and orientation relative to the actual physical world coordinate system (or world coordinate system). The world coordinate system is a user-defined three-dimensional coordinate system used to describe the position of an object in real three-dimensional space.

[0068] Step (4) Pixel mapping table generation: Pixel mapping table is constructed based on the camera intrinsic and extrinsic parameters of each camera.

[0069] Step (5) Image stitching: Use a pixel mapping table to perform pixel mapping on the images captured by each camera, and finally stitch all the mapped images into a unified, distortion-free panoramic image.

[0070] The static calibration image stitching technology has the following characteristics: 1) Static calibration provides high calibration accuracy in a fixed environment, ensuring the accuracy of image stitching. 2) Once calibration is complete, the pixel mapping table remains unchanged unless the vehicle's hardware configuration, such as the cameras mounted on the vehicle body, changes, which ensures the stability of the surround view system. 3) The calibration process requires a specific calibration environment and a preset calibration pattern, limiting the application of static calibration image stitching technology in dynamic or moving scenarios.

[0071] Although image stitching technology has significant advantages in surround view systems, its limitations become particularly prominent when facing scenarios with dynamic vehicle changes, such as replacing the first trailer attached to the tractor with a second trailer. The following problems exist: 1. Poor adaptability: Once the trailer attached to the tractor changes, the original pixel mapping table becomes inapplicable. Static calibration needs to be re-performed after changing the trailer or adjusting the camera position, which is time-consuming and costly. 2. Insufficient real-time performance: The static calibration process needs to be completed under specific calibration conditions and cannot be adjusted in real time to adapt to changes in vehicle status. Especially in scenarios with frequent vehicle body changes, the surround view system cannot respond quickly, affecting its usability. 3. Complex operation: Each re-static calibration involves the complex processes shown in steps (1) to (5) above, requiring high professional skills from operators and increasing the barrier to entry. 4. Maintenance costs: Frequent replacement and maintenance processes not only increase operation time but may also cause hardware wear due to frequent calibration processes, increasing the system's maintenance costs.

[0072] Dynamic calibration image stitching technology is a solution developed to address the limitations of static calibration technology when the vehicle body is dynamically changing. Unlike static calibration image stitching technology, dynamic calibration does not rely on a specific calibration environment or preset calibration pattern. Instead, it analyzes real-time images captured by cameras during vehicle operation and automatically adjusts and optimizes the camera stitching parameters to adapt to changes in vehicle body configuration. Dynamic calibration image stitching technology may include the following steps (2.1)-(2.4):

[0073] Step (2.1) Image Acquisition: The camera continuously captures images of the vehicle's operating environment. The images contain information such as roads, obstacles, and other vehicles, which can be used as reference points for dynamic calibration.

[0074] Step (2.2) Feature point detection and matching: Feature points in the image, such as road edges, markings, signs, etc., are automatically detected by feature detection algorithm, and matched between images captured by different cameras to construct the relative positional relationship between cameras.

[0075] Step (2.3) Real-time spatial positioning: Based on the matching results of feature points, the spatial relationship between cameras can be estimated in real time through the stitching algorithm. The spatial relationship can include parameters such as distance and angle, thereby generating a dynamic pixel mapping table or adjusting the parameters of the stitching algorithm.

[0076] Step (2.4) Image stitching and optimization: Dynamically updated stitching parameters are applied to the image stitching process to ensure that the surround view system can still provide accurate panoramic images even when the vehicle body changes (such as the trailer attached to the tractor is changed from the first trailer to the second trailer) or the camera position is slightly adjusted.

[0077] The image stitching technology with dynamic calibration has the following characteristics: 2.1) It can adjust the stitching parameters in real time to adapt to the dynamic changes of the vehicle body, without the need for complex manual calibration after changing the vehicle body. (2) Since it does not rely on a fixed calibration environment, the image stitching technology with dynamic calibration is more suitable for complex and ever-changing vehicle application scenarios, such as tractors that frequently change trailers. (3) It relies on image processing and machine learning algorithms, and has high requirements for computing resources and algorithm optimization to ensure real-time performance and accuracy.

[0078] Although dynamic calibration image stitching technology has shown significant advantages in dynamic vehicle body changing scenarios, it still has the following limitations in practical applications: (1) Accuracy issues: Dynamic calibration based on image feature points is easily affected by factors such as lighting, weather, and changes in the surrounding environment, which may reduce calibration accuracy, especially when the feature point detection of the image is inaccurate or the matching fails. (2) Algorithm complexity and computational resource requirements: The real-time performance and accuracy of dynamic calibration image stitching technology require strong computational resources, which places higher demands on the hardware configuration of vehicle equipment and may lead to increased costs. (3) Calibration time: Although dynamic calibration can be performed online, after the first calibration or a large-scale adjustment, the stitching algorithm still needs a certain amount of time to converge to accurate stitching parameters, which may affect the system response speed in some urgent or rapidly changing scenarios. (4) Stability and reliability: In complex or low-feature environments, the stability and reliability of dynamic calibration will be affected. For example, at night or in bad weather conditions, the feature point detection effect is poor, which may lead to a decrease in system performance.

[0079] Based on the above analysis, it can be seen that static calibration cannot obtain stitched images in the case of vehicle body replacement, and dynamic calibration has accuracy and latency issues in obtaining stitched images in the case of vehicle body replacement. In order to accurately obtain stitched images in the case of vehicle body replacement, this application provides a communication method, which may include: acquiring inherent parameters of a camera of a first vehicle, including at least video codec parameters, and acquiring inherent parameters of a camera of a second vehicle, including at least video codec parameters; obtaining a stitched image based on the inherent parameters of the camera of the first vehicle and the inherent parameters of the camera of the second vehicle, wherein the stitched image is a panoramic image of the second vehicle attached to the first vehicle.

[0080] The video encoding / decoding parameters are used to parse the images captured by the camera. The inherent parameters also include one or more of the following: a first parameter, camera intrinsic parameters, and camera extrinsic parameters; the first parameter indicates the mapping relationship between each pixel in the stitched image and the pixels in the image captured by the camera. Thus, compared to transmitting images or video stream data captured by the camera, the transmission of key parameters that positively impact obtaining the stitched image is added, namely the inherent parameters of the camera of the first vehicle and the camera of the second vehicle. This facilitates the rapid and accurate acquisition of the stitched image after dynamic changes in the vehicle body, such as the stitched image of the second vehicle attached to the first vehicle, based on the inherent parameters of the cameras of the first and second vehicles.

[0081] Compared to dynamically calibrated image stitching technology, this solution offers improvements in accuracy, stability, reliability, and real-time performance. These improvements are detailed below:

[0082] I. Accuracy

[0083] It is well known that image stitching accuracy based on static calibration is higher than that based on dynamic calibration. The technical solution of this application standardizes the transmission, fusion, and updating of the static calibration schemes of the first vehicle and the second vehicle to achieve dynamic updating of the stitched image after the second vehicle is attached to the first vehicle. Essentially, it achieves the effect of static calibration of the entire vehicle (the whole after the second vehicle is attached to the first vehicle). Therefore, the image stitching accuracy of the technical solution of this application should be higher than that based on dynamic calibration.

[0084] II. Stability and Reliability

[0085] Compared to image stitching technology based on dynamic calibration, the technical solution of this application does not rely on real-time images captured by the vehicle. Complex or low-feature environments will not reduce the image stitching accuracy of the technical solution of this application, nor will they cause a decrease in system performance. Therefore, the stability and reliability of the technical solution of this application should be higher than that of the dynamic calibration-based solution.

[0086] III. Real-time performance

[0087] Compared to image stitching technology based on dynamic calibration, the technical solution of this application does not require calibration time. Therefore, the real-time performance of the technical solution of this application should be higher than that of the dynamic calibration-based technology.

[0088] The communication method provided in the embodiments of this application will be described below with reference to the accompanying drawings.

[0089] Figure 1 is a schematic diagram of the architecture of a communication system provided in an embodiment of this application. As shown in Figure 1, the communication system includes a first vehicle and a second vehicle. Optionally, in a scenario consisting of a tractor and a trailer, the first vehicle in Figure 1 can be a tractor, and the second vehicle in Figure 1 can be a trailer. As shown in Figure 1, the first vehicle is equipped with a central control module and a camera subsystem, and the first vehicle includes a first mounting component. The second vehicle is equipped with a camera subsystem, and the second vehicle includes a second mounting component. Optionally, the communication system shown in Figure 1 may further include a vehicle attitude estimation module, which can be deployed in both the first and second vehicles.

[0090] The central control module, acting as the hub of the surround-view system, is primarily responsible for receiving and processing the inherent parameters and video stream data from the camera subsystems of the first and second vehicles to obtain stitched images, as well as controlling the entire communication process. Optionally, the central control module can also interact with the vehicle attitude estimation module to obtain the heading angle between the first and second vehicles, optimizing the stitched images.

[0091] The camera subsystem of the first vehicle can consist of cameras deployed on the first vehicle (such as camera 1 and camera 3). Its main responsibility is to provide the central control module with video stream data acquired by the cameras of the first vehicle and the inherent parameters of the cameras, such as video encoding / decoding parameters, the first parameters of the cameras, the internal parameters of the cameras (which can be referred to as camera intrinsic parameters), and the external parameters of the cameras (which can be referred to as camera extrinsic parameters), to support the surround-view function. The cameras included in the camera subsystem of the first vehicle can be connected to the central control module via a communication link to transmit the video stream data acquired by the cameras and the inherent parameters of the cameras.

[0092] The camera of the first vehicle (such as camera 1 and camera 3) may include modules / units for acquiring images, including modules / units for processing the acquired images, and modules / units for transmitting data (such as images or video streams acquired by the camera of the first vehicle, and data after processing of the acquired images or video streams). The module / unit for processing the acquired images or video streams may be deployed within the camera of the first vehicle or may be deployed independently. Optionally, the camera of the first vehicle may also include a unit / module for storing data.

[0093] The camera subsystem of the second vehicle can consist of cameras deployed on the second vehicle (such as camera 2 and camera 4). Its main responsibility is to provide the central control module with video stream data acquired by the cameras of the second vehicle and the inherent parameters of the cameras, such as video encoding / decoding parameters, initial parameters of the cameras, internal parameters of the cameras, and external parameters of the cameras, to support the surround-view function. The cameras included in the camera subsystem of the second vehicle can be connected to the central control module via a communication link to transmit the video stream data acquired by the cameras of the second vehicle and the inherent parameters of the cameras.

[0094] The cameras of the second vehicle (such as cameras 2 and 4) may include modules / units for acquiring images, including modules / units for processing the acquired images, and modules / units for transmitting data (such as images or video streams acquired by the cameras of the second vehicle, and data after processing of the acquired images or video streams). The modules / units for processing the acquired images or video streams may be deployed within the cameras of the second vehicle or may be deployed independently. Optionally, the cameras of the second vehicle may also include units / modules for storing data.

[0095] It should be understood that the camera subsystem of the first vehicle and the camera subsystem of the second vehicle shown in Figure 1 support hot-swapping capability. That is, in response to the second vehicle being attached to the first vehicle, the camera subsystem of the first vehicle and the camera subsystem of the second vehicle can quickly pair, transmit the inherent parameters of the camera of the first vehicle and the inherent parameters of the camera of the second vehicle, and dynamically adjust the inherent parameters when the relative posture between the first vehicle and the second vehicle changes, so as to dynamically adjust the stitched image.

[0096] The vehicle attitude estimation module is used to estimate the vehicle attitude information between the first and second vehicles, particularly the heading angle between them, which is crucial for dynamically adjusting the stitched images. The vehicle attitude estimation module transmits this heading angle to the central control module via a communication link. The input data for the vehicle attitude estimation module may include sensor data from the central control module, such as vehicle control information, inertial measurement unit (IMU) information, and global navigation satellite system (GNSS) information. It may also include video streams from different cameras in the surround-view system or specific information processed from those streams.

[0097] The first and second attachment components are used to attach the second vehicle to the first vehicle.

[0098] It is understood that the cameras in the communication system shown in Figure 1, such as the cameras deployed in the first vehicle and the cameras deployed in the second vehicle (e.g., between camera 1 and camera 2, between camera 3 and camera 4, or between camera 1 and camera 4, between camera 2 and camera 3), the cameras deployed in the first vehicle (e.g. between camera 1 and camera 3), and the cameras deployed in the second vehicle (e.g. between camera 2 and camera 4), can communicate directly or through forwarding by other devices. This application embodiment does not specifically limit this.

[0099] It is understood that Figure 1 above is merely a schematic diagram and does not constitute a limitation on the applicable scenarios of the technical solutions provided in this application. Those skilled in the art should understand that in specific implementations, the communication system may include fewer devices than those shown in Figure 1, or the communication system may include other devices. Furthermore, the number of devices in the communication system can be determined according to specific needs and is not limited. The devices in the system shown in Figure 1 are described below.

[0100] Optionally, the devices in Figure 1, such as the first vehicle, the second vehicle, the central control module, the camera subsystem of the first vehicle, the subsystem of the second camera, the vehicle attitude estimation module, etc., can also be referred to as communication devices. They can be general-purpose devices or special-purpose devices. This application embodiment does not specifically limit them.

[0101] Optionally, the functions of the devices in Figure 1 of this application, such as the first vehicle, the second vehicle, the central control module, the camera subsystem of the first vehicle, the camera subsystem of the second vehicle, and the vehicle attitude estimation module, can be implemented by one device, multiple devices working together, or by one or more functional modules within a single device. This application does not specifically limit these functions. It is understood that the aforementioned functions can be network elements in hardware devices, software functions running on dedicated hardware, a combination of hardware and software, or virtualization functions instantiated on a platform (e.g., a cloud platform).

[0102] The solutions described in this application embodiment can also be applied to communication systems, and the corresponding names can be replaced by the names of the corresponding functions in the communication system. Furthermore, the systems described in this application embodiment are for the purpose of more clearly illustrating the technical solutions of this application embodiment and do not constitute a limitation on the technical solutions provided in this application embodiment. Those skilled in the art will understand that with the evolution of network architecture and the emergence of new service scenarios, the technical solutions provided in this application embodiment are also applicable to similar technical problems.

[0103] The communication method provided in this application embodiment will be described below with reference to the communication system shown in Figure 1, taking the interaction between communication nodes / devices as an example. It should be noted that in the following embodiments of this application, the message names, parameter names, or information names between communication nodes / devices are merely examples, and may be different names in other embodiments. The method provided in this application does not specifically limit these names. It is understood that in the embodiments of this application, each communication node / device may execute some or all of the steps in the embodiments of this application. These steps or operations are merely examples, and the embodiments of this application may also execute other operations or variations of various operations. Furthermore, the steps may be executed in different orders as presented in the embodiments of this application, and it is not necessary to execute all the operations in the embodiments of this application.

[0104] It is understood that this application uses a communication device as an example to illustrate the interaction, but this application does not limit the execution subject of the interaction. For example, the method executed by the communication device in this application can also be executed by a module (e.g., a chip, chip system, or processor) applied to the communication device, or it can be implemented by a logic node, logic module, or software that can implement all or part of the functions of the communication device.

[0105] The following description, with reference to Figure 2, illustrates a communication method provided in an embodiment of this application. This communication method can be executed by a first vehicle or by a functional module within the first vehicle, such as a central control module, without limitation. The following description uses the execution by the central control module within the first vehicle as an example to illustrate the communication method. As shown in Figure 2, the method may include the following steps S201-S203:

[0106] S201: The central control module obtains the inherent parameters of the camera of the first vehicle.

[0107] Specifically, the central control module can obtain the inherent parameters of the camera of the first vehicle through the camera subsystem of the first vehicle. For example, the camera subsystem of the first vehicle sends the inherent parameters of the camera of the first vehicle to the central control module, and correspondingly, the central control module receives the inherent parameters of the camera of the first vehicle from the camera subsystem of the first vehicle.

[0108] In this context, the first vehicle is shown in Figure 1. The camera subsystem of the first vehicle refers to the camera subsystem of the first vehicle in the communication system shown in Figure 1, and the central control module refers to the central control module in the communication system shown in Figure 1. The first vehicle may include, but is not limited to, a tractor unit with towing capabilities and a display that provides information to the operator. The tractor unit with towing capabilities may also be a vehicle with towing capabilities, such as a tractor truck.

[0109] In this application, the inherent parameters of the camera include at least video encoding / decoding parameters, which are used to analyze the images captured by the camera. Video encoding / decoding parameters may include one or more of the following: codec format, frame rate, and bitrate. Codec format refers to the compression and encoding method of video data. Frame rate is a measure of the number of displayed frames, representing the number of times the graphics processor can update per second when processing a scene. Bitrate refers to the number of bits of video data transmitted per unit time, determining the video quality and file size.

[0110] In addition, the camera's inherent parameters may include one or more of the following: primary parameters, intrinsic parameters, and extrinsic parameters.

[0111] The first parameter indicates the mapping relationship between each pixel in the stitched image and the pixels in the image captured by the camera. The stitched image is a panoramic image of the second vehicle attached to the first vehicle. The first parameter may include pixel mapping tables corresponding to the cameras of each vehicle, such as a first pixel mapping table corresponding to the camera of the first vehicle and a second pixel mapping table corresponding to the camera of the second vehicle, or it may include parameters from the pixel mapping tables corresponding to the cameras of each vehicle. The first pixel mapping table describes how each pixel in the image captured by the camera of the first vehicle is mapped to a pixel position in the stitched image. The second pixel mapping table describes how each pixel in the image captured by the camera of the second vehicle is mapped to a pixel position in the stitched image. The parameters in the pixel mapping tables corresponding to the cameras of each vehicle can refer to existing pixel mapping tables, which will not be elaborated here.

[0112] Camera intrinsic parameters describe the internal geometric and optical characteristics of the camera. These parameters may include one or more of the following: focal length, principal point position, pixel size, lens distortion parameters, etc. Camera intrinsic parameters determine how the camera projects points from three-dimensional time onto a two-dimensional image plane. Focal length (f) is usually expressed in pixels and represents the focal length on the image plane. In practical applications, focal length can be divided into the focal length in the x-direction and the focal length in the y-direction, denoted as fx and fy, respectively. x and f y This is because the pixels of an image sensor may not be square. The principal point, also known as the image center point, is usually represented by (c... x c y The center point of the image plane is represented by a scalar (p1), which is the intersection of the optical axis and the image plane. Pixel size describes the actual physical size of each pixel, usually in micrometers (μm). Lens distortion parameters, also known as distortion parameters or distortion coefficients, can be categorized into radial distortion, tangential distortion, and non-polynomial distortion models. Radial distortion describes the lens distortion at different radii and is typically represented by polynomial coefficients such as k1, k2, k3, etc. Tangential distortion describes the lens distortion in different directions and is usually represented by p1, p2, etc. Non-polynomial distortion models can include piecewise linear models, spline interpolation models, and lookup table-based models. Piecewise linear models divide the image into multiple regions, each using a different linear model to describe the distortion. Spline interpolation models use spline functions to fit the distortion and are suitable for complex distortion situations. The lookup table-based model pre-calculates the distortion value of each pixel and stores the distortion value in the lookup table. The lookup table-based model is suitable for real-time applications.

[0113] Camera extrinsic parameters describe the camera's position and orientation in the world coordinate system, determining its rotation and translation relative to the world coordinate system. These parameters can include a rotation matrix, rotation vector, Euler angles, quaternions, and a 4x4 homogeneous transformation matrix. The rotation matrix, typically a 3x3 matrix, represents the rotation of the camera coordinate system relative to the world coordinate system and can be represented as R. The rotation vector represents the rotation axis and angle, and can be converted into a 3D vector using the Rodrigues formula. Euler angles represent rotation using three angles (usually rotation around the x-axis, y-axis, and z-axis). A quaternion is a four-dimensional number used to represent rotation and is typically represented as (q0, q1, q2, q3). The 4x4 homogeneous transformation matrix is ​​a 4x4 matrix that contains both rotation and translation information and is typically represented as... Where R is a 3x3 rotation matrix and t is a 3x1 translation vector.

[0114] It should be understood that in practical applications, camera intrinsic and extrinsic parameters are usually used together to form a complete camera model. For example, a common camera model can be represented as: Where u and v are pixel coordinates, f x f y c x c y R and t are described above and will not be repeated here.

[0115] Furthermore, the camera's inherent parameters may also include panoramic image parameters. Panoramic image parameters include one or more of the following: the resolution of the stitched image, the pixel coordinates of the traction axis in the stitched image, and the actual ground area corresponding to the stitched image.

[0116] The resolution of a stitched image can typically be expressed as the number of pixels in each direction, or as its width and height. The width of the stitched image can be defined as the maximum value of the v-axis in its pixel coordinate system, and the height as the maximum value of the u-axis. The pixel coordinate system can be defined with the top-left corner of the image as the origin and pixels as the coordinate unit. In this system, the u-axis represents the column number of a pixel in the image, and the v-axis represents the row number of a pixel.

[0117] The pixel coordinates of the traction axle in the stitched image refer to the pixel coordinates of the traction axle within the pixel coordinates of the stitched image. The pixel coordinates of the stitched image are described above and will not be repeated here. Optionally, the traction axle can be the connection point between the first and second mounting components in Figure 1. Optionally, the origin of the vehicle body coordinate system can be located at the traction axle, with the x-axis pointing forward of the vehicle, the y-axis pointing to the left side of the vehicle, and the z-axis pointing upwards of the vehicle.

[0118] The actual ground range corresponding to the stitched image can refer to the actual ground range of the physical world corresponding to the stitched image. It can be represented by the coordinates of the pixel coordinates representing the range of the stitched image in the unified world coordinate system. The unified world coordinate system is used to describe the position of each point in the physical world of the vehicle body (vehicle coordinate system).

[0119] For example, taking a rectangular stitched image as an example, the range of the stitched image can be represented by the pixel coordinates of the diagonal points in the pixel coordinates of the stitched image. For example, the pixel coordinates of the upper left corner of the stitched image correspond to (x0, y0) in the unified world coordinate system, and the pixel coordinates of the lower right corner of the stitched image correspond to (x1, y1) in the unified world coordinate system. The actual ground range corresponding to the stitched image is from x0 to x1 and from y0 to y1.

[0120] Furthermore, the inherent parameters may also include the communication protocol version. The communication protocol version includes supported application layer protocol versions, which can be the surround-view system communication protocol version. The surround-view communication protocol is mainly used to standardize the communication mechanism between various components in the surround-view system, such as the central control module, the camera subsystem of the first vehicle, the camera subsystem of the second vehicle, and the vehicle attitude estimation module. The communication mechanism may include data transmission formats, control commands, status reports, etc. The data transmission format may include the transmission format of the camera's inherent parameters and the transmission format of various parameters during the surround-view system initialization process.

[0121] The camera subsystem of the first vehicle can send the inherent parameters of the camera of the first vehicle to the central control module according to the transmission format predefined in the protocol, or the camera subsystem of the first vehicle can send the inherent parameters of the camera of the first vehicle to the central control module according to the negotiated transmission format. The negotiated transmission format can refer to the transmission format determined by the central control module, or the transmission format determined by the camera subsystem of the first vehicle and the central control module through negotiation.

[0122] S202: The central control module obtains the inherent parameters of the camera in the second vehicle.

[0123] Specifically, the central control module can obtain the inherent parameters of the second vehicle's camera through the second vehicle's camera subsystem. For example, the second vehicle's camera subsystem sends the inherent parameters of the second vehicle's camera to the central control module, and correspondingly, the central control module receives the inherent parameters of the second vehicle's camera from the second vehicle's camera subsystem.

[0124] In this context, the camera subsystem of the second vehicle refers to the camera subsystem of the second vehicle in the communication system shown in Figure 1, and the central control module refers to the central control module in the communication system shown in Figure 1.

[0125] The inherent parameters are described in the relevant descriptions above and will not be repeated here.

[0126] The camera subsystem of the second vehicle can send the inherent parameters of the camera of the second vehicle to the central control module according to the transmission format predefined in the protocol, or the camera subsystem of the second vehicle can send the inherent parameters of the camera of the second vehicle to the central control module according to the negotiated transmission format. The negotiated transmission format can refer to the transmission format determined by the central control module, or the transmission format negotiated and determined by the camera subsystem of the second vehicle and the central control module.

[0127] Optionally, to ensure the security of data transmission between the camera subsystems of each vehicle (such as the camera subsystems of the first vehicle and the second vehicle) and the central control module, each vehicle's camera subsystem can encrypt the inherent parameters of its own vehicle's camera and then send the encrypted inherent parameters to the central control module. The central control module decrypts the received inherent parameters to obtain the inherent parameters of the vehicle's camera. Alternatively, each vehicle's camera subsystem (such as the camera subsystems of the first vehicle and the second vehicle) can send its own inherent parameters to the central control module according to a predefined security protocol.

[0128] Optionally, to ensure the accuracy and consistency of the acquired inherent parameters of the cameras of each vehicle, and to avoid inaccurate stitched images due to incorrect parameters, the central control module can verify the inherent parameters of the cameras of each vehicle (e.g., the camera subsystem of the first vehicle, the camera subsystem of the second vehicle) after acquiring the inherent parameters of the cameras of each vehicle. For example, the parameter verification process may include: verifying whether the value of a parameter in the inherent parameters of a vehicle's camera is within the threshold range of the parameter; if the value of a parameter in the inherent parameters of a vehicle's camera is within the threshold range, the inherent parameter verification of the vehicle's camera is successful; if the value of a parameter in the inherent parameters of a vehicle's camera is not within the threshold range, the inherent parameter verification of the vehicle's camera fails, and the inherent parameters of the vehicle's camera need to be acquired again.

[0129] For example, taking the central control module obtaining the inherent parameters of the camera of the first vehicle through the camera subsystem of the first vehicle and then verifying the inherent parameters of the camera of the first vehicle as an example, if the value of the principal point position in the inherent parameters of the camera of the first vehicle is less than the floating-point value of the resolution, and the value of the resolution is a positive integer, the inherent parameters of the camera of the first vehicle are successfully verified; if the value of the principal point position in the inherent parameters of the camera of the first vehicle is greater than or equal to the floating-point value of the resolution, or the value of the resolution is not a positive integer, the inherent parameters of the camera of the first vehicle fail to be verified, and the inherent parameters of the camera of the first vehicle need to be obtained again.

[0130] S203: The central control module obtains a stitched image based on the inherent parameters of the camera of the first vehicle and the inherent parameters of the camera of the second vehicle.

[0131] The inherent parameters of the camera in the first vehicle and the inherent parameters of the camera in the second vehicle are described above and will not be repeated here.

[0132] Among them, the stitched image can be used to represent a seamless panoramic view of the environment around the first vehicle and the second vehicle, so the stitched image can also be called a panoramic image.

[0133] In one example, the inherent parameters include at least a first parameter and panoramic image parameters. The central control module can obtain a first pixel mapping table and a second pixel mapping table, normalize the first pixel mapping table and the second pixel mapping table to obtain a global pixel mapping table, and further obtain the stitched image based on the global pixel mapping table.

[0134] The first parameter can be either the original parameter or a parameter processed from the original parameter, such as a parameter after type conversion, data validation, or data correction. The first parameter is described in the relevant description above and will not be repeated here. The global pixel mapping table covers every pixel in the stitched image, recording which camera each pixel corresponds to and its pixel coordinates within that camera. Panoramic image parameters are described in the relevant description above and will not be repeated here.

[0135] The first pixel mapping table is determined by the first parameters of the camera of the first vehicle, and the second pixel mapping table is determined by the first parameters of the camera of the second vehicle, which may include the following two possible cases:

[0136] Scenario 1: If the first parameter includes the pixel mapping table corresponding to the camera of each vehicle, the pixel mapping table corresponding to the camera of the first vehicle is the first pixel mapping table, and the pixel mapping table corresponding to the camera of the second vehicle is the second pixel mapping table. Therefore, the first pixel mapping table can be directly obtained from the first parameter of the camera of the first vehicle, and the second pixel mapping table can be directly obtained from the first parameter of the camera of the second vehicle.

[0137] Scenario 2: If the first parameter includes the parameters in the pixel mapping table corresponding to the camera of each vehicle, the central control module can integrate the parameters in the pixel mapping table corresponding to the camera of the first vehicle to obtain the first pixel mapping table; and integrate the parameters in the pixel mapping table corresponding to the camera of the second vehicle to obtain the second pixel mapping table.

[0138] Optionally, normalizing the first and second pixel mapping tables to obtain a global pixel mapping table may include: mapping each pixel value in the first and second pixel mapping tables to a preset range, such as [0, 1], using a normalization method; and further merging the normalized first and second pixel mapping tables into a global pixel mapping table. Common normalization methods may include max-min normalization, Z-score normalization, etc.

[0139] Optionally, obtaining the stitched image based on the global pixel mapping table may include: mapping each pixel recorded in the global pixel mapping table to the pixel coordinates corresponding to each pixel to obtain the stitched image.

[0140] In another example, the inherent parameters include at least camera intrinsic and extrinsic parameters. The central control module can calculate the correspondence between the resolution of each camera and the resolution of the stitched image based on the camera intrinsic and extrinsic parameters. Furthermore, based on this correspondence, a global pixel mapping table is generated. Finally, the stitched image is obtained based on the global pixel mapping table. The correspondence between the resolution of each camera and the resolution of the stitched image can be a mathematical transformation relationship between them. The camera intrinsic and extrinsic parameters are described above and will not be repeated here.

[0141] For example, the central control module can calculate the mathematical transformation between the resolution of each camera and the resolution of the stitched image based on the camera's intrinsic and extrinsic parameters. This mathematical transformation maps the pixel positions in the images captured by each camera to the pixel positions in the stitched image. Furthermore, the pixel positions in the images captured by each camera are mapped to their corresponding pixel positions in the stitched image using the same mathematical transformation, thus generating a global pixel mapping table. The stitched image is then obtained based on this global pixel mapping table. Existing techniques for calculating the mathematical transformation between the resolution of each camera and the stitched image based on camera intrinsic and extrinsic parameters can be referenced and will not be elaborated upon here.

[0142] Optionally, to ensure the integrity and accuracy of the stitched image, the central control module can verify the stitching result after obtaining the image. This could involve displaying the stitched image on a monitor, allowing the vehicle operator to confirm whether the image fully covers the surrounding environment of the first and second vehicles, and whether the image accurately represents the surrounding environment of the first and second vehicles. If the stitching result verification is successful, the stitched image is successfully obtained; if the verification fails, a new stitched image is acquired.

[0143] Based on the communication method shown in Figure 2, compared to transmitting image or video stream data acquired by the camera, the transmission of key parameters that have a positive impact on obtaining the stitched image is added, namely the inherent parameters of the camera of the first vehicle and the inherent parameters of the camera of the second vehicle. This allows the processing unit / module, such as the central control module, to quickly and accurately obtain the stitched image after the dynamic changes of the vehicle body, such as the stitched image of the second vehicle attached to the first vehicle, based on the inherent parameters of the camera of the first vehicle and the inherent parameters of the camera of the second vehicle.

[0144] Optionally, to ensure secure and efficient communication, the communication method shown in Figure 2 may further include the following steps before step S201:

[0145] S200: In response to the fact that both the cameras of the first vehicle and the second vehicle support hot-swapping, the central control module performs a matching authentication operation on the cameras of the first vehicle and the second vehicle.

[0146] The cameras of the first vehicle and the second vehicle both support hot-swapping, meaning that the cameras of the first vehicle and the second vehicle can be used immediately without static calibration of the first and second vehicles. That is, after the second vehicle is attached to the first vehicle, the camera subsystems of the first vehicle and the camera subsystems of the second vehicle can support the following matching and authentication operations.

[0147] In one example, the central control module's execution of the matching authentication operation may include: a connection establishment process and an identity authentication process, each of which is described below:

[0148] The connection establishment process may include: the central control module establishing a connection with the camera subsystem of the first vehicle and the camera subsystem of the second vehicle via wired or wireless means. If the connection is successfully established, the central control module discovers the cameras included in the camera subsystem of the first vehicle and the cameras included in the camera subsystem of the second vehicle through wireless or wired scanning. The cameras included in the camera subsystem of the first vehicle are the cameras of the first vehicle, and the cameras included in the camera subsystem of the second vehicle are the cameras of the second vehicle.

[0149] The identity authentication process may include: the central control module authenticates the cameras of the first vehicle and the second vehicle through the authentication module, thereby establishing a secure and reliable communication relationship between the central control module and the cameras of the first and second vehicles, and preventing unauthorized cameras from interacting with the central control module.

[0150] For example, the central control module can transmit the identification identifiers of the cameras in the first and second vehicles to the authentication module. If the identification identifiers of the cameras in the first and second vehicles are already registered in the authentication module, the authentication module sends a confirmation message to the central control module. This confirmation message indicates that the authentication of the cameras in the first and second vehicles has been successful, and data transmission can then occur between the central control module, the cameras in the first and second vehicles. If the identification identifiers of the cameras in the first and second vehicles are not registered in the authentication module, the authentication module sends a negative response to the central control module. This negative response indicates that the authentication of the cameras in the first and second vehicles has failed, and the cameras in the first and second vehicles need to register their identities with the authentication module to achieve successful authentication. The camera's identification identifier is used to uniquely identify the camera and can be its product number, serial number, etc.

[0151] Optionally, changes in the relative pose between the first vehicle and the second vehicle will also change the stitched image of the surrounding environment of the first vehicle and the second vehicle. Therefore, in order to ensure the accuracy and real-time performance of the stitched image when the relative pose between the first vehicle and the second vehicle changes, the communication method shown in Figure 2 may further include the following steps after step S203:

[0152] S204: The central control module adjusts the stitched image based on the heading angle between the first vehicle and the second vehicle.

[0153] The heading angle between the first vehicle and the second vehicle refers to the difference between the heading angle of the first vehicle and the heading angle of the second vehicle. The heading angle usually refers to the angle between the vehicle's direction of travel and a certain reference direction (such as due north).

[0154] Optionally, the central control module can adjust the stitched image based on the heading angle between the first vehicle and the second vehicle. This adjustment may include: the central control module obtaining the heading angle between the first vehicle and the second vehicle, adjusting the global pixel mapping table according to the heading angle, and further adjusting the stitched image according to the adjusted global pixel mapping table.

[0155] For example, after the central control module obtains the heading angle between the first vehicle and the second vehicle, it uses the heading angle to adjust the inherent parameters of the camera of the first vehicle and the camera of the second vehicle. Then, it uses the adjusted inherent parameters of each camera to obtain a new global pixel mapping table. Each pixel recorded in the new global pixel mapping table is mapped to the pixel coordinates corresponding to each pixel to obtain a new stitched image.

[0156] In one example, the central control module can obtain the heading angle between the first vehicle and the second vehicle based on the video stream data collected by the camera of the first vehicle and the video stream data collected by the second vehicle. For example, the central control module can send the video stream data and sensor data to the vehicle attitude estimation module. The video stream data includes the video stream data collected by the camera of the first vehicle and the video stream data collected by the second vehicle. The vehicle attitude estimation module obtains the heading angle between the first vehicle and the second vehicle based on the video stream data and sensor data. The vehicle attitude estimation module then sends the heading angle to the central control module.

[0157] Sensor data can include one or more of the following: angle sensor data, body control data, and Controller Area Network (CAN) data. Angle sensor data refers to angle data provided by angle sensors, such as the steering wheel rotation angle. Body control data can refer to data characterizing the vehicle's current motion state, such as data measured by accelerometers or gyroscopes. CAN data refers to various types of data transmitted on the CAN bus, such as engine control data and chassis control data.

[0158] For example, the vehicle attitude estimation module receives video stream data and sensor data from the central control module. Using image analysis techniques and machine learning algorithms, it performs feature point detection and matching on the video stream data to obtain multiple matching feature point pairs. The direction of each matching feature point pair is calculated. Through statistical analysis of the directions of multiple feature point pairs, the initial heading angle between the first and second vehicles can be estimated. Furthermore, the vehicle attitude estimation module can use a Kalman filter algorithm to optimize the initial heading angle (i.e., the initial heading angle between the first and second vehicles) to obtain the optimized heading angle. The input data for the Kalman filter algorithm can include sensor data and the initial heading angle, and the output data can be the optimized heading angle.

[0159] In another example, the central control module can obtain the heading angle between the first and second vehicles based on other data from the first and second vehicles. Other data refers to data other than the video stream data captured by the camera, such as data measured by the inertial navigation system. An inertial navigation system is an autonomous navigation system that can provide information such as the speed, heading angle, and position of an object equipped with the system.

[0160] For example, the central control module can receive the heading angle of the first vehicle provided by the inertial navigation system of the first vehicle, and the heading angle of the second vehicle provided by the inertial navigation system of the second vehicle, and calculate the difference between the heading angle of the first vehicle and the heading angle of the second vehicle to obtain the heading angle between the first vehicle and the second vehicle.

[0161] As can be seen from the above process, the communication method provided in this application embodiment can include three processes: inherent parameter transmission, stitched image initialization, and stitched image adjustment. To better understand this application, the following describes these three processes in conjunction with the interaction behavior between the first camera subsystem, the central control module, the second camera subsystem, and the vehicle posture estimation module in the system architecture shown in Figure 1:

[0162] 1. Inherent parameter transmission process

[0163] Figure 3 is a schematic diagram of an inherent parameter transmission process provided in an embodiment of this application. As shown in Figure 3, the inherent parameter transmission process may include: S301 establishing a connection, S302 identity authentication, S303 transmission of inherent parameters of the camera, and S304 parameter verification. The inherent parameters of the camera include the inherent parameters of the camera of the first vehicle and the inherent parameters of the camera of the second vehicle.

[0164] As shown in Figure 3, after successful identity authentication, the inherent parameters of the camera are transmitted. After the inherent parameters of the camera are transmitted, the S301 parameter verification is performed. If the parameter verification is successful, the pairing between the modules is completed, that is, the pairing between the central control module, the camera subsystem of the first vehicle and the camera subsystem of the second vehicle is successful. The modules can transmit data safely, efficiently and accurately, and execute the following stitching image initialization process.

[0165] The execution process of establishing a connection in S301 and authenticating identity in S302 in the above parameter transmission process can be referred to the relevant description in S200. The execution process of transmitting and verifying the camera's inherent parameters can be referred to the relevant descriptions in S201 and S202 above, and will not be repeated here.

[0166] 2. Image stitching initialization process

[0167] Figure 4 is a schematic diagram of a stitched image initialization process provided in an embodiment of this application. As shown in Figure 4, the stitched image initialization process may include: S401 collecting first parameters and panoramic image parameters, S402 parameter normalization, S403 pixel mapping table merging, and S404 stitching result verification.

[0168] In Figure 4, S401 collects the first parameters and panoramic image parameters, which may include acquiring the first parameters of the camera of the first vehicle, acquiring the panoramic image parameters corresponding to the first parameters of the camera of the first vehicle, acquiring the first parameters of the camera of the second vehicle, and acquiring the panoramic image parameters corresponding to the first parameters of the camera of the second vehicle; S402 parameter normalization in Figure 4 refers to normalizing each first parameter and the panoramic image parameters corresponding to each first parameter; S403 pixel mapping table merging in Figure 4 refers to merging the first pixel mapping table and the second pixel mapping table to generate a global pixel mapping table.

[0169] As shown in Figure 4, after successfully collecting the first parameter and panoramic image parameters in step S401, parameter normalization is performed in step S402. After parameter normalization in step S402, pixel mapping table merging is performed in step S403. If pixel mapping table merging in step S403 is successful, a global pixel mapping table is obtained. After obtaining the global pixel mapping table, stitching result verification can be performed in step S404. If the stitching result verification is successful, it indicates that the stitched image initialization is complete. If the relative pose between the first vehicle and the second vehicle changes, the following stitched image adjustment process is executed. If the stitching result verification fails, a new stitched image is acquired.

[0170] Figure 5 is a schematic diagram of another image stitching initialization process provided in the embodiment of this application. As shown in Figure 5, the image stitching initialization process may include: S501 collecting camera intrinsic parameters and camera extrinsic parameters, S502 calculating mathematical transformation relationships, S503 generating a global pixel mapping table, and S504 verifying the stitching results.

[0171] In Figure 5, S501 collects camera intrinsic and extrinsic parameters, which may include acquiring the camera intrinsic and extrinsic parameters of the camera of the first vehicle and acquiring the camera intrinsic and extrinsic parameters of the camera of the second vehicle; S502 in Figure 5 calculates mathematical transformation, which refers to calculating the mathematical transformation relationship between each camera and the resolution of the stitched image; S503 in Figure 5 generates a global pixel mapping table, which refers to generating a global pixel mapping table based on the mathematical transformation between each camera and the resolution of the stitched image.

[0172] As shown in Figure 5, after successfully collecting the camera's intrinsic and extrinsic parameters in step S501, step S502 calculates the mathematical transformation relationship; after calculating the mathematical transformation relationship in S502, step S503 generates a global pixel mapping table, resulting in the global pixel mapping table; after obtaining the global pixel mapping table, step S504 verifies the stitching result. If the stitching result verification is successful, it indicates that the stitched image initialization is complete. If the relative pose between the first and second vehicles changes, the following stitched image adjustment process can be executed; if the stitching result verification fails, a new stitched image is acquired.

[0173] The specific descriptions of each operation in the above image stitching initialization process can be found in the relevant descriptions in S203 above, and will not be repeated here.

[0174] 3. Image stitching adjustment process

[0175] Figure 6 is a schematic diagram of a stitched image adjustment process provided in an embodiment of this application. As shown in Figure 6, the stitched image adjustment process may include: S601 the central control module transmits video stream data and sensor data to the attitude estimation module; S602 the attitude estimation module obtains the heading angle (i.e., the heading angle between the first vehicle and the second vehicle); S603 the attitude estimation module transmits the heading angle to the central control module; and S604 the central control module adjusts the stitched image.

[0176] The specific descriptions of each operation in the above-mentioned image stitching adjustment process can be found in the relevant descriptions in S204 above, and will not be repeated here.

[0177] The foregoing mainly describes the solutions provided in the embodiments of this application from the perspective of the interaction between various devices. It is understood that each device, such as the first vehicle, the second vehicle, the central control module, the camera subsystem of the first vehicle, the camera subsystem of the second vehicle, the vehicle attitude estimation module, etc., includes corresponding hardware structures and / or software modules for executing each function in order to achieve the above-mentioned functions. Those skilled in the art should readily recognize that, in conjunction with the algorithm steps of the various examples described in the embodiments disclosed herein, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed in a hardware or computer software-driven hardware manner depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0178] This application embodiment can group the central control module, the camera subsystem of the first vehicle, and the camera subsystem of the second vehicle into functional modules according to the above method example. For example, each functional group can be assigned to a specific functional module, or two or more functions can be integrated into one processing module. The integrated module can be implemented in hardware or as a software functional module. It should be noted that the grouping of modules in this application embodiment is illustrative and only represents one logical functional grouping; other grouping methods may be used in actual implementation.

[0179] Figure 7 shows a structural diagram of a communication device 700, which can be used to perform the functions of the central control module involved in the above embodiments. As one possible implementation, the communication device 700 shown in Figure 7 includes: a transceiver unit 701 and a processing unit 702;

[0180] The transceiver unit 701 is used to acquire inherent parameters of the camera of the first vehicle and inherent parameters of the camera of the second vehicle. The inherent parameters include at least video encoding / decoding parameters used to parse images captured by the cameras. The inherent parameters also include one or more of the following: a first parameter, camera intrinsic parameters, and camera extrinsic parameters. The first parameter indicates the mapping relationship between each pixel in the stitched image and the pixels in the image captured by the cameras. For example, the transceiver unit 701 can support the communication device 700 in executing S201 and S202.

[0181] Optionally, the transceiver unit 701 is also used to obtain the heading angle between the first vehicle and the second vehicle.

[0182] Optionally, the transceiver unit 701 is further configured to send video stream data and sensor data to the vehicle attitude estimation module. The video stream data includes video stream data collected by each camera in the first vehicle and the second vehicle. The sensor data includes one or more of the following: angle sensor data, vehicle control data, and vehicle controller local area network data.

[0183] Processing unit 702 is configured to obtain a stitched image based on the inherent parameters of the camera of the first vehicle and the inherent parameters of the camera of the second vehicle. The stitched image is a panoramic image of the second vehicle attached to the first vehicle. For example, processing unit 702 may support communication device 700 in executing S203.

[0184] Optionally, the processing unit 702 is further configured to, in response to the fact that both the camera of the first vehicle and the camera of the second vehicle support hot-swapping, discover the camera of the first vehicle and the camera of the second vehicle; and authenticate the camera of the first vehicle and the camera of the second vehicle. For example, the processing unit 702 may also support the communication device 700 in executing S200.

[0185] Optionally, the processing unit 702 is further configured to adjust the global pixel mapping table according to the heading angle between the first vehicle and the second vehicle; and adjust the stitched image according to the adjusted global pixel mapping table. For example, the processing unit 702 may also support the communication device 700 in executing S204.

[0186] The descriptions of inherent parameters, heading angle between the first and second vehicles, video stream data, sensor data, stitched images, and global pixel mapping tables can be found in the above method embodiments.

[0187] Specifically, all relevant content of each step involved in the central control module in the method embodiment shown in Figure 2 can be referenced from the functional description of the corresponding functional module, and will not be repeated here. The communication device 700 is used to execute the function of the central control module in the communication method shown in Figure 2, and therefore can achieve the same effect as the above-described communication method.

[0188] Figure 8 shows a structural diagram of a communication device 800, which can be used to perform the functions of the camera subsystem of the first vehicle involved in the above embodiments. As one possible implementation, the communication device 800 shown in Figure 8 includes: a transceiver unit 801;

[0189] The transceiver unit 801 is used to transmit inherent parameters of the camera of the first vehicle. These inherent parameters are used to determine a stitched image, which is a panoramic image of the second vehicle attached to the first vehicle. The inherent parameters include at least video encoding / decoding parameters used to parse the image captured by the camera. The inherent parameters also include one or more of the following: a first parameter, camera intrinsic parameters, and camera extrinsic parameters. The first parameter indicates the mapping relationship between each pixel in the stitched image and the pixels in the image captured by the camera. For example, the transceiver unit 801 can support the communication device 800 in executing S201.

[0190] The descriptions of the inherent parameters and the stitched images can be found in the above method embodiments.

[0191] Specifically, all relevant content of each step involved in the camera subsystem of the first vehicle in the method embodiment shown in Figure 2 can be referenced from the functional description of the corresponding functional module, and will not be repeated here. The communication device 800 is used to execute the function of the camera subsystem of the first vehicle in the communication method shown in Figure 2, and therefore can achieve the same effect as the above-described communication method.

[0192] Figure 9 shows a structural diagram of a communication device 900, which can be used to perform the functions of the camera subsystem of the second vehicle involved in the above embodiments. As one possible implementation, the communication device 900 shown in Figure 9 includes: a transceiver unit 901;

[0193] The transceiver unit 901 is used to transmit inherent parameters of the camera of the second vehicle. These inherent parameters are used to determine the stitched image, which is a panoramic image of the second vehicle mounted on top of the second vehicle. The inherent parameters include at least video encoding / decoding parameters used to parse the image captured by the camera. The inherent parameters also include one or more of the following: a first parameter, camera intrinsic parameters, and camera extrinsic parameters. The first parameter indicates the mapping relationship between each pixel in the stitched image and the pixels in the image captured by the camera. For example, the transceiver unit 901 can support the communication device 900 in executing S202.

[0194] The descriptions of the inherent parameters and the stitched images can be found in the above method embodiments.

[0195] Specifically, all relevant content of each step involved in the camera subsystem of the second vehicle in the method embodiment shown in Figure 2 can be referenced from the functional description of the corresponding functional module, and will not be repeated here. The communication device 900 is used to execute the function of the camera subsystem of the second vehicle in the communication method shown in Figure 2, and therefore can achieve the same effect as the above-described communication method.

[0196] The processing unit mentioned above can be a processing module, a processor, or a controller. It can implement or execute various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. A processor can also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc. A transceiver unit can be a communication module, a transceiver circuit, or a communication interface, etc. Any of the communication devices mentioned above can also include a storage unit for storing the program code and data of any communication device. The storage unit can be a storage module or a memory. When the processing module is a processor, the communication module is a communication interface, and the storage module is a memory, the communication devices 700, 800, and 900 involved in the embodiments of this application can be the communication device 1000 shown in FIG. 10. For example, the central control module, the camera subsystem of the first vehicle, and the camera subsystem of the second vehicle mentioned above can adopt the composition structure shown in FIG. 10 or include the components shown in FIG. 10. Figure 10 is a schematic diagram of the composition of a communication device 1000 provided in an embodiment of this application. As shown in Figure 10, the communication device 1000 may include a processor 1001, and optionally, may also include a communication line 1002 and a communication interface 1003.

[0197] Furthermore, the communication device 1000 may also include a memory 1004. The processor 1001, the memory 1004, and the communication interface 1003 can be connected via a communication line 1002.

[0198] The processor 1001 can be a central processing unit (CPU), a network processor (NP), a digital signal processor (DSP), a microprocessor, a microcontroller, a programmable logic device (PLD), or any combination thereof. The processor 1001 can also be other communication devices with processing capabilities, such as circuits, devices, or software modules.

[0199] Communication line 1002 is used to transmit information between the components included in communication device 1000.

[0200] Communication interface 1003 is used for communication with other devices or other communication networks. These other communication networks can be Ethernet, radio access network (RAN), wireless local area network (WLAN), etc. Communication interface 1003 can be a radio frequency module, transceiver, or any communication device capable of communication. This application embodiment uses a radio frequency module as an example to illustrate communication interface 1003. The radio frequency module can include an antenna, radio frequency circuitry, etc., and the radio frequency circuitry can include a radio frequency integrated chip, a power amplifier, etc.

[0201] Memory 1004 is used to store instructions. These instructions can be computer programs.

[0202] The memory 1004 can be a read-only memory (ROM) or other type of static storage device that can store static information and / or instructions; it can also be a random access memory (RAM) or other type of dynamic storage device that can store information and / or instructions; it can also be an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage, magnetic disk storage medium or other magnetic storage device. Optical disc storage includes compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.

[0203] It should be noted that the memory 1004 can exist independently of the processor 1001, or it can be integrated with the processor 1001. The memory 1004 can be used to store instructions, program code, or some data, etc. The memory 1004 can be located inside or outside the communication device 1000, without limitation. The processor 1001 is used to execute the instructions stored in the memory 1004 to implement the random access procedure preamble transmission method provided in the following embodiments of this application.

[0204] In one example, processor 1001 may include one or more CPUs, such as CPU0 and CPU1 in Figure 10.

[0205] As an optional implementation, the communication device 1000 may include multiple processors, for example, in addition to the processor 1001 in FIG10, it may also include a processor 1007.

[0206] As an optional implementation, the communication device 1000 also includes an output device 1005 and an input device 1006. The input device 1006 is a keyboard, mouse, microphone, or joystick, etc., and the output device 1005 is a display screen, speaker, etc.

[0207] It should be noted that the communication device 1000 can be a desktop computer, a portable computer, a web server, a mobile phone, a tablet computer, a wireless terminal, an embedded device, a chip system, or a device with a similar structure to that shown in Figure 10. Furthermore, the composition shown in Figure 10 does not constitute a limitation on the communication device. In addition to the components shown in Figure 10, the communication device may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0208] In this embodiment of the application, the chip system may be composed of chips or may include chips and other discrete devices.

[0209] This application also provides a computer-readable storage medium. All or part of the processes in the above method embodiments can be implemented by a computer program instructing related hardware. This program can be stored in the computer-readable storage medium, and when executed, it can include the processes of the above method embodiments. The computer-readable storage medium can be a terminal device of any of the foregoing embodiments, such as an internal storage unit including a data transmission end and / or a data receiving end, like a hard disk or memory of the terminal device. The computer-readable storage medium can also be an external storage device of the terminal device, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the terminal device. Further, the computer-readable storage medium can include both internal storage units and external storage devices of the terminal device. The computer-readable storage medium is used to store the computer program and other programs and data required by the terminal device. The computer-readable storage medium can also be used to temporarily store data that has been output or will be output.

[0210] It should be understood that the collection, storage, use, processing, transmission, provision, and disclosure of user personal information involved in the technical solution of this application all comply with relevant laws and regulations and do not violate public order and good morals. For example, in the technical solution of this application, the processing of user personal information is carried out with the user's authorization, and this will not be repeated hereafter.

[0211] It should be noted that the terms "first" and "second," etc., in the specification, claims, and drawings of this application are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or apparatuses.

[0212] It should be understood that in this application, "at least one (item)" means one or more, "more than one" means two or more, "at least two (items)" means two or three or more, and "and / or" is used to describe the relationship between related objects, indicating that there can be three relationships. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the related objects before and after are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0213] It should be understood that in the embodiments of this application, "B corresponding to A" means that B is associated with A. For example, B can be determined based on A. It should also be understood that determining B based on A does not mean that B is determined solely based on A; B can also be determined based on A and / or other information. Furthermore, the term "connection" in the embodiments of this application refers to various connection methods, such as direct connection or indirect connection, to achieve communication between devices, and the embodiments of this application do not impose any limitations on this.

[0214] Unless otherwise specified, the term "transmission" in the embodiments of this application refers to bidirectional transmission, encompassing the actions of sending and / or receiving. Specifically, "transmission" in the embodiments of this application includes sending data, receiving data, or both sending and receiving data. In other words, data transmission here includes uplink and / or downlink data transmission. Data may include channels and / or signals; uplink data transmission refers to uplink channel and / or uplink signal transmission, and downlink data transmission refers to downlink channel and / or downlink signal transmission. The terms "network" and "system" in the embodiments of this application refer to the same concept; a communication system is a communication network.

[0215] Through the above description of the embodiments, those skilled in the art can clearly understand that, for the sake of convenience and brevity, only the grouping of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0216] In the several embodiments provided in this application, it should be understood that the disclosed communication devices and methods can be implemented in other ways. For example, the communication device embodiments described above are merely illustrative. For instance, the grouping of modules or units is only a logical functional grouping, and in actual implementation, there may be other grouping methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces, or indirect coupling or communication connection between devices or units, and may be electrical, mechanical, or other forms.

[0217] The units described as separate components may or may not be physically separate. A component shown as a unit can be one or more physical units; that is, it can be located in one place or distributed in multiple different locations. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0218] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0219] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, essentially, or the parts that contribute to the prior art, or all or part of the technical solutions, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions to cause a device, such as a microcontroller, chip, or processor, to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media for storing program code, such as USB flash drives, portable hard drives, ROM, RAM, magnetic disks, or optical disks.

[0220] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A communication method, characterized in that, The method includes: The inherent parameters of the camera of the first vehicle and the inherent parameters of the camera of the second vehicle are obtained; the inherent parameters include at least video encoding and decoding parameters, which are used to parse the images captured by the camera; the inherent parameters also include one or more of the following: a first parameter, camera intrinsic parameters, and camera extrinsic parameters; the first parameter is used to indicate the mapping relationship between each pixel in the stitched image and the pixels in the image captured by the camera; A stitched image is obtained based on the inherent parameters of the camera of the first vehicle and the inherent parameters of the camera of the second vehicle. The stitched image is a panoramic image of the second vehicle attached to the first vehicle.

2. The method according to claim 1, characterized in that, The inherent parameters also include panoramic image parameters, which include one or more of the following: the resolution of the stitched image, the pixel coordinates of the traction axis in the stitched image, and the actual ground area corresponding to the stitched image.

3. The method according to claim 1 or 2, characterized in that, The inherent parameters also include the communication protocol version, which includes supported application layer protocol versions.

4. The method according to any one of claims 1-3, characterized in that, The obtained stitched image includes: Obtain a first pixel mapping table and a second pixel mapping table; the first pixel mapping table is determined by the first parameter of the camera of the first vehicle, and the second pixel mapping table is determined by the first parameter of the camera of the second vehicle. The first pixel mapping table and the second pixel mapping table are normalized to obtain a global pixel mapping table; The stitched image is obtained based on the global pixel mapping table.

5. The method according to any one of claims 1-3, characterized in that, The obtained stitched image includes: Based on the camera intrinsic parameters and the camera extrinsic parameters, calculate the correspondence between the resolution of each camera and the resolution of the stitched image; The global pixel mapping table is generated based on the correspondence between the resolution of each camera and the stitched image; The stitched image is obtained based on the global pixel mapping table.

6. The method according to claim 4 or 5, characterized in that, The method further includes: Obtain the heading angle between the first vehicle and the second vehicle; Adjust the global pixel mapping table according to the heading angle; The stitched image is adjusted according to the adjusted global pixel mapping table.

7. The method according to claim 6, characterized in that, The method further includes: The system sends video stream data and sensor data to the vehicle attitude estimation module. The video stream data includes video stream data collected by each camera in the first vehicle and the second vehicle. The sensor data includes one or more of the following: angle sensor data, vehicle control data, and vehicle controller local area network (CAN) data. The step of obtaining the heading angle between the first vehicle and the second vehicle includes: receiving the heading angle from the vehicle attitude estimation module.

8. The method according to any one of claims 1-7, characterized in that, The method further includes: In response to the fact that both the camera of the first vehicle and the camera of the second vehicle support hot-swapping, the camera of the first vehicle and the camera of the second vehicle are detected. The cameras of the first vehicle and the second vehicle are authenticated.

9. A communication method, characterized in that, The method includes: The inherent parameters of the camera of the first vehicle are sent, the inherent parameters being used to determine the stitched image, the stitched image being a panoramic image of the second vehicle attached to the first vehicle; The inherent parameters include at least video encoding / decoding parameters used to parse images captured by the camera; the inherent parameters also include one or more of the following: a first parameter, camera intrinsic parameters, and camera extrinsic parameters; the first parameter is used to indicate the mapping relationship between each pixel in the stitched image and the pixels in the image captured by the camera.

10. The method according to claim 9, characterized in that, The inherent parameters also include panoramic image parameters, which include one or more of the following: the resolution of the stitched image, the pixel coordinates of the traction axis in the stitched image, and the actual ground area corresponding to the stitched image.

11. The method according to claim 9 or 10, characterized in that, The inherent parameters also include the communication protocol version and / or transmission mode, wherein the communication protocol version includes supported application layer protocol versions.

12. A communication method, characterized in that, The method includes: Send the inherent parameters of the camera of the second vehicle, the inherent parameters being used to determine the stitched image, the stitched image being a panoramic image of the second vehicle attached to the first vehicle; The inherent parameters include at least video encoding / decoding parameters used to parse images captured by the camera; the inherent parameters also include one or more of the following: a first parameter, camera intrinsic parameters, and camera extrinsic parameters; the first parameter is used to indicate the mapping relationship between each pixel in the stitched image and the pixels in the image captured by the camera.

13. The method according to claim 12, characterized in that, The inherent parameters also include panoramic image parameters, which include one or more of the following: the resolution of the stitched image, the pixel coordinates of the traction axis in the stitched image, and the actual ground area corresponding to the stitched image.

14. The method according to claim 12 or 13, characterized in that, The inherent parameters also include the communication protocol version, which includes supported application layer protocol versions.

15. A communication device, characterized in that, The communication device includes a module or unit for performing the method as described in any one of claims 1-8, or the communication device includes a module or unit for performing the method as described in any one of claims 9-11, or the communication device includes a module or unit for performing the method as described in any one of claims 12-14.

16. A communication device, characterized in that, The communication device includes a processor, the processor being configured to support the communication device in performing the method as described in any one of claims 1-8, or the processor being configured to support the communication device in performing the method as described in any one of claims 9-11, or the processor being configured to support the communication device in performing the method as described in any one of claims 12-14.

17. A communication system, characterized in that, The communication system includes a communication device for performing the method as described in any one of claims 1-8, a communication device for performing the method as described in any one of claims 9-11, and a communication device for performing the method as described in any one of claims 12-14.

18. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that, when executed on a computer, cause the computer to perform the method as described in any one of claims 1-8, or the method as described in any one of claims 9-11, or the method as described in any one of claims 12-14.

19. A computer program product, characterized in that, The computer program product includes computer instructions that, when executed on a computer, cause the computer to perform the method as described in any one of claims 1-8, or the method as described in any one of claims 9-11, or the method as described in any one of claims 12-14.

Images