Reversing display method and related apparatus

Abstract

A reversing display method and a related apparatus. The method comprises: acquiring a plurality of first images, wherein the plurality of first images comprise environment images, which are collected by a vehicle from a plurality of directions, and the plurality of directions at least comprise the front and rear of the vehicle; stitching the plurality of first images to obtain a second image, wherein the second image comprises a front region picture and a rear region picture of the vehicle, and the area of the rear region picture in the second image is greater than the area of the front region picture in the second image; and displaying the second image in a reversing image of the vehicle.

Description

Reversing display method and related devices

[0001] This application claims priority to Chinese patent application No. 202410918214.0, filed on July 9, 2024, the entire contents of which are incorporated herein by reference. Technical Field

[0002] This disclosure relates to the field of vehicle image processing, and more particularly to a reversing display method and related apparatus. Background Technology

[0003] A vehicle's panoramic image is a 360° bird's-eye view of the vehicle created by rendering and stitching together multiple images captured from different directions using image processing and other methods. The vehicle then displays this panoramic image to the user through an in-vehicle display, which can reduce traffic accidents caused by blind spots. Summary of the Invention

[0004] This disclosure provides a reversing display method and related apparatus. During vehicle reversing, the area of ​​the rear region of the vehicle in the stitched second image is larger than the area of ​​the front region, improving safety during reversing.

[0005] Firstly, a method for displaying a reversing vehicle is provided, the method comprising:

[0006] Acquire multiple first images, the first images including environmental images captured by the vehicle from multiple directions, the multiple directions including at least the front and rear;

[0007] A second image is obtained based on the multiple first images. The second image includes a front area view and a rear area view of the vehicle. The area of ​​the rear area view in the second image is larger than the area of ​​the front area view in the second image.

[0008] The second image is displayed in the reversing camera view of the vehicle.

[0009] In the above method, the vehicle stitches multiple first images into a second image, making the rear area of ​​the second image larger than the front area, thus providing the user with a wider rear view. This improves driving safety and user operability.

[0010] In some embodiments, the origin of the camera coordinate system corresponding to the second image is located in the region between the geometric center of the vehicle and the rear of the vehicle.

[0011] In the above method, the camera coordinate system corresponding to the second image can be understood as the viewfinder of a virtual camera, and the origin of the camera coordinate system can be understood as the position of the virtual camera. In related technologies, the virtual camera is located at the geometric center of the vehicle, making the area of ​​the region behind the vehicle in the second image equal to the area of ​​the region in front of the vehicle, which is not conducive to the user's observation of the rear area. However, in some embodiments of this disclosure, the origin of the camera coordinate system is located in the region between the geometric center of the vehicle and the rear of the vehicle. That is, compared with related technologies, the position of the virtual camera in some embodiments of this disclosure has moved towards the rear of the vehicle. Therefore, the viewfinder of the virtual camera has also moved towards the rear of the vehicle, making the area behind the vehicle in the viewfinder larger than the area in front of the vehicle. Therefore, the imaging area of ​​elements containing the rear area of ​​the vehicle in multiple first images is also correspondingly increased in the second image, making the area of ​​the rear area of ​​the vehicle in the second image larger than the area of ​​the front area.

[0012] In some embodiments, obtaining the second image based on the plurality of first images includes:

[0013] The multiple first images are transformed from being represented in pixel coordinates to being represented in world coordinates;

[0014] The multiple first images, represented in the world coordinate system, are converted to be represented in the camera coordinate system.

[0015] The multiple first images, represented in the camera coordinate system, are converted to be represented in the image coordinate system to obtain the second image.

[0016] In the above method, the vehicle transforms multiple first images from pixel coordinates to world coordinates, which can be understood as performing a model transformation on multiple first images. Because each first image has its own corresponding pixel coordinate system, transforming multiple first images to the same coordinate system makes subsequent transformations easier.

[0017] The vehicle transforms multiple first images represented in the world coordinate system to those represented in the camera coordinate system, which can be understood as performing a view transformation on these first images. A common view transformation operation can be compared to taking a photograph with a camera: moving the camera to the desired shooting position, the elements presented in the camera's viewfinder are the elements that can be seen or rendered. Therefore, transforming multiple first images from the world coordinate system to the camera coordinate system can be understood as determining the elements that can be rendered from the multiple first images through the viewfinder of a virtual camera, i.e., determining the second image. The vehicle transforms multiple first images represented in the camera coordinate system to those represented in the image coordinate system, which can be understood as performing a projection transformation on these first images. The view transformation is equivalent to determining the position of the virtual camera's viewfinder, i.e., determining the imaging area of ​​the multiple first images in the second image. The projection transformation is equivalent to pressing the shutter button of the virtual camera, allowing the second image to be formed.

[0018] In some embodiments, converting the plurality of first images represented in the world coordinate system to those represented in the camera coordinate system includes:

[0019] Determine the origin of the camera coordinate system;

[0020] The transformation matrix is ​​determined based on the origin of the camera coordinate system;

[0021] Based on the transformation matrix, the multiple first images represented in the world coordinate system are transformed to be represented in the camera coordinate system.

[0022] In the above method, determining the origin of the camera coordinate system is to determine the position of the virtual camera during view transformation; the position of the virtual camera can be the origin of the camera coordinate system. Based on the origin of the camera coordinate system, the transformation matrix corresponding to transforming the first image to the camera coordinate system is determined. Then, the coordinate system transformation is performed based on the transformation matrix, transforming the physical space transformation into mathematical calculations, thus simplifying the image processing and image stitching steps.

[0023] In some embodiments, the second image includes a trajectory line configured to indicate the direction of rotation of the vehicle, the direction of rotation indicated by the trajectory line being consistent with the direction of rotation of the steering wheel angle.

[0024] In the above method, the rotation direction of the trajectory line in the processed second image is consistent with the rotation direction of the steering wheel. Therefore, based on the processed second image, the user can intuitively perceive the vehicle's steering situation at the current steering wheel angle, as well as the area where the vehicle will reverse. Thus, the user can more easily perceive how to adjust the steering wheel angle to control the vehicle to reverse to the desired position. Therefore, based on the trajectory line whose rotation direction is consistent with the steering wheel angle, the user can intuitively perceive the relationship between the steering wheel angle and the trajectory line. This improves driving safety without affecting the algorithm's real-time performance and robustness.

[0025] In some embodiments, the trajectory line is determined based on the vehicle's direction of travel and the steering wheel angle, and the trajectory line is also configured to indicate the vehicle's rotation angle.

[0026] In the above method, the trajectory line is determined based on the vehicle's direction of travel and the steering wheel angle. Therefore, users can perceive changes in the vehicle's direction of travel and / or steering based on changes in the trajectory line, thus enabling better vehicle control. For example, when the vehicle is traveling forward, the user focuses more on the vehicle's forward movement. Therefore, the trajectory line can be forward along the vehicle's direction, indicating the area the vehicle will travel in. When the vehicle is traveling backward, the user focuses more on the vehicle's backward movement. Therefore, the trajectory line can be backward along the vehicle's direction, indicating the area the vehicle will reverse in.

[0027] In some embodiments, displaying the second image in the reversing camera of the vehicle includes:

[0028] The second image, which includes the trajectory line, is displayed in the reversing image of the vehicle after being rotated and mirrored.

[0029] In the above method, by rotating and mirroring the second image, the rear of the virtual vehicle in the second image can be positioned above the front of the virtual vehicle, and the front of the virtual vehicle below the second image. Since the trajectory lines are used to indicate the vehicle's direction and angle of rotation, when the direction of travel is reverse, the trajectory lines are displayed behind the rear of the vehicle. That is, after processing the second image, the trajectory lines are still displayed behind the rear of the vehicle. Correspondingly, after processing the second image, the trajectory lines are also displayed above the second image. Therefore, by processing the second image, the rotation direction of the trajectory lines can be made consistent with the rotation direction of the steering wheel angle.

[0030] In some embodiments, stitching the plurality of first images together to obtain a second image includes:

[0031] In the scenario of the vehicle reversing, the multiple first images are stitched together to obtain the second image.

[0032] In the above method, since the vehicle needs to provide the user with a large rear view when reversing, the vehicle stitches multiple first images into a second image, making the rear area of ​​the second image larger than the front area, thereby providing the user with a larger rear view.

[0033] Secondly, a reversing display device is provided, the device including a communication unit and a processing unit.

[0034] The communication unit is configured to acquire multiple first images, the first images including environmental images captured by the vehicle from multiple directions, the multiple directions including at least the front and rear;

[0035] The processing unit is configured to obtain a second image based on the plurality of first images, the second image including a front area image and a rear area image of the vehicle, wherein the area of ​​the rear area image in the second image is larger than the area of ​​the front area image in the second image;

[0036] The communication unit is also configured to display the second image in the vehicle's reversing camera view.

[0037] In some embodiments, the origin of the camera coordinate system corresponding to the second image is located in the region between the geometric center of the vehicle and the rear of the vehicle.

[0038] In some embodiments, the processing unit is further configured to obtain the second image based on the plurality of first images, including:

[0039] Transform multiple of the first images from being represented in pixel coordinates to being represented in world coordinates;

[0040] The multiple first images, represented in the world coordinate system, are converted to be represented in the camera coordinate system.

[0041] The multiple first images, represented in the camera coordinate system, are converted to be represented in the image coordinate system to obtain the second image.

[0042] In some embodiments, the processing unit is further configured to transform the plurality of first images represented in the world coordinate system to those represented in the camera coordinate system, including:

[0043] Determine the origin of the camera coordinate system;

[0044] The transformation matrix is ​​determined based on the origin of the camera coordinate system;

[0045] Based on the transformation matrix, the multiple first images represented in the world coordinate system are transformed to be represented in the camera coordinate system.

[0046] In some embodiments, the second image includes a trajectory line configured to indicate the direction of rotation of the vehicle, the direction of rotation indicated by the trajectory line being consistent with the direction of rotation of the steering wheel angle.

[0047] In some embodiments, the trajectory line is determined based on the vehicle's direction of travel and steering wheel angle, and the trajectory line is also configured to indicate the vehicle's rotation angle.

[0048] In some embodiments, the communication unit is further configured to display the second image in the reversing camera image of the vehicle, including:

[0049] The second image, which includes the trajectory line, is displayed in the reversing image of the vehicle after being rotated and mirrored.

[0050] In some embodiments, the processing unit is further configured to stitch the plurality of first images together to obtain a second image, including:

[0051] In the scenario of the vehicle reversing, the multiple first images are stitched together to obtain the second image.

[0052] Thirdly, an electronic device is provided, comprising a processor and a memory; the processor is coupled to the memory, the memory being configured to store a computer program, and the processor being configured to invoke and run the computer program such that the electronic device performs the method described in the first aspect above.

[0053] In some embodiments, the electronic device further includes a communication interface configured to receive and / or transmit data, and / or configured to provide input and / or output to the processor.

[0054] It should be noted that the above embodiments are illustrated using a processor (or general-purpose processor) that executes the method by invoking a computer-specified instruction. In practice, the processor can also be a dedicated processor, in which case the computer instructions are pre-loaded into the processor. In some embodiments, the processor may include both dedicated and general-purpose processors.

[0055] In some embodiments, the processor and memory are integrated into a single device.

[0056] Fourthly, a vehicle is provided that includes the electronic equipment described in the third aspect.

[0057] Fifthly, a computer-readable storage medium is provided, wherein a computer program is stored therein, which, when executed on a computer or processor, implements the method described in the first aspect above.

[0058] The beneficial effects of the technical solutions provided in the second to fifth aspects in some embodiments of this disclosure can be referred to the beneficial effects of the technical solutions in the first aspect, and will not be repeated here. Attached Figure Description

[0059] The accompanying drawings used in the description of some embodiments of this disclosure will be briefly introduced below.

[0060] Figure 1 is a schematic diagram of a trajectory line according to some embodiments;

[0061] Figure 2 is a schematic diagram of the architecture of a vehicle according to some embodiments;

[0062] Figure 3 is a flowchart of a reversing display method according to some embodiments;

[0063] Figure 4 is a schematic diagram of a virtual camera according to some embodiments;

[0064] Figure 5 is a schematic diagram of a second image according to some embodiments;

[0065] Figure 6 is a schematic diagram of another trajectory line according to some embodiments;

[0066] Figure 7 is a flowchart of another reversing display method according to some embodiments;

[0067] Figure 8 is a block diagram of a reversing display device according to some embodiments;

[0068] Figure 9 is a schematic diagram of an electronic device according to some embodiments. Detailed Implementation

[0069] The following describes some embodiments of this disclosure in detail with reference to the accompanying drawings.

[0070] The terms "first," "second," "third," and "fourth," etc., in this disclosure, claims, and accompanying drawings are used to distinguish different objects and not to describe a particular order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to such processes, methods, products, or apparatus.

[0071] To facilitate understanding of some embodiments of this disclosure, the technical problems to be solved by this disclosure will be analyzed and proposed below.

[0072] A vehicle's panoramic image is typically a 360° bird's-eye view of the vehicle, or second image, synthesized by rendering and stitching together multiple first images captured from different directions. Accordingly, the vehicle may include multiple camera devices, each used to capture first images from different directions. For example, the panoramic image can be stitched together from first images captured by four cameras located at the front, left front, right front, and rear of the vehicle. The four first images captured by these four cameras cover the environmental information around the vehicle. Therefore, based on the panoramic image stitched together from these four first images, a 360° bird's-eye view of the vehicle containing information about its surrounding environment can be obtained.

[0073] For example, a panoramic image can be stitched together from the first images captured by six cameras on the vehicle. These six cameras are located at the front, front left, front right, rear left, rear right, and rear of the vehicle. The six first images captured by these six cameras can cover the environmental information around the vehicle. Therefore, based on the panoramic image stitched together from these six first images, a 360° bird's-eye view of the vehicle, including information about its surrounding environment, can be obtained.

[0074] The vehicle renders and stitches the four first images based on the intrinsic and extrinsic parameters of the camera devices corresponding to each of the four images, creating a 360° bird's-eye view of the vehicle. Alternatively, the vehicle renders and stitches the six first images based on the intrinsic and extrinsic parameters of the camera devices corresponding to each of the six images, creating a 360° bird's-eye view of the vehicle. The vehicle then displays the composite bird's-eye view on its onboard display.

[0075] However, in panoramic imaging, to ensure that the front, rear, left, and right sides of the vehicle are all displayed, and that the aspect ratio of the panoramic image matches the requirements of the in-vehicle display, the rear area of ​​the vehicle is displayed in a smaller area. For example, if the front and rear areas of the vehicle are the same size in the panoramic image, then during reversing, the displayed area of ​​the rear area will be smaller. Users will not be able to intuitively perceive whether there are obstacles in the distant area behind the vehicle based on the panoramic image, affecting driving safety.

[0076] Therefore, during vehicle reversing, a panoramic imaging system is needed to display a large area behind the vehicle for the user. This provides the user with a wider field of vision behind the vehicle, improving reversing safety.

[0077] The forward region refers to the area in front of the vehicle's hood. In some implementations, the vehicle determines the forward region in the panoramic image based on environmental images captured by a forward-facing camera. For example, the forward region can be the region shown in Figure 5.

[0078] The rear area refers to the area behind the rear of the vehicle. In some implementations, the vehicle determines the rear area in the panoramic image based on environmental images captured by the rear-view camera. For example, the rear area can be the rear area shown in Figure 5.

[0079] In addition, the panoramic image includes virtual vehicles used to indicate the vehicle's direction and angle of rotation. The virtual vehicle's front-end orientation is generally consistent with the user's driving perspective. For example, when a user is driving, the front of the vehicle is in front of the user, meaning the front of the vehicle is facing forward. Therefore, the virtual vehicles included in the panoramic image are generally also displayed facing forward, with the front of the virtual vehicle at the top of the panoramic image and the rear of the virtual vehicle at the bottom.

[0080] Furthermore, the panoramic image also displays trajectory lines. These trajectory lines are extensions along the vehicle's direction and are used to indicate the vehicle's direction and angle of rotation. For example, if the vehicle is turning left, the trajectory line in the panoramic image will be in front of the vehicle and will also be skewed to the left accordingly, indicating the area the vehicle will be traveling in. When the vehicle is reversing, the trajectory line is behind the vehicle, indicating the area the vehicle will be reversing in. For example, during reversing, when the user turns the steering wheel to the right, the rear of the vehicle will turn to the right, and the trajectory line will also skew accordingly. Since the rear of the vehicle and the trajectory line are below the panoramic image, when the rear of the vehicle turns to the right, the trajectory line will skew counterclockwise. However, at this time, turning the steering wheel to the right will cause the steering wheel to turn clockwise. It can be seen that when reversing, the direction of steering wheel rotation and the direction of trajectory line rotation are opposite. Therefore, users cannot intuitively perceive the relationship between the steering wheel rotation angle and the trajectory line rotation through the trajectory line displayed in the panoramic image, thus affecting the safety of reversing the vehicle.

[0081] For example, please refer to Figure 1, which is a schematic diagram of a trajectory line according to some embodiments. As shown in Figure 1, the front of the virtual vehicle is above the panoramic image, and the trajectory line of the vehicle when reversing is behind the virtual vehicle. When the user turns the steering wheel to the right, it can be seen that the rear of the virtual vehicle turns to the right, and the end of the trajectory line also deflects accordingly. At this time, the trajectory line rotates counterclockwise. However, when the user turns the steering wheel to the right, the steering wheel rotates clockwise, so when reversing, the direction of steering wheel rotation is opposite to the direction of trajectory line rotation. The user reversing by operating the steering wheel in the opposite direction to the trajectory line rotation can easily lead to misjudgment and safety accidents.

[0082] In summary, some embodiments of this disclosure provide a reversing display method. During reversing, first images acquired from multiple directions are stitched together to form a second image, such that the area of ​​the rear region in the second image is larger than the area of ​​the front region. The vehicle can also rotate and mirror the second image so that the rotation direction of the trajectory lines contained in the second image is consistent with the rotation direction of the steering wheel. This solves the problem that the area of ​​the rear region of the vehicle in the panoramic image is smaller than the area of ​​the front region when reversing, and also solves the problem that the rotation direction of the trajectory lines in the panoramic image is opposite to the rotation direction of the steering wheel. This increases the user's field of vision while reversing and improves reversing safety.

[0083] The following describes the system architecture used in some embodiments of this disclosure. It should be noted that the system architecture and business scenarios described in this disclosure are for the purpose of more clearly illustrating the technical solutions of this application and do not constitute a limitation on the technical solutions provided by this application. Those skilled in the art will understand that, with the evolution of system architecture and the emergence of new business scenarios, the technical solutions provided by this application are equally applicable to similar technical problems.

[0084] Please refer to Figure 2, which is a schematic diagram of the architecture of a vehicle according to some embodiments. As shown in Figure 2, the vehicle 20 includes a plurality of camera devices and a display device 201. The plurality of camera devices are configured to acquire first images from different directions. For example, the plurality of camera devices include a front-view camera 202, a left-view camera 203, a right-view camera 204, and a rear-view camera 205 as shown in Figure 2. It should be noted that the types, number, and positions of the plurality of camera devices are not limited to the case shown in Figure 2, and are not limited here.

[0085] Vehicle 20 can be a vehicle powered by electricity, a vehicle powered by gasoline, or a vehicle powered by a new energy hybrid powertrain. For example, when vehicle 20 is powered by electricity, it can be a new energy vehicle, such as a pure electric vehicle, a range-extended electric vehicle, a hybrid electric vehicle, or a fuel cell electric vehicle. As another example, when vehicle 20 is powered by gasoline, it can be a car, an agricultural transport vehicle, a tractor, or a trailer. Furthermore, when vehicle 20 is a car, it can be a sedan, an SUV, a truck, a bus, or a van.

[0086] Display device 201 is configured to display a second image. For example, display device 201 can be a vehicle infotainment system, tablet, or other similar device. The vehicle 20 displays the stitched second image to the user via display device 201, allowing the user to control the vehicle 20 to reverse based on the second image displayed on display device 201, thus improving reversing safety.

[0087] The forward-view camera 202 is generally installed on or near the front emblem of the vehicle 20 and is configured to capture images of the front of the vehicle 20. The forward-view camera 202 includes, but is not limited to, a fisheye camera or a wide-angle camera, which is not limited here.

[0088] The left-view camera 203 is generally installed at the left-side rearview mirror of the vehicle 20, or at the lower part of the vehicle body below the left-side rearview mirror, and is configured to capture images of the left side of the vehicle 20. The left-view camera 203 includes, but is not limited to, a fisheye camera or a wide-angle camera, which is not limited here.

[0089] The right-view camera 204 is generally installed at the right-side rearview mirror of the vehicle 20, or at the lower part of the vehicle body below the right-side rearview mirror, and is configured to capture images of the right side of the vehicle 20. The right-view camera 204 includes, but is not limited to, a fisheye camera or a wide-angle camera, which is not limited here.

[0090] The rearview camera 205 is generally installed on or near the rear emblem of the vehicle 20 and is configured to capture images of the area behind the vehicle 20. The rearview camera 205 includes, but is not limited to, a fisheye camera or a wide-angle camera, which is not specified here.

[0091] For example, vehicle 20 acquires a first image of the front of vehicle 20 via front-view camera 202, a first image of the left side of vehicle 20 via left-view camera 203, a first image of the right side of vehicle 20 via right-view camera 204, and a first image of the rear of vehicle 20 via rear-view camera 205. In some embodiments, when vehicle 20 is reversing, vehicle 20 stitches multiple first images together to obtain a second image, in which the rear area of ​​vehicle 20 is larger than the front area of ​​vehicle 20. In some embodiments, when vehicle 20 is not reversing, vehicle 20 stitches multiple first images together to obtain a second image, in which the rear area of ​​vehicle 20 is the same size as the front area of ​​vehicle 20.

[0092] In some embodiments, the camera devices included in the vehicle 20 are not limited to the aforementioned front-view camera 202, left-view camera 203, right-view camera 204, and rear-view camera 205. The vehicle 20 can also stitch together a second image from the first images captured by the six camera devices. For example, the six camera devices can be positioned at the front, front left, front right, rear left, rear right, and rear of the vehicle 20, respectively, to capture the first images from the front, front left, front right, rear left, rear right, and rear of the vehicle 20.

[0093] Please refer to Figure 3, which is a flowchart of a reversing display method according to some embodiments, applied to the vehicle shown in Figure 2. As shown in Figure 3, the method includes, but is not limited to, the following steps S301 to S303.

[0094] Step S301: Acquire multiple first images.

[0095] The multiple first images include environmental images captured by the vehicle from multiple directions, including at least the front and rear. For example, the multiple directions may include the front, rear, left, or right sides of the vehicle.

[0096] For example, a vehicle can capture multiple first images using four cameras. These four cameras are located at the front, left front, right front, and rear of the vehicle, respectively, to capture first images of the front, left, right, and rear of the vehicle. The four first images captured by these four cameras can cover the environmental information around the vehicle, so the vehicle can stitch these four first images together to form a second image.

[0097] For example, a vehicle can capture multiple first images using six cameras. These six cameras are located at the front, front left, front right, rear left, rear right, and rear of the vehicle, respectively, to capture first images of the front, front left, front right, rear left, rear right, and rear of the vehicle. The six first images captured by these six cameras can cover the environmental information around the vehicle, so the vehicle can stitch these six first images together to form a second image.

[0098] Step S302: Obtain the second image based on multiple first images.

[0099] The second image can be a 360° bird's-eye view of the vehicle. The second image includes views of the front and rear areas of the vehicle, with the rear area occupying a larger area than the front area.

[0100] In some embodiments, since the vehicle needs to provide the user with a larger rearward field of view when reversing, the vehicle stitches multiple first images into a second image during the reversing process. This makes the area of ​​the rear region in the second image larger than the area of ​​the front region, thereby providing the user with a larger rearward field of view.

[0101] In some embodiments, the vehicle transforms multiple first images from pixel coordinates to world coordinates. Then, it transforms the multiple first images in world coordinates to multiple first images in camera coordinates. Finally, it transforms the multiple first images in camera coordinates to image coordinates to obtain a second image.

[0102] In some embodiments, a world coordinate system is used to represent multiple first images. Because the multiple first images were acquired by different camera devices, their corresponding pixel coordinate systems are different. Since the multiple first images are to be stitched together into a single second image, it is necessary to first transform them into the same coordinate system for subsequent stitching processing. For example, transforming the multiple first images into the world coordinate system allows for the representation of elements within the same coordinate system, facilitating subsequent stitching processing.

[0103] For example, transforming multiple first images from pixel coordinates to world coordinates can be understood as performing model transformations on them. Model transformation involves setting the model's position and orientation, and then using translation, rotation, and scaling to give the model a suitable position and size. Because each first image has its own corresponding local coordinate system, transforming them to the same coordinate system makes subsequent transformations easier.

[0104] Model transformation involves placing the elements from multiple first images that need to be imaged in the second image within a single scene. In other words, it transforms multiple first images from their respective local coordinate systems to the world coordinate system. This can be understood as placing the elements from the multiple first images within the world coordinate system scene, ensuring that the elements in the first images have appropriate positions and sizes.

[0105] A model transformation is performed on the first image, for example, converting the first image from model space to world space. This is done through affine transformations or similar methods, transforming the coordinates of the first image in model space to coordinates in world space. For example, the coordinates of the first image in model space can be multiplied by the model matrix to obtain the coordinates of the first image in world space, thus completing the transformation of the first image from the pixel coordinate system to the world coordinate system.

[0106] The model matrix consists of three parts: scaling, rotation, and translation. These are used to perform scaling, rotation, and translation transformations on the elements in the first image, respectively, so that the elements in the first image have appropriate size and position in the world coordinate system.

[0107] In some embodiments, the camera coordinate system is configured to determine the imaging area of ​​multiple first images in the second image. Since the size of the second image is limited, the area of ​​the first images presented on the second image is also limited. Therefore, the vehicle transforms the first images from their respective pixel coordinate systems to the world coordinate system to unify the coordinate representation of the multiple first images. The vehicle then transforms the multiple first images from the world coordinate system to the camera coordinate system to determine the imaging area of ​​the multiple first images in the second image, facilitating the rendering and stitching of the multiple first images.

[0108] For example, transforming multiple first images represented in the world coordinate system to multiple first images represented in the camera coordinate system can be understood as performing a view transformation on multiple first images. View transformation, also known as viewing model transformation, refers to observing a model from different positions. A common view transformation operation can be analogized to taking a photograph with a camera: moving the camera to the desired shooting position and pointing it in the desired direction. The elements presented in the camera's viewfinder are the elements that can be seen or rendered. Therefore, transforming multiple first images from the world coordinate system to multiple first images represented in the camera coordinate system can be understood as using a virtual camera's viewfinder to determine the elements that can be rendered from the multiple first images, i.e., determining the second image.

[0109] In some embodiments, the vehicle first determines the origin of the camera coordinate system, and then determines a transformation matrix based on the origin of the camera coordinate system. Based on the transformation matrix, the vehicle transforms multiple first images represented in the world coordinate system to multiple first images represented in the camera coordinate system.

[0110] Determining the origin of the camera coordinate system is to determine the position of the virtual camera in the aforementioned view transformation. The position of the virtual camera can be the origin of the camera coordinate system. In related technologies, the position of the virtual camera, i.e., the origin of the camera coordinate system, is directly above the geometric center of the vehicle. Therefore, in the stitched second image, the area behind the vehicle is equal in size to the area in front of the vehicle, which is not conducive to the user's observation of the rear area when reversing.

[0111] However, in some embodiments of this disclosure, the origin of the camera coordinate system corresponding to the second image is located in the region between the geometric center of the vehicle and the rear of the vehicle. That is, compared with related technologies, the position of the virtual camera used for view transformation in some embodiments of this disclosure is moved towards the rear of the vehicle. Therefore, the viewfinder of the virtual camera is also moved towards the rear of the vehicle, making the area behind the vehicle in the viewfinder larger than the area in front of the vehicle. Therefore, the imaging area of ​​elements containing the rear area of ​​the vehicle in multiple first images is correspondingly increased in the second image, making the area of ​​the rear area of ​​the vehicle in the second image larger than the area of ​​the front area.

[0112] In some embodiments, the first image is transformed from the world coordinate system to the camera coordinate system through view transformation. First, the view space of the camera coordinate system needs to be determined, and then the transformation matrix corresponding to transforming the first image to the camera coordinate system needs to be determined. Determining the view space requires determining the position of the virtual camera, the orientation of the virtual camera, and the vector of the virtual camera's upward direction.

[0113] For example, the position of the virtual camera, that is, the location of the origin of the camera coordinate system, determines the distance between the virtual camera's viewfinder and the elements in the first image. This can be understood as the photographer determining the appropriate distance between themselves and the object (e.g., the elements contained in the first image), placing the object to be photographed within the viewfinder, and the elements displayed in the viewfinder becoming the elements contained in the second image.

[0114] For example, please refer to Figure 4, which is a schematic diagram of a virtual camera according to some embodiments. As shown in Figure 4, the virtual camera is located directly above the area between the geometric center of the vehicle and its rear. The elements in the viewfinder of the virtual camera include the front, rear, left, and right areas of the vehicle. It can be seen that the rear area of ​​the vehicle in the viewfinder is larger than the front area. This allows the area of ​​the rear area of ​​the vehicle in the final image to also be larger than the area of ​​the front area.

[0115] The orientation of the virtual camera refers to the direction in which it observes the model. For example, as shown in Figure 4, the virtual camera is oriented directly downwards towards the vehicle, resulting in a bird's-eye view of the vehicle in the second image.

[0116] For example, the upward vector of the virtual camera can be used to determine the virtual camera's orientation. For instance, although the virtual camera in Figure 4 is facing downwards, it can still tilt forward, backward, left, and right. Therefore, by determining the upward vector of the virtual camera, the image in the viewfinder can be made to be upright, thus ensuring that the resulting second image is also upright.

[0117] Therefore, the vehicle determines the camera coordinate system based on the virtual camera's position, orientation, and upward vector, thus determining the transformation matrix for the view transformation. Then, the vehicle multiplies the coordinates of the first image in world space with the aforementioned transformation matrix to obtain the coordinates of the first image in view space, thereby completing the transformation of the first image from the world coordinate system to the camera coordinate system.

[0118] In some embodiments, the image in the image coordinate system can be displayed in the reversing camera. Since the image in the camera coordinate system is three-dimensional, it needs to be converted from a three-dimensional (3D) image to a two-dimensional (2D) image in order to be displayed on the in-vehicle display. Therefore, the vehicle can convert multiple first images from the camera coordinate system to the image coordinate system so that the resulting second image can be displayed on the in-vehicle display.

[0119] For example, transforming multiple first images represented in a camera coordinate system to those represented in an image coordinate system can be understood as performing a projection transformation on these first images. The projection transformation defines a view space where objects outside the view space are not rendered on the screen. Performing a projection transformation involves transforming points in three-dimensional coordinates onto a 2D coordinate space using a matrix transformation. The view transformation essentially determines the position of the virtual camera's viewfinder, that is, it determines the imaging area of ​​the multiple first images in the second image. The projection transformation, on the other hand, is like pressing the shutter button of the virtual camera, allowing the second image to be captured.

[0120] Performing a projection transformation on the first image can transform it from view space to projection space. For example, through orthogonal projection transformation or perspective projection transformation, the coordinates of the first image in view space can be transformed to coordinates in projection space. Alternatively, the coordinates of the first image in view space can be multiplied by the transformation matrix of the projection transformation to obtain the coordinates of the first image in projection space, thus completing the process of transforming the first image from the camera coordinate system to the image coordinate system.

[0121] For example, please refer to Figure 5, which is a schematic diagram of a second image according to some embodiments. As shown in Figure 5, the second image includes a virtual vehicle located at the center of the second image. The second image also includes a front region, a rear region, a left region, and a right region. It can be seen that the rear region of the second image determined by the virtual camera shown in Figure 4 is larger than the front region, which meets the user's field of vision requirements when reversing.

[0122] In summary, some embodiments of this disclosure use model transformation, view transformation and projection transformation in 3D rendering technology to render and stitch multiple first images into a second image, so that the rear area of ​​the vehicle in the second image is larger than the front area of ​​the vehicle.

[0123] For example, the vehicle renders and stitches multiple first images based on the transformation matrix in the view transformation and the intrinsic and extrinsic parameters of each pre-calibrated camera device to synthesize a second image that includes information about the vehicle's surroundings.

[0124] In some embodiments, the vehicle acquires multiple raw videos through cameras in multiple directions, and then renders and stitches the multiple raw videos into a 360° panoramic bird's-eye view video of the vehicle's surroundings.

[0125] Step S303: Display the second image in the vehicle's reversing camera.

[0126] For example, a vehicle can display a second image to the user via an in-vehicle display, such as a car infotainment system or a tablet. The vehicle displays this stitched second image to the user through the reversing camera system, allowing the user to control the vehicle while reversing based on this image, thus improving reversing safety.

[0127] In some embodiments, to assist the user in driving, the vehicle may also render trajectory lines on the second image. The trajectory lines indicate the vehicle's direction and angle of rotation. The user can perceive changes in the vehicle's attitude based on changes in the trajectory lines, thereby enabling better control of the vehicle.

[0128] For example, when a vehicle is traveling forward, the user is more concerned with changes in the vehicle's forward movement. Therefore, the trajectory line can be forward along the vehicle's direction, and the trajectory line indicates the area the vehicle will travel in. When a vehicle is traveling backward, the user is more concerned with changes in the vehicle's backward movement. Therefore, the trajectory line can be backward along the vehicle's direction, and the trajectory line indicates the area the vehicle will reverse in.

[0129] In some embodiments, the vehicle determines a trajectory line based on its direction of travel and steering wheel angle. A second image containing the trajectory line is then displayed in the vehicle's reversing camera image.

[0130] The vehicle can display a second image directly through the reversing camera, or it can display a second image containing trajectory lines through the reversing camera; there is no restriction here.

[0131] Users control the vehicle's steering angle and direction using the steering wheel. Therefore, the vehicle can determine the desired steering angle based on the steering wheel angle. The vehicle can then directly render a trajectory line that follows the vehicle's movement based on the steering wheel angle. For example, the vehicle can determine the curvature of the trajectory line based on the steering wheel angle; the larger the steering wheel angle, the greater the curvature of the trajectory line. If the vehicle is traveling forward, the trajectory line will move forward along the vehicle's direction.

[0132] In some embodiments, the vehicle rotates and mirrors the second image. Then, a trajectory line determined based on the vehicle's direction of travel and steering wheel angle is displayed on the processed second image.

[0133] For example, the vehicle can be rotated 180° in the second image, and then the rotated second image can be mirrored so that the rear of the virtual vehicle is above the second image and the front of the virtual vehicle is below the second image. Since the trajectory lines are used to indicate the vehicle's direction and angle of rotation, when the vehicle is traveling in reverse, the trajectory lines are displayed behind the rear of the vehicle. That is, after processing the second image, the trajectory lines are still displayed behind the rear of the vehicle. Furthermore, since the rear of the vehicle is now above the second image after processing, the trajectory lines are also displayed above the second image accordingly. In some embodiments, the direction of rotation indicated by the trajectory lines is consistent with the direction of rotation of the steering wheel angle.

[0134] For example, in the second image after rotation and mirroring, the rear of the virtual vehicle is at the top of the second image. Furthermore, when reversing, the rear of the virtual vehicle rotates in the same direction as the steering wheel. For example, when the steering wheel is turned clockwise, the rear of the virtual vehicle also rotates clockwise. Therefore, the trajectory line used to indicate the direction of vehicle rotation also deflects as the vehicle rotates. That is, the trajectory line also deflects clockwise, consistent with the direction of steering wheel rotation.

[0135] For example, please refer to Figure 6, which is a schematic diagram of another trajectory line according to some embodiments. As shown in Figure 6, the second image in Figure 6 is a second image after rotation and mirroring. In the processed second image, the rear of the virtual vehicle is at the top of the image, and the front of the vehicle is at the bottom of the image. Figure 6 shows the posture of the virtual vehicle when the steering wheel is rotated clockwise. It can be seen that the rear of the virtual vehicle also exhibits a clockwise rotation posture. Therefore, behind the rear of the virtual vehicle, the trajectory line also rotates clockwise accordingly, and at this time, the rotation direction of the trajectory line is consistent with the rotation direction of the steering wheel angle.

[0136] It should be noted that Figure 1 is the second image before processing, and Figure 6 is the second image after processing. As can be seen, rotating Figure 1 by 180° and then mirroring it results in Figure 6. In Figure 1, the rear of the virtual vehicle is below the front, and the user's focus is on the lower part of Figure 1. In Figure 6, the rear of the virtual vehicle is above the front, and the user's focus is on the upper part. When the steering wheel is rotated clockwise, the virtual vehicle and trajectory line in Figure 1 rotate counterclockwise, while in Figure 6 they rotate clockwise. Therefore, compared to Figure 1, Figure 6 shows a more intuitive understanding of the vehicle's steering direction and the area where it will reverse, as the trajectory line's rotation direction matches the steering wheel's rotation direction. Thus, users can more easily perceive how to adjust the steering wheel angle to control the vehicle's reversing position based on Figure 6. Therefore, the second image, after rotation and mirroring, provides a more intuitive understanding of the relationship between the steering wheel angle and the trajectory line compared to the original image. This improves driving safety without compromising the algorithm's real-time performance and robustness.

[0137] Please refer to Figure 7, which is a flowchart of another reversing display method according to some embodiments. As shown in Figure 7, the method includes one or more steps in S701-S709.

[0138] S701, acquire multiple first images.

[0139] Multiple first images include environmental images captured by the vehicle from multiple directions, such as the front, rear, left, or right of the vehicle.

[0140] For example, a vehicle can acquire the first image using four cameras. These four cameras are located at the front, left front, right front, and rear of the vehicle, respectively, and are used to acquire the first image from the front, left, right, and rear of the vehicle.

[0141] For example, a vehicle can acquire a first image using six camera devices. These six camera devices are located at the front, left front, right front, left rear, right rear, and rear of the vehicle, respectively, and are used to acquire the first image from the front, left front, right front, left rear, right rear, and rear of the vehicle.

[0142] S702, determine if the vehicle is in reverse gear. If the vehicle is in reverse gear, then execute S703. If the vehicle is not in reverse gear, then execute S704.

[0143] S703, determine the transformation matrix for view transformation during reversing.

[0144] To determine the transformation matrix for the view transformation during reversing, the origin of the camera coordinate system can be determined first. Determining the origin of the camera coordinate system is to determine the position of the virtual camera during the view transformation; the virtual camera's position can be the origin of the camera coordinate system. When the vehicle is reversing, the origin of the camera coordinate system is located in the region between the vehicle's geometric center and its rear. That is, in some embodiments of this disclosure, the position of the virtual camera used for the view transformation has moved towards the rear of the vehicle compared to when the vehicle is not reversing. Therefore, the virtual camera's viewfinder also moves towards the rear of the vehicle, making the area behind the vehicle in the viewfinder larger than the area in front of the vehicle. Therefore, the imaging area of ​​elements containing the rear area of ​​the vehicle in multiple first images is correspondingly increased in the second image, making the area of ​​the rear area of ​​the vehicle in the second image larger than the area of ​​the front area.

[0145] S704, determine the transformation matrix for view transformations when not in reverse.

[0146] Determining the transformation matrix for the view transformation begins by determining the origin of the camera coordinate system. Determining the origin of the camera coordinate system is crucial for determining the position of the virtual camera during the view transformation; the virtual camera's position can be the origin of the camera coordinate system. When the vehicle is not reversing, the origin of the camera coordinate system is located directly above the vehicle's geometric center. Therefore, when the vehicle is not reversing, the area behind the vehicle in the stitched second image is equal in size to the area in front of the vehicle.

[0147] S705, stitches multiple first images together to form a second image.

[0148] The vehicle uses 3D rendering technology, including model transformation, view transformation, and projection transformation, to render and stitch multiple first images into a second image. This results in the rear area of ​​the vehicle being larger than the front area in the second image. For example, the vehicle renders and stitches multiple first images based on the transformation matrix in the view transformation and pre-calibrated intrinsic and extrinsic parameters of each camera device to synthesize a second image that includes information about the vehicle's surroundings.

[0149] Here, the intrinsic and extrinsic parameters of the multiple camera devices used to acquire the first image can be obtained through calibration. For example, the intrinsic parameters of the camera devices can be calibrated using a checkerboard calibration board. This operation can be performed as follows: The camera device to be calibrated takes a photograph of the checkerboard calibration board, ensuring that the checkerboard covers various areas of the camera device at various angles to guarantee the accuracy of its intrinsic parameter estimation. After each camera device captures multiple checkerboard images, the intrinsic parameter matrix and distortion parameters of the camera device are obtained using the camera device intrinsic parameter calibration function. Then, the pose of the camera devices is jointly calibrated based on the bundle adjustment method, thereby achieving the extrinsic parameter calibration of the camera devices. Thus, the intrinsic and extrinsic parameters of the camera devices are obtained through calibration.

[0150] For example, the vehicle captures multiple raw videos through cameras in multiple directions, and then renders and stitches these raw videos into a 360° panoramic bird's-eye view video of the vehicle's surroundings.

[0151] S706, rendering trajectory lines.

[0152] The trajectory lines are used to indicate the vehicle's direction and angle of rotation. Users can perceive changes in the vehicle's attitude by observing changes in the trajectory lines, thus allowing for better vehicle control. For example, when the vehicle is moving forward, the user is more focused on the forward movement. Therefore, the trajectory lines can be placed forward along the vehicle's direction, indicating the area the vehicle will travel. When the vehicle is reversing, the user is more focused on the backward movement. Therefore, the trajectory lines can be placed backward along the vehicle's direction, indicating the area the vehicle will reverse.

[0153] For example, a vehicle determines its trajectory based on the steering wheel angle, and the user controls the vehicle's steering angle and direction using the steering wheel. Therefore, the vehicle can determine the desired steering input by obtaining the steering wheel angle. The vehicle can then directly render a trajectory line that follows the vehicle's movement based on the steering wheel angle.

[0154] S707, determine if the vehicle is in reverse gear. If the vehicle is in reverse gear, then execute S708. If the vehicle is not in reverse gear, then execute S709.

[0155] S708, rotate and mirror the second image.

[0156] For example, the vehicle image can be rotated 180°, and then the rotated image can be mirrored so that the rear of the virtual vehicle is above the front of the second image, and the front of the virtual vehicle is below the second image. Since the trajectory lines are used to indicate the vehicle's direction and angle of rotation, when reversing, the trajectory lines are behind the rear of the vehicle. That is, after processing the second image, the trajectory lines remain behind the rear of the vehicle. Furthermore, since the rear of the vehicle is now above the second image after processing, the trajectory lines are also correspondingly above the second image.

[0157] In the second image, after rotation and mirroring, the rear of the virtual vehicle is positioned above the other image. Furthermore, when reversing, the rear of the virtual vehicle rotates in the same direction as the steering wheel. For example, when the steering wheel is turned clockwise, the rear of the virtual vehicle also rotates clockwise. Therefore, the trajectory line used to indicate the vehicle's rotation direction also deflects as the vehicle rotates. That is, the trajectory line also deflects clockwise, aligning with the direction of the steering wheel's rotation.

[0158] S709, display the second image.

[0159] The vehicle can display a second image to the user via an in-vehicle display. For example, the in-vehicle display could be a car infotainment system, tablet, or other similar device. The vehicle displays this stitched second image to the user through the reversing camera system, allowing the user to control the vehicle while reversing based on this image, thus improving reversing safety.

[0160] The vehicle can display a second image directly through the reversing camera, or it can display a second image containing trajectory lines through the reversing camera; there is no restriction here.

[0161] The methods of some embodiments of this disclosure have been described in detail above. The apparatus of some embodiments of this disclosure is described below.

[0162] Please refer to Figure 8, which is a block diagram of a reversing display device according to some embodiments. The reversing display device 80 may include a communication unit 801 and a processing unit 802. The reversing display device 80 is configured to implement the aforementioned reversing display method, such as the reversing display method shown in Figure 3.

[0163] It should be noted that the above division of multiple units is only a logical division based on function and is not intended to limit the structure of the reversing display device 80. In some embodiments, some functional modules may be subdivided into more smaller functional modules, and some functional modules may be combined into a single functional module.

[0164] In some embodiments, the communication unit 801 is configured to acquire multiple first images. The first images include environmental images captured by the vehicle from multiple directions, including at least the front and rear directions;

[0165] The processing unit 802 is configured to obtain a second image based on multiple first images. The second image includes a front area image and a rear area image of the vehicle, and the area of ​​the rear area image in the second image is larger than the area of ​​the front area image in the second image.

[0166] The communication unit 801 is configured to display a second image in the vehicle's reversing camera.

[0167] In some embodiments, the origin of the camera coordinate system corresponding to the second image is located in the region between the geometric center of the vehicle and the rear of the vehicle.

[0168] In some embodiments, the processing unit 802 is configured to obtain a second image based on a plurality of first images, including:

[0169] Transform multiple first images from being represented in pixel coordinates to being represented in world coordinates;

[0170] Transform multiple first images represented in the world coordinate system into multiple first images represented in the camera coordinate system;

[0171] The first image, which is represented in the camera coordinate system, is transformed into the image coordinate system to obtain the second image.

[0172] In some embodiments, the processing unit 802 is configured to transform multiple first images represented in a world coordinate system to multiple first images represented in a camera coordinate system, including:

[0173] Determine the origin of the camera coordinate system;

[0174] The transformation matrix is ​​determined based on the origin of the camera coordinate system;

[0175] The transformation matrix is ​​used to convert multiple first images represented in the world coordinate system to multiple first images represented in the camera coordinate system.

[0176] In some embodiments, the second image includes a trajectory line configured to indicate the direction of rotation of the vehicle, the direction of rotation indicated by the trajectory line being consistent with the direction of rotation of the steering wheel angle.

[0177] In some embodiments, the trajectory line is determined based on the vehicle's direction of travel and the steering wheel angle, and the trajectory line is also configured to indicate the vehicle's rotation angle.

[0178] In some embodiments, the communication unit 801 is configured to display a second image in the vehicle's reversing camera, including:

[0179] The vehicle's reversing camera displays a second image containing the trajectory lines, which has been rotated and mirrored.

[0180] In some embodiments, the processing unit 802 is configured to stitch together multiple first images to obtain a second image, including:

[0181] In a vehicle reversing scenario, multiple first images are stitched together to obtain a second image.

[0182] It should be noted that, in some embodiments of this disclosure, the implementation and technical effects of each unit can also be described in accordance with the corresponding description of the method embodiment shown in FIG3.

[0183] Please refer to Figure 9, which is a schematic diagram of an electronic device according to some embodiments. As shown in Figure 9, the electronic device 90 may include one or more processors 901, one or more memories 902, and one or more communication interfaces 903. These components may be connected via a bus 904 or other means; Figure 9 illustrates a connection via a bus 904.

[0184] The communication interface 903 can be used by the electronic device 90 to communicate with other communication devices, such as other electronic devices. For example, the communication interface 903 can be a wired interface.

[0185] The memory 902 may be coupled to the processor 901 via a bus 904 or an input / output port, or the memory 902 may be integrated with the processor 901. The memory 902 is configured to store various software programs and / or multiple sets of instructions or data. For example, the memory 902 may be a read-only memory (ROM) or other types of static storage devices capable of storing static information and instructions, random access memory (RAM) or other types of dynamic storage devices capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto.

[0186] Memory 902 may include high-speed random access memory and may also include non-volatile memory, such as one or more disk storage devices, flash memory devices, or other non-volatile solid-state storage devices. Memory 902 may store an operating system (hereinafter referred to as the system), such as uCOS, VxWorks, RTLinux, or other embedded operating systems. Memory 902 may also store network communication programs that can be used to communicate with one or more additional devices, one or more user devices, or one or more terminals. Memory 902 may exist independently and be connected to processor 901 via bus 904. Memory 902 may also be integrated with processor 901.

[0187] Memory 902 is configured to store application code that executes the above scheme, and its execution is controlled by processor 901. Processor 901 is configured to execute the application code stored in memory 902.

[0188] Processor 901 may be a central processing unit, a general-purpose processor, a digital signal processor, an application-specific integrated circuit, a field-programmable gate array, or other programmable logic device, transistor logic device, hardware component, or any combination thereof. It may implement or execute the various exemplary logic blocks, modules, and circuits described in connection with this disclosure. Processor 901 may also be a combination that implements a specific function, such as a combination of one or more microprocessors, a combination of a digital signal processor and a microprocessor, etc.

[0189] Some embodiments of this disclosure also provide a vehicle 20, which includes the electronic device 90 shown in FIG9.

[0190] Some embodiments of this disclosure also provide a computer-readable storage medium storing instructions that, when executed on at least one processor, implement the aforementioned reversing display method, such as the method in FIG3.

[0191] Some embodiments of this disclosure also provide a computer program product that includes computer instructions that, when executed by an electronic device, implement the aforementioned reversing display method, such as the method in FIG3.

[0192] In some embodiments of this disclosure, the terms "for example" or "for instance" are used to indicate that something is an example, illustration, or description. Any embodiment or design described as "for example" or "for instance" in some embodiments of this disclosure should not be construed as being more preferred or advantageous than other embodiments or designs. Rather, the use of terms such as "for example" or "for instance" is intended to present the relevant concepts in a specific manner.

[0193] In some embodiments of this disclosure, "at least one" refers to one or more items, and "more than one" refers to two or more items. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or multiple items. For example, at least one of a, b, or c can represent: a, b, c, (a and b), (a and c), (b and c), or (a and b and c), where a, b, and c can be single or multiple. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0194] Furthermore, unless otherwise stated, in some embodiments of this disclosure, the use of ordinal numbers such as "first" and "second" is for distinguishing multiple objects and is not for limiting the order, sequence, priority, or importance of the multiple objects. For example, "first device" and "second device" are used only for ease of description and do not indicate differences in the structure, importance, etc. of the first device and the second device. In some embodiments, the first device and the second device may also be the same device.

[0195] In the above embodiments, the term "when..." can be interpreted, depending on the context, as meaning "if...", "after...", "in response to determining...", or "in response to detecting...". The above are merely optional embodiments of this disclosure and are not intended to limit this disclosure. Any modifications, equivalent substitutions, improvements, etc., made within the concept and principles of this disclosure should be included within the scope of protection of this disclosure.

[0196] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0197] The above are merely specific embodiments of this disclosure, but the scope of protection of this disclosure is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this disclosure, and these modifications or substitutions should all be covered within the scope of protection of this disclosure. Therefore, the scope of protection of this disclosure should be determined by the scope of the claims.

Claims

1. A method for displaying a reverse parking indicator, comprising: Acquire multiple first images, wherein the multiple first images include environmental images captured by the vehicle from multiple directions, the multiple directions including at least the front and rear; A second image is obtained based on the plurality of first images, wherein the second image includes a front area image and a rear area image of the vehicle, and the area of ​​the rear area image in the second image is larger than the area of ​​the front area image in the second image; The second image is displayed in the reversing camera view of the vehicle.

2. The method according to claim 1, wherein, The origin of the camera coordinate system corresponding to the second image is located in the region between the geometric center of the vehicle and the rear of the vehicle.

3. The method according to claim 2, wherein, The step of obtaining the second image based on the plurality of first images includes: The multiple first images are transformed from being represented in pixel coordinates to being represented in world coordinates; The multiple first images, represented in the world coordinate system, are converted to be represented in the camera coordinate system. The multiple first images, represented in the camera coordinate system, are converted to be represented in the image coordinate system to obtain the second image.

4. The method according to claim 3, wherein, The step of converting the multiple first images represented in the world coordinate system to those represented in the camera coordinate system includes: Determine the origin of the camera coordinate system; The transformation matrix is ​​determined based on the origin of the camera coordinate system; Based on the transformation matrix, the multiple first images represented in the world coordinate system are transformed to be represented in the camera coordinate system.

5. The method according to any one of claims 1-4, wherein, The second image includes a trajectory line configured to indicate the direction of rotation of the vehicle, the direction of rotation indicated by the trajectory line being consistent with the direction of rotation of the steering wheel angle.

6. The method according to claim 5, wherein, The trajectory line is determined based on the vehicle's direction of travel and the steering wheel angle, and the trajectory line is also configured to indicate the vehicle's rotation angle.

7. The method according to claim 5 or 6, wherein, Displaying the second image in the reversing camera of the vehicle includes: The second image, which includes the trajectory line, is displayed in the reversing image of the vehicle after being rotated and mirrored.

8. The method according to any one of claims 1 to 7, wherein, The step of obtaining the second image based on the plurality of first images includes: In the scenario of the vehicle reversing, the multiple first images are stitched together to obtain the second image.

9. A reversing display device, comprising: A communication unit is configured to acquire multiple first images, wherein the multiple first images include environmental images captured by the vehicle from multiple directions, the multiple directions including at least the front and rear; and The processing unit is configured to obtain a second image based on the plurality of first images, wherein the second image includes a front area image and a rear area image of the vehicle, and the area of ​​the rear area image in the second image is larger than the area of ​​the front area image in the second image. The communication unit is also configured to display the second image in the reversing camera image of the vehicle.

10. An electronic device comprising a processor coupled to a memory configured to store a computer program, the processor configured to invoke and run the computer program such that the electronic device performs the method according to any one of claims 1-8.

11. A vehicle comprising the electronic device according to claim 10.

12. A computer-readable storage medium, wherein, The computer-readable storage medium stores a computer program, the computer program including instructions configured to perform the method according to any one of claims 1-8.

Images