A content display method and apparatus, an electronic device, and a storage medium

Abstract

The present disclosure provides a content display method and device, electronic equipment and storage medium, relates to the technical field of computers, and particularly relates to the technical field of automatic driving, autonomous parking and intelligent traffic. The specific implementation scheme is as follows: a starting pointing position of a virtual camera is determined, the starting pointing position has a corresponding relationship with a starting shooting area of the virtual camera; the virtual camera is rotated multiple times from the starting pointing position according to a preset angle and a preset direction, and an image shot by the virtual camera from a preset three-dimensional model after each rotation is obtained; a first image is generated based on the image shot by the virtual camera from the preset three-dimensional model after each rotation, the first image includes road condition information around a target vehicle, and the road condition information around the target vehicle includes road condition information of a region corresponding to the real camera; and the first image is displayed.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of computer technology, and particularly relates to the technical field of automatic driving, autonomous parking and intelligent transportation. Specifically, a content display method and device, an electronic device and a storage medium are provided. BACKGROUND

[0002] At present, a vehicle can collect images around the vehicle through a vehicle-mounted camera, and present the images to a driver in a bird's eye view manner to assist the driver in driving. SUMMARY

[0003] The present disclosure provides a content display method and device, an electronic device and a storage medium with higher safety.

[0004] According to an aspect of the present disclosure, a content display method is provided. The method includes: determining a starting pointing position of a virtual camera, the starting pointing position having a corresponding relationship with a starting shooting area of the virtual camera; starting from the starting pointing position, rotating the virtual camera multiple times according to a preset angle and a preset direction, and obtaining images shot by the virtual camera from a preset three-dimensional model after each rotation, the preset three-dimensional model being used to represent a real scene in which a real camera is located, the real camera being one or more cameras included in a target vehicle; generating a first image based on the images shot by the virtual camera from the preset three-dimensional model after each rotation, the first image including road condition information around the target vehicle, the road condition information around the target vehicle including road condition information of an area corresponding to the real camera; and displaying the first image.

[0005] According to another aspect of the present disclosure, a content display device is provided. The device includes: a determining module, a processing module, an obtaining module and a displaying module; the determining module is configured to determine a starting pointing position of a virtual camera, the starting pointing position having a corresponding relationship with a starting shooting area of the virtual camera; the processing module is configured to start from the starting pointing position, rotate the virtual camera multiple times according to a preset angle and a preset direction; the obtaining module is configured to obtain images shot by the virtual camera from a preset three-dimensional model after each rotation, the preset three-dimensional model being used to represent a real scene in which a real camera is located, the real camera being one or more cameras included in a target vehicle; the processing module is further configured to generate a first image based on the images shot by the virtual camera from the preset three-dimensional model after each rotation, the first image including road condition information around the target vehicle, the road condition information around the target vehicle including road condition information of an area corresponding to the real camera; and the displaying module is configured to display the first image.

[0006] According to another aspect of the present disclosure, an electronic device is provided. The electronic device includes at least one processor, and a memory connected to the at least one processor in communication; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the content display method provided by the present disclosure.

[0007] According to another aspect of the present disclosure, a non-transitory computer readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to perform the content display method provided by the present disclosure.

[0008] According to another aspect of the present disclosure, a computer program product is provided, including a computer program which, when executed by a processor, implements the content display method provided by the present disclosure.

[0009] According to another aspect of the present disclosure, an autonomous vehicle is provided, including the electronic device provided by the present disclosure.

[0010] Based on any of the above aspects, in the present disclosure, the electronic device can generate a first image corresponding to a specific shooting area (i.e. the area composed between the starting shooting area and the shooting area after rotating the virtual camera) with a specific field of view range (i.e. the horizontal field of view angle of the first image and the preset vertical field of view angle) based on the image taken by the virtual camera from the preset three-dimensional model after each rotation. In addition, since the road condition information on the side of the target vehicle driving position is included in the first image, i.e. the electronic device can display the road condition information around the target vehicle, which can assist the driver to drive the target vehicle, without the need for the driver to manually (or turn his head) to observe, and can improve the safety during driving.

[0011] It should be understood that the content described in this part is not intended to identify key or important features of the embodiments of the present disclosure, nor to limit the scope of the present disclosure. Other features of the present disclosure will become apparent from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0012] The accompanying drawings are used to better understand the present scheme, and do not constitute a limitation on the present disclosure. Among them:

[0013] Figure 1 An application scenario schematic diagram provided by an embodiment of the present disclosure is shown;

[0014] Figure 2 A flowchart schematic diagram of a content display method provided by an embodiment of the present disclosure is shown;

[0015] Figure 3 An imaging principle schematic diagram of an omnidirectional model provided by an embodiment of the present disclosure is shown;

[0016] Figure 4 FIG. 6 shows a flow diagram of another content display method according to an embodiment of the present disclosure;

[0017] Figure 5 FIG. 7 shows a schematic diagram of a unit sphere grid model according to an embodiment of the present disclosure;

[0018] Figure 6 FIG. 8 shows a schematic diagram of a preset three-dimensional model according to an embodiment of the present disclosure;

[0019] Figure 7 FIG. 9 shows a schematic diagram of an image generated and displayed by an electronic device according to an embodiment of the present disclosure;

[0020] Figure 8 FIG. 10 shows a schematic diagram of a display interface according to an embodiment of the present disclosure;

[0021] Figure 9 FIG. 11 shows a schematic diagram of a second image according to an embodiment of the present disclosure;

[0022] Figure 10 FIG. 12 shows a schematic diagram of a content display device according to an embodiment of the present disclosure;

[0023] Figure 11 FIG. 13 shows a schematic diagram of an electronic device according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0024] Exemplary embodiments of the present disclosure are described herein with reference to the accompanying drawings, in which various details are outlined in order to facilitate an understanding. It will be appreciated that various embodiments of the present disclosure can be practiced without such specific details. In other instances, well-known methods, structures and techniques have not been described in detail in order to avoid obscuring the present disclosure. The following detailed description is not intended to limit the present disclosure, as claimed.

[0025] It should be noted that the terms "first", "second", and the like in the description and in the claims of the present disclosure and the above-described drawings are used to distinguish similar objects and are not necessarily used to describe a specific sequence or order. It should be understood that the terms used in this way can be interchanged under appropriate circumstances, so that the embodiments of the present disclosure described herein can be implemented in an order other than that illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Rather, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0026] It should also be understood that the term "comprising" indicates the presence of the described feature, whole, step, operation, element and / or component, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements and / or components.

[0027] The collection, storage, use, processing, transmission, provision, and disclosure of user personal information involved in the technical solution disclosed herein comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0028] In related technologies, vehicles can use onboard cameras to capture images of their surroundings and present them to the driver in a bird's-eye view to assist driving. However, this method can only display the relative position or distance between the vehicle and the vehicle in front (or behind), and cannot display other information about the vehicle's surroundings. This other information needs to be observed manually by the driver, increasing the complexity of the driving process and posing safety risks.

[0029] Based on this, embodiments of this disclosure provide a content display method in which an electronic device generates a first image corresponding to a specific shooting area (i.e., the area from the initial shooting area to the shooting area after rotating the virtual camera) and having a specific field of view (i.e., the horizontal field of view and the preset vertical field of view of the first image) based on images captured by a virtual camera from a preset 3D model after each rotation. Furthermore, since the first image includes road condition information around the target vehicle, the electronic device can display this information, assisting the driver in driving the target vehicle without requiring manual observation (or turning the head), thus improving driving safety.

[0030] The technical solutions provided in this disclosure can be applied to autonomous parking scenarios. Specifically, in autonomous parking scenarios, electronic devices, based on the content display method provided in this disclosure, can display road condition information around the target vehicle, assisting drivers in autonomous parking, ensuring driving safety during the autonomous parking process, and improving the driver's driving experience.

[0031] For example, such as Figure 1 The image shows an example of the aforementioned autonomous parking scenario. Specifically, when the parking space is located on the right side of the road into which the vehicle is about to enter, the electronic device can display road condition information on the left side of the vehicle based on the content display method provided in this embodiment, to assist the driver in driving the vehicle out of the parking space and into the corresponding road. Of course, the electronic device can also display road condition information on the right front of the vehicle based on the content display method provided in this embodiment.

[0032] It should be noted that, Figure 1The autonomous parking scenario shown is only one example of application of the embodiments of the present disclosure. In actual use, the autonomous parking scenario can also include various parking scenarios such as reversing into a parking space, side parking, and various scenarios of driving out of a parking space. In addition, the method of the embodiments of the present disclosure can also be applied to scenarios that require assistance to the driver during driving (not limited to autonomous driving or manual driving).

[0033] For example, the electronic device that executes the technical solutions provided by the embodiments of the present disclosure can be a tablet computer, a desktop computer, a laptop computer, a handheld computer, a notebook computer, an ultra-mobile personal computer (UMPC), a netbook, a cellular phone, a personal digital assistant (PDA), an augmented reality (AR) \ virtual reality (VR) device, a vehicle-mounted device, a smart car, and the like. The present disclosure does not specially limit the specific form of the electronic device. It can interact with the user through one or more ways such as a keyboard, a touchpad, a touch screen, a remote control, voice interaction, or a handwriting device.

[0034] Optionally, the electronic device can also be a server, which is not limited by the embodiments of the present disclosure. The server can be a standalone physical server, a server cluster or a distributed system composed of multiple physical servers, or a cloud server that provides cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery network (CDN), and basic cloud computing services such as big data and artificial intelligence platforms.

[0035] The technical solutions provided by the embodiments of the present disclosure are described in detail below with reference to the accompanying drawings.

[0036] As Figure 2 The content display method provided by the embodiments of the present disclosure can include S201-S204.

[0037] S201, the electronic device determines a starting pointing position of a virtual camera.

[0038] The starting pointing position has a corresponding relationship with a starting shooting area of the virtual camera.

[0039] It should be understood that the virtual camera is a camera included in a three-dimensional model (or a three-dimensional scene), which is used to observe and shoot the content observed in the three-dimensional model, so that the electronic device can generate a corresponding image based on the shot content.

[0040] It can be understood that the starting shooting area of the virtual camera is an area included in the three-dimensional model and capable of being shot by the virtual camera in a starting state (or initial state). The starting shooting area can be an area pre-stored in the electronic device or an area selected by the driver.

[0041] In the embodiments of the present disclosure, the starting pointing position of the virtual camera is a position pointed by the virtual camera when observing and shooting the starting shooting area.

[0042] S202, the electronic device starts from the starting pointing position, rotates the virtual camera multiple times according to the preset angle and the preset direction, and obtains images shot by the virtual camera from the preset three-dimensional model after each rotation.

[0043] The preset three-dimensional model is used to represent a real scene in which the real camera is located, and the real camera can be one or more cameras included in the target vehicle.

[0044] Optionally, the real camera can be a camera located on one side of the driving position of the target vehicle.

[0045] It should be noted that when the real camera includes multiple cameras in the target vehicle, the preset three-dimensional model can be multiple, that is, one camera corresponds to one preset three-dimensional model, which is used to represent the real scene in which the corresponding camera is located. Then in S202, the electronic device can shoot images from multiple preset three-dimensional models by rotating the virtual camera. Alternatively, when the real camera includes multiple cameras in the target vehicle, the preset three-dimensional model can be one, that is, multiple cameras correspond to one preset three-dimensional model, which is used to represent the real scene in which the multiple cameras are located. Then in S202, the electronic device can shoot images from the one preset three-dimensional model by rotating the virtual camera.

[0046] Specifically, the electronic device starts from the starting pointing position, rotates the virtual camera once according to the preset angle and the preset direction, and the virtual camera points to a new position (or shooting area). The virtual camera can perform a shooting function after pointing to a new position, and thus images shot by the virtual camera from the preset three-dimensional model after each rotation can be obtained.

[0047] In an optional implementation, the preset angle can be a preset horizontal unit field of view, such as 5° (degrees). The preset direction can be a certain horizontal direction, for example, the longitude direction.

[0048] For example, assuming that the longitude and latitude of the above-mentioned starting pointing position is (0°, 0°), the preset horizontal unit field of view is 5°, and the preset direction is the longitude direction. Then, the electronic device determines that the longitude and latitude of the position pointed to by the virtual camera after the first rotation is (5°, 0°).

[0049] In another optional implementation, the above-mentioned preset angle can also be a preset vertical unit field of view. For example, the preset vertical unit field of view can be 10°, and the preset direction can be a certain vertical direction, for example, the latitude direction.

[0050] In yet another optional implementation, the above-mentioned preset angle can include a preset horizontal unit field of view and a preset vertical unit field of view. Correspondingly, the preset method includes a certain horizontal direction corresponding to the preset horizontal unit field of view, and a certain vertical direction corresponding to the preset vertical unit field of view. It can be understood that in this implementation, the virtual camera will rotate in two directions each time to capture an image of a new shooting area pointed to.

[0051] In the embodiments of the present disclosure, the electronic device rotates the virtual camera according to the preset angle and the preset direction multiple times, and obtains the images captured by the virtual camera from the preset three-dimensional model after each rotation. Clear images with a wide-angle view can be obtained, and the image quality is improved.

[0052] In one implementation of the embodiments of the present disclosure, the target vehicle can be deployed with a camera (or camera head) capable of providing a clear wide-angle view. For example, the target vehicle can be deployed with 4 or more cameras. Based on this, the electronic device can assist the driver to perform driving operations based on the 4 or more cameras. Specifically, at least one camera can be arranged in each of the front, rear, left and right directions of the target vehicle. For example, a camera can be arranged on the left door handle (or left side mirror) of the target vehicle, a camera can be arranged on the right door handle (or right side mirror) of the target vehicle, a camera can be arranged directly in front of the vehicle head of the target vehicle, and a camera can be arranged directly behind the vehicle head of the target vehicle.

[0053] The above-mentioned real camera can be one or more of the 4 or more cameras included in the target vehicle. Specifically, which cameras in the target vehicle are included in the real camera can be set according to the actual scene requirements. For example, Figure 1 As shown in the autonomous parking scene, the real camera can include a camera deployed on the left side of the target vehicle (such as the left side mirror, or the side of the driver's seat), and can also include a camera deployed on the right side of the target vehicle (such as the right side mirror, or the other side of the driver's seat). The number and position of the cameras included in the real camera are not specifically limited in the embodiments of the present disclosure.

[0054] In the embodiments of the present disclosure, the camera arranged on the target vehicle adopts an omnidirectional model lens. The omnidirectional model is a model used by a camera that images by using a plane mirror to reflect, and the main difference from the pinhole model is that light is first imaged on a unit sphere instead of a normalized plane, which can have a wider field of view angle.

[0055] Optionally, the omnidirectional model lens can be an MEI model fisheye lens, and the MEI model is one of the omnidirectional models.

[0056] For example, as shown in FIG. 1, light can be first imaged on a unit sphere, and then imaged on a two-dimensional plane. Figure 3

[0057] S203, the electronic device generates a first image based on the image captured by the virtual camera from the preset three-dimensional model after each rotation.

[0058] The first image includes the road condition information around the target vehicle, and the road condition information around the target vehicle includes the road condition information of the area corresponding to the real camera. For example, for a scenario in which the driving position of the target vehicle is located on the left side of the target vehicle (i.e., left-hand driving), and the real camera includes a camera located on one side (e.g., the left side) of the driving position of the target vehicle, the first image includes the road condition information on one side (e.g., the left side) of the driving position of the target vehicle. For another example, the real camera also includes a camera located on the other side (e.g., the right side) of the driving position of the target vehicle, and the first image also includes the road condition information on the other side (e.g., the right side) of the driving position of the target vehicle.

[0059] In an optional implementation, when the preset angle is a preset horizontal unit field of view angle and the preset direction is a horizontal direction, the horizontal field of view angle of the image captured by the virtual camera from the preset three-dimensional model after each rotation is the preset horizontal unit field of view angle, and the horizontal field of view angle of the first image has a corresponding relationship with the number of rotations of the virtual camera. At this time, the vertical field of view angle of the image captured by the virtual camera from the preset three-dimensional model after each rotation is the same as the vertical field of view angle of the first image.

[0060] It is worth noting that the horizontal field of view angle of an image is the field of view range of the image in the horizontal direction, and the vertical field of view angle of the image is the field of view range of the image in the vertical direction.

[0061] Optionally, the electronic device can determine the product of the preset angle (or the preset horizontal unit field of view angle) and the number of rotations of the virtual camera as the horizontal field of view angle of the first image.

[0062] ​For example, assuming that the horizontal field of view of the image taken by the virtual camera from the preset three-dimensional model after each rotation is 5°, and the number of rotations of the virtual camera is 24, the horizontal field of view of the first image generated by the electronic device is 120°.

[0063] In another optional implementation, when the preset angle is a preset vertical unit field of view and the preset direction is a vertical direction, the vertical field of view of the image taken by the virtual camera from the preset three-dimensional model after each rotation is the preset vertical unit field of view, and the vertical field of view of the first image has a corresponding relationship with the number of rotations of the virtual camera. At this time, the horizontal field of view of the image taken by the virtual camera from the preset three-dimensional model after each rotation is the same as the horizontal field of view of the first image.

[0064] In another optional implementation, when the preset angle includes a preset horizontal unit field of view and a preset vertical unit field of view, and the preset direction includes a horizontal direction and a vertical direction, the horizontal field of view of the image taken by the virtual camera from the preset three-dimensional model after each rotation is the preset horizontal unit field of view, the vertical field of view is the preset vertical unit field of view, the vertical field of view of the first image has a corresponding relationship with the number of rotations of the virtual camera, and the horizontal field of view of the first image also has a corresponding relationship with the number of rotations of the virtual camera.

[0065] In an optional implementation, the road condition information on the side of the target vehicle driving position includes roads on the side of the target vehicle driving position, parking spaces on the side of the target vehicle driving position, other vehicles on the side of the target vehicle driving position, pedestrians, and obstacles, and the like. The contents (or information) included in the road condition information on the right side of the target vehicle, the road condition information in front of the target vehicle, and the road condition information behind the target vehicle are the same, and are not described herein again.

[0066] S204, the electronic device displays the first image.

[0067] It should be understood that the electronic device displays the first image, which is to display the road condition information around the target vehicle (including the area corresponding to the real camera), can assist the driver to drive the target vehicle, without the need for the driver to manually (or turn his head) to observe, and can improve the safety during driving.

[0068] The technical solutions provided by the above embodiments can bring at least the following beneficial effects: as can be seen from S201-S204, the electronic device can first determine the starting pointing position of the virtual camera; then the electronic device starts from the starting pointing position, rotates the virtual camera by a preset angle and in a preset direction, and obtains images captured by the virtual camera after each rotation from the preset three-dimensional model (i.e., the scene in which the real camera is located); then, the electronic device can generate a first image based on the images captured by the virtual camera after each rotation from the preset three-dimensional model, and display the first image. In the embodiments of the present disclosure, the electronic device can generate a first image corresponding to a specific shooting area (i.e., the area composed between the starting shooting area and the shooting area after rotating the virtual camera) and having a specific field of view (i.e., the horizontal field of view angle and the preset vertical field of view angle of the first image) based on the images captured by the virtual camera after each rotation from the preset three-dimensional model. In addition, since the road condition information around the target vehicle is included in the first image, i.e., the electronic device can display the road condition information around the target vehicle, the driving personnel can be assisted in driving the target vehicle, without the need for the driving personnel to manually (or turn their heads) to observe, and the safety during driving can be improved.

[0069] In the embodiments of the present disclosure, before obtaining the images captured by the virtual camera after each rotation from the preset three-dimensional model, the electronic device needs to first generate the preset three-dimensional model, specifically, to construct the real scene in which the real camera is located.

[0070] Based on this, as shown in the following table, Figure 4 The content display method provided by the embodiments of the present disclosure includes S401-S408.

[0071] S401, the electronic device obtains a plurality of images collected by a real camera.

[0072] Among them, the plurality of images are images collected by the real camera at each time in a plurality of time periods included in the current time period.

[0073] In combination with the description of the above embodiments, it should be understood that the real camera is one or more cameras included in the target vehicle. For example, the real camera can be a camera located on one side of the driving position of the target vehicle.

[0074] S402, the electronic device projects the plurality of images to a preset grid model to obtain a preset three-dimensional model.

[0075] It should be understood that the electronic device can establish the preset grid model based on the camera coordinate system of the real camera, with the position (or coordinates) of the real camera as the center, the position of the real camera being the position of the real camera in the vehicle coordinate system.

[0076] It can be understood that the electronic device can project a real camera (or a camera in the real camera) at a large angle by optimizing the omni-directional projection method.

[0077] Optionally, the preset grid model can be a unit sphere grid model.

[0078] It can be understood that after obtaining the preset three-dimensional model, the electronic device can set the virtual camera at a point (for example, a center point of the preset three-dimensional model) in the preset three-dimensional model, and then capture the environment or object in the preset three-dimensional model based on the virtual camera.

[0079] For example, the electronic device can project a plurality of images to a unit sphere grid model as shown in FIG. 2, and then obtain a preset three-dimensional model. Figure 5

[0080] In the embodiments of the present disclosure, the electronic device can obtain a plurality of images collected by a real camera, and project the plurality of images to a preset grid model, so as to conveniently and quickly obtain a preset three-dimensional model which can accurately represent a real scene where the real camera is located. Then, a high-accuracy image can be generated based on the preset three-dimensional model.

[0081] S403, the electronic device determines a starting pointing position of the virtual camera.

[0082] The starting pointing position has a corresponding relationship with a starting capturing area of the virtual camera.

[0083] S404, the electronic device rotates the virtual camera multiple times from the starting pointing position according to a preset angle and a preset direction, and obtains images captured by the virtual camera from the preset three-dimensional model after each rotation.

[0084] The preset three-dimensional model is used to represent a real scene where the real camera is located.

[0085] For example, it is assumed that the preset angle is 2.5° and the preset vertical field of view is 80°. The camera starts from the preset three-dimensional model towards the longitude and latitude position (0°, 0°), and after rotating the virtual camera 60 times according to the preset angle and the preset direction (for example, the longitude direction), the total horizontal field of view of the imaging is 150°. In addition, the total vertical field of view of the imaging (that is, the preset vertical field of view) can be preset as 80°.

[0086] It should be noted that the explanation and description in S403-S404 above are the same as or similar to those in S201-S202 above, and will not be repeated here.

[0087] ​In an implementation form of the embodiment of the disclosure, the electronic device obtains images captured by the virtual camera from the preset three-dimensional model after each rotation, which can specifically include steps A-B.

[0088] Step A, the electronic device determines a camera matrix of the virtual camera and a projection matrix of the virtual camera.

[0089] The camera matrix is used to represent the position of the virtual camera in the preset three-dimensional model and the camera coordinate system of the virtual camera, and the projection matrix is used to represent the preset angle and the preset vertical field of view angle of the image captured by the virtual camera from the preset three-dimensional model.

[0090] In combination with the description of the above embodiment, it should be understood that the preset angle is the horizontal field of view angle of the image captured by the virtual camera from the preset three-dimensional model.

[0091] In an optional implementation form, the electronic device determines the camera matrix of the virtual camera and the projection matrix of the virtual camera, which can specifically include steps A1-A2.

[0092] Step A1, the electronic device obtains a three-dimensional coordinate of the virtual camera, a direction angle of the virtual camera, and a camera extrinsic parameter of the virtual camera.

[0093] The direction angle of the virtual camera includes the pitch angle of the virtual camera, the yaw angle of the virtual camera, and the roll angle of the virtual camera. The camera extrinsic parameter of the virtual camera includes the focal length of the virtual camera and the optical center of the virtual camera.

[0094] Step A2, the electronic device determines the camera matrix of the virtual camera based on the three-dimensional coordinate of the virtual camera and the direction angle of the virtual camera, and determines the projection matrix of the virtual camera based on the camera extrinsic parameter of the virtual camera.

[0095] It should be understood that the three-dimensional coordinate of the virtual camera is used to represent the position of the virtual camera in the preset three-dimensional model, and the direction angle of the virtual camera is used to represent the orientation of the virtual camera. The position of the virtual camera in the preset three-dimensional model and the orientation of the virtual camera constitute the pose of the virtual camera. The electronic device can determine the camera matrix of the virtual camera based on the pose of the virtual camera.

[0096] Optionally, the electronic device can input the three-dimensional coordinate of the virtual camera and the direction angle of the virtual camera into a computer graphics tool (such as opengl) to obtain the camera matrix of the virtual camera, and the electronic device can also input the camera extrinsic parameter of the virtual camera into the computer graphics tool to obtain the projection matrix of the virtual camera.

[0097] In the embodiments of the present disclosure, the electronic device can conveniently and quickly determine the camera matrix of the virtual camera based on the three-dimensional coordinates of the virtual camera and the direction angle of the virtual camera, and conveniently and quickly determine the projection matrix of the virtual camera based on the camera extrinsic parameters of the virtual camera. Then, the electronic device can conveniently and quickly obtain each image captured by the virtual camera from the preset three-dimensional model.

[0098] Step B, the electronic device obtains the image captured by the virtual camera from the preset three-dimensional model after each rotation based on the camera matrix and the projection matrix.

[0099] In an implementation form of the embodiments of the present disclosure, the electronic device obtains the image captured by the virtual camera from the preset three-dimensional model after each rotation based on the camera matrix and the projection matrix, and specifically can include steps B1-B3.

[0100] Step B1, the electronic device obtains the three-dimensional coordinates of each object captured by the virtual camera from the preset three-dimensional model after each rotation.

[0101] It should be understood that the content observed and captured by the virtual camera in the preset three-dimensional model is a 3D object (or a 3D scene), and the electronic device can obtain the three-dimensional coordinates of each 3D object, which can be specifically the three-dimensional coordinates of each pixel point included in each 3D object.

[0102] Step B2, the electronic device respectively left multiplies the camera matrix and the projection matrix based on the three-dimensional coordinates of each object to obtain the two-dimensional coordinates of each object on the two-dimensional plane.

[0103] Step B3, the electronic device obtains the image captured by the virtual camera from the preset three-dimensional model after each rotation based on the two-dimensional coordinates of each object on the two-dimensional plane.

[0104] It can be understood that the electronic device can map the above-mentioned 3D object (or 3D model) to the two-dimensional plane through the camera matrix and the projection matrix, that is, the image captured by the virtual camera in the preset three-dimensional model can be obtained.

[0105] In the embodiments of the present disclosure, the electronic device can accurately and effectively complete the mapping process from the three-dimensional model to the two-dimensional plane by respectively left multiplying the camera matrix and the projection matrix based on the three-dimensional coordinates of each object (specifically each pixel point included in each object), and can obtain a two-dimensional image with high accuracy.

[0106] For example, Figure 6As shown, point C is the position of the virtual camera in the preset three-dimensional model (specifically, the center point of the preset three-dimensional model). The electronic device can map the content captured by the virtual camera in the preset three-dimensional model onto a 2D plane, and obtain the image captured by the virtual camera in the preset three-dimensional model.

[0107] In the embodiments of the present disclosure, the electronic device can obtain the image captured by the virtual camera after each rotation from the preset three-dimensional model based on the camera matrix of the virtual camera and the projection matrix of the virtual camera. The mapping of the three-dimensional content in the preset three-dimensional model to the two-dimensional plane can be accurately and effectively completed, and the image captured by the virtual camera in the preset three-dimensional model with high accuracy can be obtained.

[0108] In S405, the electronic device generates a first image based on the image captured by the virtual camera after each rotation from the preset three-dimensional model.

[0109] The first image includes road condition information around the target vehicle, and the road condition information around the target vehicle includes road condition information of a region corresponding to the real camera.

[0110] In an implementation manner of the embodiments of the present disclosure, the electronic device generates the first image based on the image captured by the virtual camera after each rotation from the preset three-dimensional model, and specifically can include step C.

[0111] In step C, the electronic device performs stitching processing on each image captured by the virtual camera in the preset three-dimensional model, and performs fusion processing on the edges of each image to obtain the first image.

[0112] It should be understood that the electronic device performs stitching processing on each image, which can integrate each image together to form a complete first image. In addition, the edges of each image can have some distortion. In order to make the image more smooth, the edges of each image can be fused to ensure the authenticity of the image.

[0113] Optionally, the fusion processing can be weighted fusion.

[0114] Specifically, for two adjacent images (e.g., the first image and the second image), the electronic device can determine an overlapping area between the first image and the second image, and determine a splicing area between the first image and the second image. Then, the electronic device can determine the RGB component value of the first image, the RBG component value of the second image, the width of the splicing area, and the distance between a pixel point included in the overlapping area and the right boundary of the splicing area. Finally, the electronic device can determine the RGB component value of the overlapping area according to the RGB component value of the first image, the RBG component value of the second image, the width of the splicing area, and the distance between the pixel point included in the overlapping area and the right boundary of the splicing area, that is, an image processed through weighted fusion can be obtained.

[0115] As shown in the example, Figure 7 suppose Figure 7 The upper half (i.e., image 1) is an example of an image obtained by the content display method provided in the embodiments of the present disclosure, and Figure 7 The lower half (i.e., image 2) is an example of an image obtained by the related art.

[0116] Suppose the resolution of the original image is 1280*720, the resolution of image 1 is 1420*720, the horizontal field of view of image 1 is 165°, the resolution of image 2 is 960*720, and the horizontal field of view of image 2 is 130°.

[0117] As can be seen from Figure 7 Compared with the related art, the content display method provided in the embodiments of the present disclosure can obtain an image with a larger horizontal field of view. In addition, compared with the original image, image 1 has a higher resolution even though the horizontal field of view is increased. Compared with the original image, image 2 has a smaller horizontal field of view even though part of the area is still blurred. That is, in the content display method provided in the embodiments of the present disclosure, even if the resolution and the field of view of the image are increased, the content in the image can still be clearly displayed.

[0118] S406, the electronic device displays the first image.

[0119] In an implementation manner of the embodiments of the present disclosure, the electronic device can further generate and display a third image based on the content display method provided in the above embodiments, and the third image corresponds to a different starting pointing position and / or rotation direction from the first image.

[0120] Specifically, the electronic device can start from another position (the position corresponds to an area which can be the left rear part of the target vehicle, and the above-mentioned starting shooting area can be the left front part of the target vehicle), rotate the virtual camera multiple times at a certain angle (which can be the same as the above-mentioned preset angle, or can be different) and in a certain direction (which can be the same as the above-mentioned preset direction, or can be different), and obtain the images shot from the preset three-dimensional model after each rotation of the virtual camera. Then, the electronic device can generate the third image based on the images shot from the preset three-dimensional model after each rotation of the virtual camera.

[0121] In an optional implementation, the horizontal unit field of view angle of the images shot from the preset three-dimensional model after each rotation of the virtual camera based on which the electronic device generates the third image can be 2.5°, the horizontal field of view angle (or horizontal total field of view) of the third image can be 120°, and the vertical field of view angle of the third image can be 50°. In order to consider the feelings of the driver, the same content as the above-mentioned first image is avoided to be displayed in the third image.

[0122] In the embodiments of the present disclosure, the third image can represent the content that can be observed by the driver through the rearview mirror on a certain side (for example, the left side) of the target vehicle. By displaying the third image, the electronic device can avoid the driving risk caused by manually turning the head to observe the rearview mirror, and improve the safety of vehicle driving.

[0123] In an optional implementation, the electronic device (specifically, the display screen) can include a rearview mirror function on-off button and a rearview mirror function adjustment button. The rearview mirror function on-off button is used to determine whether to start the rearview mirror function, and the electronic device can display the above-mentioned third image when the rearview mirror function is started. The rearview mirror function adjustment button is used to adjust the starting shooting area and the rotation direction of the above-mentioned third image, and is used to simulate the process of adjusting the rearview mirror by the driver.

[0124] Optionally, the electronic device can configure the above-mentioned rearview mirror function adjustment button in four directions (up, down, left and right) of the rearview mirror of the target vehicle.

[0125] For example, as shown in FIG. 8, Figure 8 As shown in FIG. 8, it is assumed that the driver needs to drive the target vehicle out of a certain parking space and drive into a certain road. At this time, the electronic device can display the above-mentioned first image in the display area 801, that is, the road condition information around the target vehicle (including the road condition information on the side of the driving position of the target vehicle), because the driving position is located on the left side of the target vehicle, the Figure 8 The display area 801 for observing the left super-clear large horizontal field of view angle is contained in the

[0126] Also, the electronic device can display the third image in a display area 802, which can be understood as a rearview mirror area.

[0127] In addition, the electronic device can display a region (or button, i.e., button 803) for turning on and off the rearview mirror function in the display area 802, which can be understood as a rearview mirror function on / off button. Figure 8 Figure 8 In addition, the electronic device can display a region (or button, i.e., button 804) for adjusting the visual angle of the display area 802 by the periphery of the display area 802, which can be understood as a rearview mirror function adjustment button.

[0128] S407, in a case where the turn signal of the target vehicle is in an on state and / or the current driving speed of the target vehicle is less than a speed threshold, the electronic device determines the current gear of the target vehicle.

[0129] It should be understood that, in a case where the turn signal of the target vehicle is in an on state and / or the current driving speed of the target vehicle is less than a speed threshold, it indicates that the target vehicle is about to be (or is) in a parking state, at which time the electronic device can determine the current gear of the target vehicle.

[0130] S408, in a case where the current gear of the target vehicle is a forward gear, the electronic device displays a second image.

[0131] The second image includes road condition information in front of the target vehicle.

[0132] It can be understood that, in a case where the current gear of the target vehicle is a forward gear, it indicates that the target vehicle is about to (or is) driving forward, at which time the electronic device can display a second image containing road condition information in front of the target vehicle.

[0133] Optionally, the road condition information in front of the target vehicle can include a road in front of the target vehicle, an object (such as a vehicle, an obstacle, a pedestrian) in front of the target vehicle, a relative position between the target vehicle and the object, and a distance between the target vehicle and the object.

[0134] In the embodiments of the present disclosure, in a case where the turn signal of the target vehicle is in an on state and / or the current driving speed of the target vehicle is less than a speed threshold, it indicates that the target vehicle is about to be (or is) in a parking state, at which time the electronic device can determine the current gear of the target vehicle. In a case where the current gear of the target vehicle is a forward gear, it indicates that the target vehicle is about to (or is) driving forward, at which time the electronic device can display a second image, i.e., display road condition information in front of the target vehicle, which can assist the driver to complete the parking process well and reduce the complexity of the driving process.

[0135] ​For example, suppose the target vehicle has already parked in its previous parking space and is about to enter the lane. The electronic device can determine whether the target vehicle's turn signal is on, whether its current speed is below a speed threshold, and its current gear. If the target vehicle's turn signal is on, its current speed is below the speed threshold, and its current gear is drive, the electronic device can display something like this: Figure 9 The second image shown includes the road in front of the target vehicle.

[0136] In one alternative implementation, if the target vehicle is in reverse gear, it indicates that the target vehicle is about to (or is) moving backward. At this time, the electronic device can display a fourth image containing road condition information behind the target vehicle.

[0137] It is understood that, in actual implementation, the electronic device described in the embodiments of this disclosure may include one or more hardware structures and / or software modules for implementing the aforementioned corresponding content display method, and these hardware structures and / or software modules may constitute an electronic device. Those skilled in the art should readily recognize that, in conjunction with the algorithm steps of the various examples described in connection with the embodiments disclosed herein, this disclosure can be implemented in hardware or a combination of hardware and computer software. Whether a function is implemented in hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this disclosure.

[0138] Based on this understanding, the present disclosure also provides a content display device. Figure 10 A schematic diagram of the structure of a content display device provided in an embodiment of this disclosure is shown. For example... Figure 10 As shown, the content display device 100 may include: a determination module 1001, a processing module 1002, an acquisition module 1003, and a display module 1004.

[0139] The determining module 1001 is used to determine the starting pointing position of the virtual camera, which corresponds to the starting shooting area of ​​the virtual camera.

[0140] The processing module 1002 is used to rotate the virtual camera multiple times from the starting pointing position according to a preset angle and a preset direction.

[0141] The acquisition module 1003 is configured to acquire an image captured by the virtual camera after each rotation from a preset three-dimensional model, and the preset three-dimensional model is used to represent a real scene in which the real camera is located, and the real camera is one or more cameras included in the target vehicle.

[0142] The processing module 1002 is further configured to generate a first image based on the image captured by the virtual camera after each rotation from the preset three-dimensional model, and the first image includes road condition information around the target vehicle, and the road condition information around the target vehicle includes road condition information of a region corresponding to the real camera.

[0143] The display module 1004 is configured to display the first image.

[0144] Optionally, the determination module 1001 is further configured to determine a camera matrix of the virtual camera and a projection matrix of the virtual camera, the camera matrix is used to represent a position of the virtual camera in the preset three-dimensional model and a camera coordinate system of the virtual camera, and the projection matrix is used to represent the preset angle and a preset vertical field of view angle, and the preset vertical field of view angle is a vertical field of view angle of the image captured by the virtual camera from the preset three-dimensional model.

[0145] The processing module 1002 is further configured to obtain the image captured by the virtual camera after each rotation from the preset three-dimensional model based on the camera matrix and the projection matrix.

[0146] Optionally, the acquisition module 1003 is further configured to acquire a three-dimensional coordinate of the virtual camera, a direction angle of the virtual camera, and a camera extrinsic parameter of the virtual camera, the direction angle of the virtual camera includes a pitch angle, a yaw angle and a roll angle of the virtual camera, and the camera extrinsic parameter of the virtual camera includes a focal length and an optical center of the virtual camera.

[0147] The determination module 1001 is specifically configured to determine the camera matrix of the virtual camera based on the three-dimensional coordinate of the virtual camera and the direction angle of the virtual camera, and determine the projection matrix of the virtual camera based on the camera extrinsic parameter of the virtual camera.

[0148] Optionally, the acquisition module 1003 is further configured to acquire a three-dimensional coordinate of each object captured by the virtual camera after each rotation from the preset three-dimensional model.

[0149] The processing module 1002 is specifically configured to respectively left-multiply the camera matrix and the projection matrix by the three-dimensional coordinate of each object to obtain a two-dimensional coordinate of each object on a two-dimensional plane.

[0150] The processing module 1002 is further specifically configured to obtain an image captured by the virtual camera from the preset three-dimensional model after each rotation of the virtual camera based on the two-dimensional coordinates of each object in the two-dimensional plane.

[0151] Optionally, the processing module 1002 is specifically configured to perform stitching processing on each image captured by the virtual camera in the preset three-dimensional model, and perform fusion processing on edges of the each image to obtain the first image.

[0152] Optionally, the acquisition module 1003 is further configured to acquire a plurality of images captured by the real camera, the plurality of images being images captured by the real camera at each moment in a plurality of moments included in a current time period.

[0153] The processing module 1002 is further configured to project the plurality of images to a preset grid model to obtain the preset three-dimensional model.

[0154] Optionally, the determination module 1001 is further configured to determine the current gear of the target vehicle in a case that a turn signal of the target vehicle is in an on state and / or a current driving speed of the target vehicle is less than a speed threshold.

[0155] The display module 1004 is further configured to display a second image including road condition information in front of the target vehicle in a case that the current gear of the target vehicle is a forward gear.

[0156] According to embodiments of the present disclosure, the present disclosure further provides an electronic device, a readable storage medium and a computer program product.

[0157] Figure 11 A schematic block diagram of an example electronic device 1100 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptops, desktops, tablets, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular telephones, smartphones, wearable devices, and other similar computing devices. The components shown here, their connections and relationships, and their functions, are meant to be examples only, and are not meant to limit implementations of the present disclosure described and / or claimed in this document.

[0158] As Figure 11As shown, the electronic device 1100 includes a computing unit 1101 that can perform various appropriate actions and processes in accordance with a computer program stored in a read-only memory (ROM) 1102 or a computer program loaded into a random access memory (RAM) 1103 from a storage unit 1108. Various programs and data required for the operation of the electronic device 1100 can also be stored in the RAM 1103. The computing unit 1101, the ROM 1102, and the RAM 1103 are connected to each other through a bus 1104. An input / output (I / O) interface 1105 is also connected to the bus 1104.

[0159] Various components in the electronic device 1100 are connected to the I / O interface 1105, including an input unit 1106 such as a keyboard, a mouse, and the like, an output unit 1107 such as various types of displays, a speaker, and the like, a storage unit 1108 such as a magnetic disk, an optical disk, and the like, and a communication unit 1109 such as a network card, a modem, a wireless communication transceiver, and the like. The communication unit 1109 allows the electronic device 1100 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0160] The computing unit 1101 can be various general and / or special purpose processing components having processing and computing capabilities. Some examples of the computing unit 1101 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, and the like. The computing unit 1101 performs various methods and processes described above, such as the content display method. For example, in some embodiments, the content display method can be implemented as a computer software program that is tangibly embodied in a machine-readable medium, such as the storage unit 1108. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 1100 via the ROM 1102 and / or the communication unit 1109. When the computer program is loaded into the RAM 1103 and executed by the computing unit 1101, one or more steps of the content display method described above can be performed. Alternatively, in other embodiments, the computing unit 1101 can be configured to perform the content display method by any other appropriate means, such as by means of firmware.

[0161] The various embodiments of the systems and techniques described above can be implemented in digital electronic circuitry, integrated circuitry, a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on a chip (SOC), a complex programmable logic device (CPLD), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include implementation in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.

[0162] Program code for carrying out methods of the present disclosure can be written in any combination of one or more programming languages. The program code can be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the program code, when executed by the processor or controller, produces the functions / operations specified in the flowcharts and / or the block diagrams. The program code can be executed entirely on a machine, partially on a machine, partially on a machine as a stand-alone software package, or entirely on a remote machine or server.

[0163] In the context of the present disclosure, a machine-readable medium can be a tangible medium that contains or stores a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include but is not limited to an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of the machine-readable storage medium will include one or more lines of electrical connections, portable computer disks, hard disk drives, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or Flash memory), optical fibers, portable compact disc read-only memories (CD-ROMs), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0164] To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the computer. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

[0165] The systems and techniques described here can be implemented in a computing system that includes a back end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front end component (e.g., a user computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0166] The computer system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0167] It should be understood that various forms of flow shown above can be used, with steps reordered, added, or removed. For example, steps recited in the present disclosure can be performed in parallel, in series, or in a different order, without limitation, so long as the desired results of the technology disclosed in the present disclosure are achieved.

[0168] The specific embodiments described above are not intended to limit the scope of the present disclosure. Those skilled in the art will understand that various modifications, combinations, sub-combinations, and alternatives can be made to the specific embodiments without departing from the spirit and principles of the present disclosure. Any further modifications, equivalent substitutions, improvements, and the like, either alone or in some combination or sub-combination, are intended to be included within the scope of the present disclosure.

Claims

1. A content display method, comprising: determining a starting pointing position of a virtual camera, the starting pointing position having a corresponding relationship with a starting shooting area of the virtual camera; rotating the virtual camera multiple times from the starting pointing position according to a preset angle and a preset direction; determining a camera matrix of the virtual camera and a projection matrix of the virtual camera, the camera matrix being used to represent a position of the virtual camera in a preset three-dimensional model and a camera coordinate system of the virtual camera, the projection matrix being used to represent the preset angle and a preset vertical field of view angle, the preset vertical field of view angle being a vertical field of view angle of an image shot by the virtual camera from the preset three-dimensional model; the preset three-dimensional model being used to represent a real scene in which a real camera is located, the real camera being one or more cameras included in a target vehicle; obtaining, based on the camera matrix and the projection matrix, an image shot by the virtual camera from the preset three-dimensional model after each rotation; generating, based on the image shot by the virtual camera from the preset three-dimensional model after each rotation, a first image, the first image including road condition information around the target vehicle, the road condition information around the target vehicle including road condition information of a region corresponding to the real camera; displaying the first image.

2. The method of claim 1, wherein, The determining of the camera matrix of the virtual camera and the projection matrix of the virtual camera comprises: obtaining a three-dimensional coordinate of the virtual camera, a direction angle of the virtual camera, and a camera extrinsic parameter of the virtual camera, the direction angle of the virtual camera including a pitch angle, a yaw angle, and a roll angle of the virtual camera, the camera extrinsic parameter of the virtual camera including a focal length and an optical center of the virtual camera; determining the camera matrix of the virtual camera based on the three-dimensional coordinate of the virtual camera and the direction angle of the virtual camera, and determining the projection matrix of the virtual camera based on the camera extrinsic parameter of the virtual camera.

3. The method of claim 1 or 2, wherein, The obtaining of the image shot by the virtual camera from the preset three-dimensional model after each rotation based on the camera matrix and the projection matrix comprises: obtaining a three-dimensional coordinate of each object shot by the virtual camera from the preset three-dimensional model after each rotation; multiplying the camera matrix and the projection matrix by the three-dimensional coordinate of each object respectively to obtain a two-dimensional coordinate of each object on a two-dimensional plane; obtaining the image shot by the virtual camera from the preset three-dimensional model after each rotation based on the two-dimensional coordinate of each object on the two-dimensional plane.

4. The method of claim 1 or 2, wherein, The generating of the first image based on the image shot by the virtual camera from the preset three-dimensional model after each rotation comprises: performing stitching processing on each image shot by the virtual camera in the preset three-dimensional model, and performing fusion processing on edges of each image to obtain the first image.

5. The method of claim 1 or 2, further comprising: acquire a plurality of images captured by the real camera, the plurality of images being images captured by the real camera at each of a plurality of time instants included in a current time period; project the plurality of images onto a preset grid model to obtain the preset three-dimensional model.

6. The method of claim 1 or 2, further comprising: determining a current gear of the target vehicle in a case that a turn signal of the target vehicle is in an on state and / or a current driving speed of the target vehicle is less than a speed threshold; displaying a second image in a case that the current gear of the target vehicle is a forward gear, the second image comprising road condition information in front of the target vehicle.

7. A content display device comprising: determining module, processing module, acquiring module, and display module; the determining module is configured to determine a starting pointing position of the virtual camera, the starting pointing position having a corresponding relationship with a starting shooting area of the virtual camera; the processing module is configured to rotate the virtual camera multiple times from the starting pointing position according to a preset angle and a preset direction; the determining module is configured to determine a camera matrix of the virtual camera and a projection matrix of the virtual camera, the camera matrix being used to represent a position of the virtual camera in a preset three-dimensional model and a camera coordinate system of the virtual camera, the projection matrix being used to represent the preset angle and a preset vertical field of view angle, the preset vertical field of view angle being a vertical field of view angle of an image captured by the virtual camera from the preset three-dimensional model, the preset three-dimensional model being used to represent a real scene in which a real camera is located, the real camera being one or more cameras included in a target vehicle; the processing module is further configured to obtain an image captured by the virtual camera from the preset three-dimensional model after each rotation based on the camera matrix and the projection matrix; the processing module is further configured to generate a first image based on the image captured by the virtual camera from the preset three-dimensional model after each rotation, the first image comprising road condition information around the target vehicle, the road condition information around the target vehicle including road condition information of a region corresponding to the real camera; the display module is configured to display the first image.

8. The apparatus of claim 7, the acquiring module is further configured to acquire a three-dimensional coordinate of the virtual camera, a direction angle of the virtual camera, and a camera extrinsic parameter of the virtual camera, the direction angle of the virtual camera including a pitch angle, a yaw angle, and a roll angle of the virtual camera, the camera extrinsic parameter of the virtual camera including a focal length and an optical center of the virtual camera; the determining module is specifically configured to determine the camera matrix of the virtual camera based on the three-dimensional coordinate of the virtual camera and the direction angle of the virtual camera, and determine the projection matrix of the virtual camera based on the camera extrinsic parameter of the virtual camera.

9. The apparatus of claim 7 or 8, the acquiring module is further configured to acquire a three-dimensional coordinate of each object captured by the virtual camera from the preset three-dimensional model after each rotation. The processing module is specifically configured to obtain two-dimensional coordinates of each object on a two-dimensional plane by respectively left-multiplying the three-dimensional coordinates of each object by the camera matrix and the projection matrix. The processing module is further specifically configured to obtain an image captured by the virtual camera after each rotation from the preset three-dimensional model based on the two-dimensional coordinates of each object on the two-dimensional plane.

10. The apparatus of claim 7 or 8, The processing module is specifically configured to perform stitching processing on each image captured by the virtual camera in the preset three-dimensional model, and perform fusion processing on edges of each image to obtain the first image.

11. The apparatus of claim 7 or 8, The acquisition module is further configured to acquire a plurality of images collected by the real camera, the plurality of images being images collected by the real camera at each of a plurality of time instants included in a current time period; The processing module is further configured to project the plurality of images onto a preset grid model to obtain the preset three-dimensional model.

12. The apparatus of claim 7 or 8, The determination module is further configured to determine a current gear of the target vehicle in a case where a turn signal of the target vehicle is in an on state and / or a current driving speed of the target vehicle is less than a speed threshold. The display module is further configured to display a second image in a case where the current gear of the target vehicle is a forward gear, the second image including road condition information in front of the target vehicle.

13. An electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-6.

14. A non-transitory computer readable storage medium having stored thereon computer instructions, wherein, The computer instructions are used to enable the computer to perform the method of any one of claims 1-6.

15. A computer program product comprising a computer program which, when executed by a processor, implements the method of any one of claims 1-6.

16. An autonomous vehicle comprising the electronic device of claim 13.

Images