Image display method and device and electronic equipment

Abstract

The invention discloses an image display method and device and electronic equipment, and belongs to the technical field of image processing. The method comprises the steps of obtaining a first image outside a vehicle; based on a moving object outside the vehicle and the first image, a second image is obtained, and the second image is a blind area image outside the vehicle; and displaying the second image on a target display screen, wherein the target display screen is at least one display screen in a plurality of vehicle-mounted display screens.

Description

Technical Field

[0001] The present application belongs to the field of image processing technology, and specifically relates to an image display method, device and electronic equipment. Background Art

[0002] At present, after the vehicle stops, if the user opens the door and gets out directly, there will be potential risks, which can easily cause traffic accidents and pose a safety threat to the user in the car and pedestrians and vehicles outside the car. In order to avoid safety accidents caused by directly opening the door, the user in the car needs to consciously observe the traffic environment outside the vehicle by looking at the rearview mirror before getting out of the car. However, if the user in the car has a small field of view through the rearview mirror or forgets to observe the rearview mirror, it is easy to cause a safety accident. At present, there is no solution on the market to remind the user to avoid potential risks when getting out of the car. Summary of the invention

[0003] The purpose of the embodiments of the present application is to provide an image display method, device and electronic device, which can at least partially solve the above-mentioned problems.

[0004] In a first aspect, an embodiment of the present application provides an image display method, comprising:

[0005] Acquiring a first image of an exterior of the vehicle;

[0006] Based on the moving object outside the vehicle and the first image, a second image is obtained, where the second image is a blind spot image outside the vehicle;

[0007] The second image is displayed on a target display screen, where the target display screen is at least one display screen among a plurality of vehicle-mounted display screens.

[0008] In a second aspect, an embodiment of the present application provides an image display device, including:

[0009] A first acquisition module, used to acquire a first image of the exterior of the vehicle;

[0010] A first processing module, configured to obtain a second image based on a moving object outside the vehicle and the first image, wherein the second image is a blind spot image outside the vehicle;

[0011] The first display module is used to display the second image on a target display screen, where the target display screen is at least one of a plurality of vehicle-mounted display screens.

[0012] In a third aspect, an embodiment of the present application further provides an electronic device, including a memory, a transceiver, and a processor:

[0013] The memory is used to store a computer program; the transceiver is used to send and receive data under the control of the processor; and the processor is used to read the computer program in the memory and execute the method as described in the first aspect above.

[0014] In a fourth aspect, an embodiment of the present application further provides a processor-readable storage medium, wherein the processor-readable storage medium stores a computer program, and the computer program is used to enable a processor to execute the method described in the first aspect above.

[0015] In an embodiment of the present application, a first image of the outside of the vehicle is first acquired, a second image is obtained based on a moving object outside the vehicle and the first image, and the second image is displayed on a target display screen. Since the second image is a blind spot image of the outside of the vehicle obtained based on the moving object outside the vehicle and the first image, the picture of the blind spot image is displayed in the form of an image on the corresponding display screen. This intuitive visual presentation method is not affected by the environment and does not require any operation by the user. It can help users in the vehicle identify the risks of getting off the vehicle and avoid various traffic accidents caused by opening the door and getting off the vehicle. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 is a flow chart of an image display method provided in an embodiment of the present application;

[0017] Figure 2 is a schematic diagram of a specific process of the image display method provided in an embodiment of the present application;

[0018] Figure 3 is a schematic structural diagram of an image display device provided in an embodiment of the present application;

[0019] Figure 4 It is a structural block diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0020] The following will be combined with the drawings in the embodiments of the present application to clearly describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the present application belong to the scope of protection of this application.

[0021] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the terms used in this way are interchangeable where appropriate, so that the embodiments of the present application can be implemented in an order other than those illustrated or described herein, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0022] The image display method provided by the embodiment of the present application is described below through specific embodiments and their application scenarios in conjunction with the accompanying drawings.

[0023] like Figure 1 As shown, the embodiment of the present application provides an image display method, which may specifically include the following steps:

[0024] Step 101 : acquiring a first image of the exterior of a vehicle.

[0025] A first image of the exterior of the vehicle is acquired in real time, and the first image displays a real-time picture of the exterior of the vehicle, so that the user can know the current environment conditions outside the vehicle from the real-time picture.

[0026] Step 102 : obtaining a second image based on the moving object outside the vehicle and the first image, wherein the second image is a blind spot image outside the vehicle.

[0027] Based on multiple moving objects outside the vehicle (such as cars, motorcycles, electric vehicles, bicycles, pedestrians, etc.) and the first image outside the vehicle, a blind spot image outside the vehicle can be obtained. The blind spot image is used to help users in the vehicle identify the risks of getting off the vehicle and avoid various traffic accidents caused by opening the door and getting off the vehicle.

[0028] Step 103: display the second image on a target display screen, where the target display screen is at least one display screen among a plurality of vehicle-mounted display screens.

[0029] A target display screen among multiple vehicle-mounted display screens (main driver's display screen or central control display screen, co-driver's display screen, rear display screen) is determined, and the second image is displayed on the target display screen, so that the user in the vehicle who needs to open the door can promptly obtain the real-time image outside the vehicle through the display image on the target display screen, thereby helping the user in the vehicle to identify the risk of getting off the vehicle and avoid various traffic accidents caused by opening the door and getting off the vehicle.

[0030] Wherein, the number of target display screens is one or more. If the number of target display screens is multiple and the number of second images is one, the second image is displayed on multiple target display screens. If the number of target display screens is multiple and the number of second images is multiple, different second images are displayed on corresponding target display screens.

[0031] In an embodiment of the present application, a first image of the outside of the vehicle is first acquired, a second image is obtained based on a moving object outside the vehicle and the first image, and the second image is displayed on a target display screen. Since the second image is a blind spot image of the outside of the vehicle obtained based on the moving object outside the vehicle and the first image, the picture of the blind spot image is displayed in the form of an image on the corresponding display screen. This intuitive visual presentation method is not affected by the environment and does not require any operation by the user. It can help users in the vehicle identify the risks of getting off the vehicle and avoid various traffic accidents caused by opening the door and getting off the vehicle.

[0032] As an optional specific embodiment, the step 101 of acquiring a first image of the exterior of the vehicle specifically includes:

[0033] Step 1011, obtaining an initial image of the exterior of the vehicle.

[0034] An image of the outside of the vehicle is captured by a vehicle-mounted camera deployed outside the vehicle, and the image is used as the initial image. Since the captured image is an image of the outside of the vehicle, the front of the vehicle can be directly viewed. Therefore, the vehicle-mounted camera can be a surround-view camera device arranged outside the vehicle, and can include one or more, for example: it can be installed in the front fender, rearview mirror and other positions, so as to capture images of the rear of the vehicle that are inconvenient for users inside the vehicle to see.

[0035] The initial image captured by the vehicle-mounted camera is subjected to dedistortion processing to eliminate the distortion of the initial image caused by lens distortion, thereby obtaining the dedistorted initial image. In order to eliminate the image distortion caused by the lens distortion of the vehicle-mounted camera, a fine dedistortion correction processing is performed on the initial image to obtain a more accurate and true corrected initial image. The corrected initial image is used as the initial image of the vehicle exterior to perform image processing below.

[0036] Step 1012, performing coordinate transformation on the coordinates of each pixel point in the initial image, determining the three-dimensional coordinates of each pixel point in the three-dimensional model in the three-dimensional space, so as to establish a mapping relationship between the initial image and the three-dimensional model, wherein the three-dimensional model is an initial three-dimensional model constructed based on the world coordinate system.

[0037] Based on the world coordinate system, the initial three-dimensional model is constructed using modeling technology and algorithms. The coordinates of each pixel in the initial image are transformed. Through this coordinate transformation process, the actual coordinates of each pixel in the initial image in the three-dimensional space can be accurately determined, thereby achieving accurate correspondence between the two-dimensional image and the three-dimensional model.

[0038] Step 1013: Based on the determined three-dimensional coordinates, image rendering is performed on the three-dimensional model to obtain the first image.

[0039] Based on the determined three-dimensional coordinates, image rendering technology and algorithms are used to perform image rendering processing on the three-dimensional model, and by adjusting rendering parameters such as lighting, shadows, and textures, a rendered first image with realistic effects is generated and output.

[0040] As another optional specific embodiment, the step 102 of obtaining the second image based on the moving object outside the vehicle and the first image may specifically include:

[0041] A second image is obtained based on a moving speed of a moving object outside the vehicle, a door opening time of a user inside the vehicle, and the first image.

[0042] Get the moving speed v (m / s) of multiple moving objects outside the vehicle. The moving speed corresponding to each moving object can be the common speed of the moving object on the current road section, or the maximum driving speed of the moving object, etc., which can be set as needed.

[0043] In addition, the door opening time t (seconds) required for different users in the car to open different doors is obtained. The users in the car include but are not limited to: the elderly, middle-aged people, young people, children, etc. The doors include: the driver's door, the co-pilot door, the rear left door, and the rear right door.

[0044] The blind spot image outside the vehicle can be obtained based on the moving speeds of multiple moving objects outside the vehicle, the door opening time of different users in the vehicle, and the first image outside the vehicle. The operating range of the blind spot image obtained through the moving speeds of the moving objects and the door opening time of different users in the vehicle is more accurate, avoiding various traffic accidents caused by opening the door and getting off the vehicle.

[0045] As another optional specific embodiment, the step of obtaining the second image based on the moving speed of the moving object outside the vehicle, the door opening time of the user inside the vehicle, and the first image specifically includes:

[0046] Step 1021, generating a projection transformation matrix based on the moving speed and the door opening time;

[0047] Step 1022: crop the first image based on the projection transformation matrix to obtain a second image.

[0048] Based on the moving speeds of all movable objects outside the vehicle and the door opening time taken by different users inside the vehicle to open the door, a projection transformation matrix (Model-View-Projection matrix,) MVP matrix is ​​generated. The first image is cropped based on the projection transformation matrix to obtain a cropped second image. This allows the user inside the vehicle to see the outside environment to the greatest extent before opening the door, thereby ensuring personal safety during the door opening process while saving computing resources.

[0049] Among them, the MVP matrix is ​​the key to converting points in three-dimensional space to screen space. This matrix combines the model matrix (Model matrix), view matrix (View matrix) and projection matrix (Projection matrix).

[0050] As another optional specific embodiment, the step 1021 generates a projection transformation matrix based on the moving speed and the door opening time, which may specifically include:

[0051] Obtaining moving speeds of multiple moving objects outside the vehicle and determining a maximum moving speed;

[0052] Obtain the door opening time required for different users in the car to open different doors, and determine the maximum door opening time;

[0053] The projection transformation matrix is ​​generated based on the maximum moving speed and the maximum door opening time.

[0054] The moving speeds v (m / s) of multiple moving objects outside the vehicle are obtained, and the moving speeds of all the moving objects are compared to determine the maximum moving speed.

[0055] In addition, the door opening time t (seconds) required for different users in the car to open different doors is obtained, and the door opening time required for different users to open different doors is compared to determine the maximum door opening time.

[0056] Then the minimum depth of field of view should be s=v*t, and the image field of view seen by the user in the car should be greater than or equal to s, that is, the most critical View matrix in the MVP matrix is ​​generated by s, and the first image is cropped by the MVP matrix. In this way, on the basis of saving computing resources, the user in the car can see the outside environment to the greatest extent before opening the door, thereby ensuring personal safety during the door opening process.

[0057] As another optional specific embodiment, the step 1012 performs coordinate transformation on the coordinates of each pixel point in the initial image to determine the three-dimensional coordinates of each pixel point in the three-dimensional model in the three-dimensional space, specifically including:

[0058] Get the intrinsic and extrinsic matrix of the vehicle camera;

[0059] The coordinates of each pixel point in the initial image are transformed by using the intrinsic parameter matrix and the extrinsic parameter matrix to determine the three-dimensional coordinates of each pixel point in the three-dimensional model in the three-dimensional space.

[0060] Obtain the internal parameter matrix (i.e., intrinsic parameter matrix) and external parameter matrix (i.e., extrinsic parameter matrix) of the vehicle-mounted camera calibration. The internal parameter matrix describes in detail the internal optical characteristics of the camera, including key parameters such as focal length and principal point position; the external parameter matrix accurately reflects the camera's posture (including rotation angle) and position (including translation distance) relative to the world coordinate system.

[0061] The coordinates of each pixel in the initial image are transformed using the intrinsic parameter matrix and the extrinsic parameter matrix. Through this coordinate transformation process, the actual coordinates of each pixel in the initial image in the three-dimensional space can be accurately determined, thereby achieving accurate correspondence between the two-dimensional image and the three-dimensional model.

[0062] Specifically, coordinate conversion is performed in the following way:

[0063]

[0064] in, is the internal parameter matrix;

[0065] is the external parameter matrix;

[0066] (u, v) is the coordinate of the pixel in the initial image;

[0067] (x, y, z) is the three-dimensional coordinate of the pixel point in three-dimensional space.

[0068] The following two specific embodiments are used to illustrate the process of determining the target display screen and the process of image display:

[0069] Embodiment 1

[0070] The method further comprises:

[0071] Acquire a vehicle signal, and determine a target display screen according to the vehicle signal;

[0072] In the case where the target display screen is not a preset display screen, the initial image is flipped according to the target display screen and the preset display screen to obtain a flipped initial image;

[0073] Wherein, when the coordinates of each pixel point in the initial image are transformed, the initial image is a flipped initial image.

[0074] Acquire an initial image of the exterior of the vehicle. Also, acquire a vehicle signal, and determine a target display screen from a plurality of vehicle-mounted display screens (such as a central control display screen, a co-pilot display screen, and a rear display screen) based on the vehicle signal, that is, determine the target display screen by triggering the vehicle signal. Set a preset display screen as the reference display screen. If the target display screen is a preset display screen, after obtaining the initial image, directly perform coordinate conversion on the coordinates of each pixel point in the initial image, determine the three-dimensional coordinates of each pixel point in the three-dimensional model in the three-dimensional space, and establish a mapping relationship between the initial image and the three-dimensional model. Based on the determined three-dimensional coordinates, render the three-dimensional model to obtain a first image, then crop the first image based on the projection transformation matrix to obtain a second image, and display the second image on the target display screen.

[0075] If the target display screen is not a preset display screen, the initial image needs to be flipped according to the parameters of the target display screen and the parameters of the preset display screen to obtain the flipped initial image, so as to display the image at the viewing angle of the user in the car, and achieve a visual effect that is intuitive, accurate and matches the actual order. The coordinates of each pixel point in the flipped initial image are transformed to determine the three-dimensional coordinates of each pixel point in the three-dimensional model in the three-dimensional space, so as to establish a mapping relationship between the flipped initial image and the three-dimensional model. Based on the determined three-dimensional coordinates, the three-dimensional model is rendered to obtain a first image, and then the first image is cropped based on the projection transformation matrix to obtain a second image, and the second image is displayed on the target display screen, so that the user in the car can intuitively see the picture outside the vehicle before opening the door, which helps the user in the car identify the risk of getting off the vehicle and avoid various traffic accidents caused by opening the door and getting off the vehicle.

[0076] For example: if there is a display screen corresponding to the rear seat behind the main driver's seat and a display screen corresponding to the rear seat behind the co-driver's seat in the rear seat. If the display screen corresponding to the main driver's seat and the display screen corresponding to the rear seat behind the main driver's seat are set as preset display screens, if the target display screen is the display screen corresponding to the main driver's seat or the display screen corresponding to the rear seat behind the main driver's seat, there is no need to flip the initial image; if the target display screen is the display screen corresponding to the co-driver's seat or the display screen corresponding to the rear seat behind the co-driver's seat, the initial image needs to be flipped according to the parameters of the display screen corresponding to the main driver's seat or the parameters of the display screen corresponding to the rear seat behind the main driver's seat.

[0077] Another example: if there is only one display screen in the back seat and it is installed in the middle position. If the display screen corresponding to the main driver's seat is set as the preset display screen, if the target display screen is the display screen corresponding to the main driver's seat, there is no need to flip the initial image; if the target display screen is the display screen corresponding to the co-driver's seat, the initial image needs to be flipped according to the parameters of the display screen corresponding to the main driver's seat; if the target display screen is the display screen corresponding to the back seat, the initial image needs to be flipped according to the parameters of the display screen corresponding to the main driver's seat and the position of the vehicle signal.

[0078] It should be noted that there is no particular order in which the steps of acquiring the initial image and determining the target display screen are performed.

[0079] Furthermore, the vehicle signal includes but is not limited to at least one of the following:

[0080] Door status signal, seat belt status signal, seat occupancy status signal, display screen signal requesting image display, sound zone signal inside the car, image acquisition signal from the onboard camera.

[0081] In some embodiments, the vehicle signal includes a door status signal. If the door status signal indicates that the door status has switched from a closed state to an open state, the display screen corresponding to the door whose door status signal has switched from a closed state to an open state is determined as the target display screen, and then whether to perform image flipping processing is determined based on the location of the target display screen.

[0082] Exemplarily, the vehicle includes a main driving display screen or a central control display screen corresponding to the main driving seat, a co-pilot display screen corresponding to the co-pilot seat, and a rear display screen corresponding to the rear seats. If the signal of the door state of the main driving seat indicates that the door state is switched from a closed state to an open state, the main driving display screen or the central control display screen corresponding to the main driving seat is determined to be the target display screen. If the main driving display screen or the central control display screen is a preset display screen, it is determined that there is no need to perform image flipping processing.

[0083] In some embodiments, the vehicle signal includes a seat belt status signal. If the seat belt status signal indicates that the seat belt status has switched from a closed state to an open state, the display screen corresponding to the seat belt whose seat belt status signal has switched from a closed state to an open state is determined as the target display screen, and then whether to perform image flipping processing is determined based on the location of the target display screen.

[0084] Exemplarily, the vehicle includes a main driving display screen or a central control display screen corresponding to the main driving seat, a co-pilot display screen corresponding to the co-pilot seat, and a rear display screen corresponding to the rear seats. If the seat belt status signals of the main driving seat and the co-pilot seat indicate that the seat belt status is switched from a closed state to an open state, it is determined that the main driving display screen or the central control display screen corresponding to the main driving seat and the co-pilot display screen corresponding to the co-pilot seat are both target display screens. If the main driving display screen or the central control display screen is a preset display screen, it is determined that there is no need to perform image flip processing; if the co-pilot display screen is a non-preset display screen, it is determined that image flip processing needs to be performed.

[0085] In some embodiments, the vehicle signal includes a signal of a seat occupancy status. If the signal of the seat occupancy status indicates that the seat occupancy status has switched from an occupied state to an idle state, the display screen corresponding to the seat whose seat occupancy status signal has switched from an occupied state to an idle state is determined as the target display screen, and then whether to perform image flipping processing is determined based on the location of the target display screen.

[0086] Exemplarily, the vehicle includes a main driving display screen or a central control display screen corresponding to the main driving seat, a co-pilot display screen corresponding to the co-pilot seat, and a rear display screen corresponding to the rear seat. If the seat occupancy status signal of the rear seat indicates that the seat occupancy status is switched from an occupied state to an idle state, the rear display screen corresponding to the rear seat is determined to be the target display screen. If the rear display screen is a preset display screen, it is determined that there is no need to perform image flipping processing.

[0087] In some embodiments, the vehicle signal includes a signal of a display screen requesting image display. If the display screen in the vehicle requests image display, the display screen requesting image display is determined as the target display screen, and then whether to perform image flipping processing is determined based on the location of the target display screen.

[0088] Exemplarily, the vehicle includes a main driving display screen or a central control display screen corresponding to the main driving seat, a co-pilot display screen corresponding to the co-pilot seat, and a rear display screen corresponding to the rear seats. If the rear display screen requests image display, the rear display screen is the target display screen. If the rear display screen is a preset display screen, it is determined that there is no need to perform image flipping processing.

[0089] In some embodiments, the vehicle signal includes an audio zone signal inside the vehicle. If the audio zone signal inside the vehicle indicates that the audio zone has acquired voice information about opening the door and getting off the vehicle, the display screen corresponding to the audio zone inside the vehicle that has acquired the voice information about opening the door and getting off the vehicle is determined as the target display screen, and then whether to perform image flipping processing is determined based on the location of the target display screen.

[0090] For example, the vehicle includes a main driving display screen or a central control display screen corresponding to the main driving seat, a co-pilot display screen corresponding to the co-pilot seat, and a rear display screen corresponding to the rear seats. If the audio zone of the rear seats acquires the voice information of the user in the vehicle closing the door and getting off the vehicle, the rear display screen corresponding to the audio zone of the rear seats is determined to be the target display screen. If the rear display screen is a preset display screen, it is determined that there is no need to perform image flipping processing.

[0091] In some embodiments, the vehicle signal includes an image acquisition signal from a vehicle-mounted camera. If the image acquisition signal from the vehicle-mounted camera indicates that the vehicle-mounted camera acquires an image outside the vehicle, the display screen corresponding to the vehicle-mounted camera that acquires the image outside the vehicle is determined as the target display screen, and then whether to perform image flipping processing is determined based on the location of the target display screen.

[0092] Exemplarily, the vehicle includes a main driving display screen or a central control display screen corresponding to the main driving seat, a co-pilot display screen corresponding to the co-pilot seat, and a rear display screen corresponding to the rear seats. If the vehicle-mounted camera outside the main driving seat acquires an image outside the vehicle, the main driving display screen or the central control display screen corresponding to the main driving seat is determined to be the target display screen. If the main driving display screen or the central control display screen is a preset display screen, it is determined that there is no need to perform image flipping processing.

[0093] Embodiment 2

[0094] The method further comprises:

[0095] Acquire a vehicle signal, and determine a target display screen according to the vehicle signal;

[0096] In the case where the target display screen is not a preset display screen, flipping the second image according to the target display screen and the preset display screen to obtain a flipped second image;

[0097] Wherein, when the second image is displayed on the target display screen, the second image is a flipped second image.

[0098] like Figure 2 As shown, an initial image of the exterior of the vehicle is obtained through the vehicle-mounted camera of the vehicle. The coordinates of each pixel point in the initial image are transformed to determine the three-dimensional coordinates of each pixel point in the three-dimensional model in the three-dimensional space, so as to establish a mapping relationship between the initial image and the three-dimensional model. Based on the determined three-dimensional coordinates, the three-dimensional model is rendered to obtain a first image, and then the first image is cropped based on the projection transformation matrix to obtain a second image.

[0099] In addition, a vehicle signal is obtained, and a target display screen is determined from multiple vehicle-mounted display screens (such as a central control display screen, a co-pilot display screen, and a rear display screen) based on the vehicle signal, that is, the target display screen is determined by the trigger source that triggers the vehicle signal. The preset display screen is set as the reference display screen. If the target display screen is a preset display screen, after obtaining the second image, the second image is directly displayed on the target display screen. If the target display screen is a non-preset display screen, it is necessary to first flip the second image according to the parameters of the target display screen and the parameters of the preset display screen to obtain the flipped second image, so as to display the image at the viewing angle of the user in the car, so as to achieve a visual effect that is intuitive, accurate, and matches the actual order. The flipped second image is displayed on the target display screen, so that the user in the car can intuitively see the picture outside the vehicle before opening the door, helping the user in the car to identify the risk of getting off the vehicle and avoid various traffic accidents caused by opening the door and getting off the vehicle.

[0100] For example: if among the back seats, the back seat behind the main driver's seat corresponds to a display screen, and the back seat behind the co-driver's seat corresponds to a display screen. If the display screen corresponding to the main driver's seat and the display screen corresponding to the back seat behind the main driver's seat are set as preset display screens, if the target display screen is the display screen corresponding to the main driver's seat or the display screen corresponding to the back seat behind the main driver's seat, there is no need to flip the second image, and the second image is directly displayed on the display screen corresponding to the main driver's seat or the display screen corresponding to the back seat behind the main driver's seat; if the target display screen is the display screen corresponding to the co-driver's seat or the display screen corresponding to the back seat behind the co-driver's seat, the second image needs to be flipped according to the parameters of the display screen corresponding to the main driver's seat or the parameters of the display screen corresponding to the back seat behind the main driver's seat, and then displayed on the display screen corresponding to the co-driver's seat or the display screen corresponding to the back seat behind the co-driver's seat.

[0101] Another example: if there is only one display screen in the back seat and it is installed in the middle position. If the display screen corresponding to the main driver's seat is set as the preset display screen, if the target display screen is the display screen corresponding to the main driver's seat, there is no need to flip the second image, and the second image can be directly displayed on the display screen corresponding to the main driver's seat. If the target display screen is the display screen corresponding to the co-driver's seat, the second image needs to be flipped according to the parameters of the display screen corresponding to the main driver's seat and displayed on the display screen corresponding to the co-driver's seat; if the target display screen is the display screen corresponding to the back seat, the second image needs to be flipped according to the parameters of the display screen corresponding to the main driver's seat and the position of the vehicle signal and displayed on the display screen corresponding to the back seat.

[0102] It should be noted that there is no particular order in which the steps of acquiring the second image and determining the target display screen are performed.

[0103] Furthermore, the vehicle signal includes but is not limited to at least one of the following:

[0104] Door status signal, seat belt status signal, seat occupancy status signal, display screen signal requesting image display, sound zone signal inside the car, image acquisition signal from the onboard camera.

[0105] As another optional specific embodiment, the method further includes:

[0106] When the signal of the seat occupation state and / or the signal of the door state meets a preset condition, the acquisition of the first image outside the vehicle is stopped.

[0107] When at least one of the seat occupancy status signal and the door status signal in the vehicle signal meets a preset condition (such as: the door is closed, the user in the car fastens the seat belt, etc.), stop acquiring the first image outside the vehicle. The stopping method can be to turn off the vehicle-mounted camera.

[0108] In some embodiments, on the basis of stopping acquiring the first image outside the vehicle, the following operations may also be performed: controlling the target display screen to exit, stopping image rendering, hiding the window, etc.

[0109] Among them, the preset conditions include but are not limited to: the signal of the seat occupancy status is the signal of the non-occupied status (that is, the seat is in an empty state), and the signal of the door status is the signal of the door closed status (that is, the door is in a closed state).

[0110] In summary, the embodiment of the present application generates the projection transformation matrix through the maximum moving speed of multiple moving objects outside the vehicle and the maximum door opening time required for different users in the vehicle to open different doors. The projection transformation matrix can be used to crop the image of the area where the user in the vehicle may cause danger by opening the door, and the image of the area is displayed in the form of an image on the corresponding display screen. This intuitive visual presentation method is not affected by the environment and does not require any operation by the user. It can help the user in the vehicle identify the risk of getting off the vehicle and avoid various traffic accidents caused by opening the door and getting off the vehicle. In addition, by integrating vehicle signals (doors, seat belts, seat occupancy status, etc.) through image processing technology, the blind spot image is restored for the user in the vehicle more intelligently and intuitively, improving the safety of the user in the vehicle getting off the vehicle. In addition, for a multi-display vehicle system, the target display screen is determined by triggering the trigger source of the vehicle signal, and the image is displayed on the specified target display screen, with accurate pointing, which is more convenient for users who need to get off the vehicle to watch. The entire solution achieves accurate conversion and rendering from two-dimensional images to three-dimensional models through steps such as building a three-dimensional model, de-distortion correction processing, obtaining vehicle-mounted camera parameters, coordinate transformation, and image rendering, providing strong technical support for the fields of vehicle-mounted vision systems, autonomous driving technology, etc.

[0111] The image display method provided in the embodiment of the present application can be executed by an image display device. In the embodiment of the present application, an image display device executing the image display method is taken as an example to illustrate the image display device provided in the embodiment of the present application.

[0112] like Figure 3 As shown, the embodiment of the present application further provides an image display device 300, which specifically includes:

[0113] A first acquisition module 301 is used to acquire a first image of the exterior of the vehicle;

[0114] A first processing module 302 is used to obtain a second image based on the moving object outside the vehicle and the first image, where the second image is a blind spot image outside the vehicle;

[0115] The first display module 303 is used to display the second image on a target display screen, where the target display screen is at least one display screen among a plurality of vehicle-mounted display screens.

[0116] Optionally, the first acquisition module 301 is specifically configured to:

[0117] Acquire an initial image of the exterior of the vehicle;

[0118] Performing coordinate transformation on the coordinates of each pixel point in the initial image to determine the three-dimensional coordinates of each pixel point in the three-dimensional model in the three-dimensional space, so as to establish a mapping relationship between the initial image and the three-dimensional model, wherein the three-dimensional model is an initial three-dimensional model constructed based on the world coordinate system;

[0119] Based on the determined three-dimensional coordinates, image rendering is performed on the three-dimensional model to obtain the first image.

[0120] Optionally, when the first acquisition module 301 performs coordinate transformation on the coordinates of each pixel point in the initial image to determine the three-dimensional coordinates of each pixel point in the three-dimensional model in the three-dimensional space, it is specifically used to:

[0121] Get the intrinsic and extrinsic matrix of the vehicle camera;

[0122] The coordinates of each pixel point in the initial image are transformed by using the intrinsic parameter matrix and the extrinsic parameter matrix to determine the three-dimensional coordinates of each pixel point in the three-dimensional model in the three-dimensional space.

[0123] Optionally, the device further comprises:

[0124] A second acquisition module is used to acquire a vehicle signal and determine a target display screen according to the vehicle signal;

[0125] A second processing module is used for, when the target display screen is not a preset display screen, performing a flipping process on the initial image according to the target display screen and the preset display screen to obtain a flipped initial image;

[0126] When the first acquisition module 301 performs coordinate transformation on the coordinates of each pixel point in the initial image, the initial image is a flipped initial image.

[0127] Optionally, the device further comprises:

[0128] A third acquisition module is used to acquire a vehicle signal and determine a target display screen according to the vehicle signal;

[0129] A third processing module, configured to, when the target display screen is not a preset display screen, perform a flipping process on the second image according to the target display screen and the preset display screen to obtain a flipped second image;

[0130] When the first display module 303 displays the second image on the target display screen, the second image is a flipped second image.

[0131] Optionally, the vehicle signal includes at least one of the following:

[0132] Door status signal, seat belt status signal, seat occupancy status signal, display screen signal requesting image display, sound zone signal inside the car, image acquisition signal from the onboard camera.

[0133] Optionally, the device further comprises:

[0134] The stopping module is used to stop acquiring the first image outside the vehicle when the signal of the seat occupation state and / or the signal of the door state meets a preset condition.

[0135] Optionally, the first processing module 302 is specifically configured to:

[0136] A second image is obtained based on a moving speed of a moving object outside the vehicle, a door opening time of a user inside the vehicle, and the first image.

[0137] Optionally, when obtaining the second image based on the moving speed of the moving object outside the vehicle, the door opening time of the user inside the vehicle, and the first image, the first processing module 302 is specifically configured to:

[0138] Generate a projection transformation matrix based on the moving speed and the door opening time;

[0139] The first image is cropped based on the projection transformation matrix to obtain a second image.

[0140] Optionally, when generating a projection transformation matrix based on the moving speed and the door opening time, the first processing module 302 is specifically configured to:

[0141] Obtaining moving speeds of multiple moving objects outside the vehicle and determining a maximum moving speed;

[0142] Obtain the door opening time required for different users in the car to open different doors, and determine the maximum door opening time;

[0143] The projection transformation matrix is ​​generated based on the maximum moving speed and the maximum door opening time.

[0144] In summary, the embodiment of the present application generates the projection transformation matrix through the maximum moving speed of multiple moving objects outside the vehicle and the maximum door opening time required for different users in the vehicle to open different doors. The projection transformation matrix can be used to crop the image of the area where the user in the vehicle may cause danger by opening the door, and the image of the area is displayed in the form of an image on the corresponding display screen. This intuitive visual presentation method is not affected by the environment and does not require any operation by the user. It can help the user in the vehicle identify the risk of getting off the vehicle and avoid various traffic accidents caused by opening the door and getting off the vehicle. In addition, by integrating vehicle signals (doors, seat belts, seat occupancy status, etc.) through image processing technology, the blind spot image is restored for the user in the vehicle more intelligently and intuitively, improving the safety of the user in the vehicle getting off the vehicle. In addition, for a multi-display vehicle system, the target display screen is determined by triggering the trigger source of the vehicle signal, and the image is displayed on the specified target display screen, with accurate pointing, which is more convenient for users who need to get off the vehicle to watch. The entire solution achieves accurate conversion and rendering from two-dimensional images to three-dimensional models through steps such as building a three-dimensional model, de-distortion correction processing, obtaining vehicle-mounted camera parameters, coordinate transformation, and image rendering, providing strong technical support for the fields of vehicle-mounted vision systems, autonomous driving technology, etc.

[0145] It should be noted here that the above-mentioned image display device provided in the embodiment of the present application can implement all the method steps implemented in the above-mentioned image display method embodiment, and can achieve the same technical effect. The parts and beneficial effects of this embodiment that are the same as the method embodiment will not be described in detail here.

[0146] It should be noted that the division of units in the embodiments of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation. In addition, each functional unit in each embodiment of the present application may be integrated into a processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0147] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a processor-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) or a processor (processor) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), disk or optical disk and other media that can store program codes.

[0148] like Figure 4 As shown, an embodiment of the present application further provides an electronic device, including a memory 420, a transceiver 410, and a processor 400:

[0149] Memory 420, used for storing computer programs;

[0150] A transceiver 410, configured to transmit and receive data under the control of the processor;

[0151] The processor 400 is used to read the computer program in the memory and execute the steps of the image display method described in any of the above embodiments.

[0152] Among them, Figure 4 In the embodiment, the bus architecture may include any number of interconnected buses and bridges, specifically linking together various circuits of one or more processors represented by processor 400 and memory represented by memory 420. The bus architecture may also link together various other circuits such as peripherals, voltage regulators, and power management circuits, which are well known in the art and are therefore not further described herein. The bus interface provides an interface. The transceiver 410 may be a plurality of components, namely, a transmitter and a receiver, providing a unit for communicating with various other devices on a transmission medium, which transmission medium may include a wireless channel, a wired channel, an optical cable, and the like. The processor 400 is responsible for managing the bus architecture and general processing, and the memory 420 may store data used by the processor 400 when performing operations.

[0153] The processor 400 may be a central processing unit (CPU), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or a complex programmable logic device (CPLD). The processor may also adopt a multi-core architecture.

[0154] The processor calls the computer program stored in the memory to execute the image display method provided in the embodiment of the present application according to the obtained executable instructions. The processor and the memory can also be arranged physically separately.

[0155] It should be noted here that the above-mentioned electronic device provided in the embodiment of the present application can implement all the method steps implemented in the above-mentioned image display method embodiment, and can achieve the same technical effect. The parts and beneficial effects of this embodiment that are the same as the method embodiment will not be described in detail here.

[0156] An embodiment of the present application further provides a processor-readable storage medium, wherein the processor-readable storage medium stores a computer program, and the computer program is used to enable the processor to execute the above-mentioned image display method.

[0157] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage and optical storage, etc.) that contain computer-usable program code.

[0158] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer executable instructions. These computer executable instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0159] These processor executable instructions may also be stored in a processor readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the processor readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0160] These processor-executable instructions may also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0161] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. An image display method, characterized in that: include: acquiring a first image of an exterior of the vehicle; Based on the moving object outside the vehicle and the first image, a second image is obtained, where the second image is a blind spot image outside the vehicle; The second image is displayed on a target display screen, where the target display screen is at least one display screen among a plurality of vehicle-mounted display screens.

2. The method according to claim 1, characterized in that The step of acquiring a first image of the exterior of the vehicle comprises: Acquire an initial image of the exterior of the vehicle; Performing coordinate transformation on the coordinates of each pixel point in the initial image to determine the three-dimensional coordinates of each pixel point in the three-dimensional model in the three-dimensional space, so as to establish a mapping relationship between the initial image and the three-dimensional model, wherein the three-dimensional model is an initial three-dimensional model constructed based on the world coordinate system; Based on the determined three-dimensional coordinates, image rendering is performed on the three-dimensional model to obtain the first image.

3. The method according to claim 2, characterized in that The step of performing coordinate transformation on the coordinates of each pixel point in the initial image to determine the three-dimensional coordinates of each pixel point in the three-dimensional model in the three-dimensional space includes: Get the intrinsic and extrinsic matrix of the vehicle camera; The coordinates of each pixel point in the initial image are transformed by using the intrinsic parameter matrix and the extrinsic parameter matrix to determine the three-dimensional coordinates of each pixel point in the three-dimensional model in the three-dimensional space.

4. The method according to claim 2 or 3, characterized in that: The method further comprises: Acquire a vehicle signal, and determine a target display screen according to the vehicle signal; In the case where the target display screen is not a preset display screen, the initial image is flipped according to the target display screen and the preset display screen to obtain a flipped initial image; Wherein, when the coordinates of each pixel point in the initial image are transformed, the initial image is a flipped initial image.

5. The method according to any one of claims 1 to 3, characterized in that: The method further comprises: Acquire a vehicle signal, and determine a target display screen according to the vehicle signal; In the case where the target display screen is not a preset display screen, flipping the second image according to the target display screen and the preset display screen to obtain a flipped second image; Wherein, when the second image is displayed on the target display screen, the second image is a flipped second image.

6. The method according to claim 4 or 5, characterized in that: The vehicle signal includes at least one of the following: Door status signal, seat belt status signal, seat occupancy status signal, display screen signal requesting image display, sound zone signal inside the car, image acquisition signal from the onboard camera.

7. The method according to any one of claims 1 to 6, characterized in that: The method further comprises: When the signal of the seat occupation state and / or the signal of the door state meets a preset condition, the acquisition of the first image outside the vehicle is stopped.

8. The method according to any one of claims 1 to 7, characterized in that: The step of obtaining a second image based on the moving object outside the vehicle and the first image comprises: A second image is obtained based on a moving speed of a moving object outside the vehicle, a door opening time of a user inside the vehicle, and the first image.

9. The method according to claim 8, characterized in that The obtaining of the second image based on the moving speed of the moving object outside the vehicle, the door opening time of the user inside the vehicle, and the first image comprises: Generate a projection transformation matrix based on the moving speed and the door opening time; The first image is cropped based on the projection transformation matrix to obtain a second image.

10. The method according to claim 9, characterized in that The generating a projection transformation matrix based on the moving speed and the door opening time includes: Obtaining moving speeds of multiple moving objects outside the vehicle and determining a maximum moving speed; Obtain the door opening time required for different users in the car to open different doors, and determine the maximum door opening time; The projection transformation matrix is ​​generated based on the maximum moving speed and the maximum door opening time.

11. An image display device, characterized in that: include: A first acquisition module, used to acquire a first image of the exterior of the vehicle; A first processing module, configured to obtain a second image based on a moving object outside the vehicle and the first image, wherein the second image is a blind spot image outside the vehicle; The first display module is used to display the second image on a target display screen, where the target display screen is at least one of a plurality of vehicle-mounted display screens.

12. An electronic device, characterized in that: Including memory, transceiver, processor: The memory is used to store computer programs; the transceiver is used to send and receive data under the control of the processor; the processor is used to read the computer program in the memory and execute the steps of the image display method as described in any one of claims 1-10.

13. A processor-readable storage medium, characterized in that: The processor-readable storage medium stores a computer program, and the computer program is used to enable the processor to execute the steps of the image display method according to any one of claims 1 to 10.

Images