Image processing method and device, vehicle, and readable storage medium

Abstract

The present disclosure relates to an image processing method and device, a vehicle, and a readable storage medium. The method includes: obtaining the real-time posture of the vehicle and obtaining a first number of frames of initial texture images; obtaining a first texture image corresponding to each frame of initial texture image based on the real-time posture and each frame of initial texture image; converting the first texture image to the coordinate system of the vehicle and obtaining the area corresponding to the bottom of the vehicle to obtain a second texture image; performing mixed rendering on the second texture images corresponding to the first number of frames of initial texture images to obtain a target image representing the scene of the bottom area of ​​the vehicle. By obtaining a target image of the scene of the bottom area of ​​the vehicle, this embodiment can monitor the road surface condition at the bottom of the vehicle, expand the field of view, that is, reduce the blind spot of the field of view, which is conducive to improving the safety of unmanned vehicles.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing technology, and in particular to an image processing method and device, a vehicle, and a readable storage medium. Background Art

[0002] Autonomous driving relies on the collaborative efforts of artificial intelligence, visual computing, radar, monitoring devices and global positioning systems, allowing computers to automatically operate vehicles without any active human intervention.

[0003] However, existing driverless cars are limited by the number of sensors and have blind spots in their field of vision, which makes it impossible to guarantee driving safety in some special scenarios and reduces the user experience. Summary of the Invention

[0004] The present disclosure provides an image processing method and apparatus, a vehicle, and a readable storage medium to address the deficiencies of related technologies.

[0005] According to a first aspect of an embodiment of the present disclosure, there is provided an image processing method, the method comprising:

[0006] Obtaining the real-time position and posture of the vehicle and obtaining a first number of frames of initial texture images;

[0007] Acquire a first texture image corresponding to each initial texture image of each frame according to the real-time posture and the initial texture image of each frame;

[0008] Converting the first texture image to the coordinate system of the vehicle and acquiring an area corresponding to the bottom of the vehicle to obtain a second texture image;

[0009] The second texture images corresponding to the first number of frames of initial texture images are mixed and rendered to obtain a target image representing the scene of the bottom area of ​​the vehicle.

[0010] Optionally, obtaining a first number of frames of initial texture images includes:

[0011] Taking the latest initial texture image at the designated position as the first frame, acquiring the initial texture images of other frames at a preset time interval to obtain the first number of frames of initial texture images;

[0012] The time interval between two adjacent frames in the first number of initial texture image frames exceeds the preset time interval.

[0013] Optionally, obtaining a first number of frames of initial texture images includes:

[0014] Taking the latest initial texture image stored in the designated position as the first frame, obtaining initial texture images of other frames according to a preset distance threshold, and obtaining the first number of initial texture images;

[0015] The distance between two adjacent frames of the first number of initial texture images exceeds the preset distance threshold.

[0016] Optionally, the method further includes the step of generating an initial texture image in a specified position, specifically comprising:

[0017] Obtaining the position and posture of the vehicle in each acquisition cycle, and obtaining a surround view image in each acquisition cycle; the surround view image is formed by stitching images acquired by at least four cameras installed on the vehicle;

[0018] Performing rendering and deformation processing on the surround view image to obtain a deformed texture image;

[0019] The posture of the vehicle and the deformed texture image are bound to obtain the initial texture image.

[0020] Optionally, rendering and deforming the surround view image includes:

[0021] Rendering the surround view image in a frame buffer manner to obtain an intermediate rendered image;

[0022] The intermediate rendering image is projected onto a preset bowl-shaped area for deformation processing to obtain the deformed texture image; wherein the shape of the bowl-shaped area matches the shape of the scene in which the vehicle is located.

[0023] Optionally, obtain the real-time position of the vehicle, including:

[0024] Acquiring inertial data collected by an inertial sensor of the vehicle, the number of pulses collected by a pulse sensor of the vehicle, a previous position of the vehicle in a previous cycle, and a previous navigation angle;

[0025] Based on a preset posture model, the real-time posture of the vehicle in the current acquisition cycle is calculated according to the inertial data, the pulse data, the previous position and the previous navigation angle; the real-time posture includes the current position and current navigation angle of the vehicle.

[0026] Optionally, obtaining a first texture image corresponding to each initial texture image of each frame according to the real-time posture and the initial texture image of each frame includes:

[0027] Get the pose of the initial texture image of each frame;

[0028] Obtaining a transformation matrix between the pose of each frame of the initial texture image and the real-time pose;

[0029] Obtaining a rotation submatrix and a translation submatrix of a transformation matrix of an initial texture image of each frame, and adjusting the rotation submatrix and the translation submatrix to a target transformation matrix;

[0030] Rendering the initial texture image corresponding to the target transformation matrix to obtain a first texture image corresponding to each frame of the initial texture image.

[0031] Optionally, converting the first texture image to the coordinate system of the vehicle and acquiring an area corresponding to the bottom of the vehicle to obtain a second texture image includes:

[0032] Converting a first texture image corresponding to each frame of the initial texture image to the coordinate system of the vehicle so that the bottom area of ​​the vehicle in the first texture image matches the area where the bottom of the vehicle passes;

[0033] A bottom region image of the bottom region of the vehicle in each frame of the first texture image is acquired to obtain a first number of frames of bottom region images and use the bottom region images as the second texture image.

[0034] Optionally, performing mixed rendering on the bottom area images of the first number of frames to obtain a target image representing the scene of the bottom area of ​​the vehicle includes:

[0035] For pixel points at the same position of the bottom area images of the first number of frames, obtaining the maximum value of each color component of the pixel points at the same position;

[0036] The values ​​of the color components are updated to corresponding maximum values ​​to obtain a rendered image, and the rendered image is used as a target image representing the scene in the bottom area of ​​the vehicle.

[0037] Optionally, the method further includes:

[0038] The target image is presented in a bottom area of ​​the vehicle corresponding model.

[0039] According to a second aspect of an embodiment of the present disclosure, there is provided an image processing apparatus, the apparatus comprising:

[0040] Real-time posture acquisition module, used to obtain the real-time posture of the vehicle

[0041] An initial image acquisition module, configured to acquire a first number of frames of initial texture images;

[0042] A first image acquisition module is used to acquire a first texture image corresponding to each initial texture image frame according to the real-time posture and each initial texture image frame;

[0043] A second image acquisition module is used to convert the first texture image to the coordinate system of the vehicle and acquire an area corresponding to the bottom of the vehicle to obtain a second texture image;

[0044] The target image acquisition module is used to perform mixed rendering on the second texture images corresponding to the first number of frames of initial texture images to obtain a target image representing the scene of the bottom area of ​​the vehicle.

[0045] Optionally, the initial image acquisition module includes:

[0046] A first acquisition submodule is configured to take the latest initial texture image at a specified position as a first frame, and acquire initial texture images of other frames at preset time intervals to obtain the first number of initial texture images;

[0047] The time interval between two adjacent frames in the first number of initial texture image frames exceeds the preset time interval.

[0048] Optionally, the initial image acquisition module includes:

[0049] A second acquisition submodule is configured to use the latest initial texture image stored in the designated position as the first frame, and acquire initial texture images of other frames according to a preset distance threshold to obtain the first number of initial texture images;

[0050] The distance between two adjacent frames of the first number of initial texture images exceeds the preset distance threshold.

[0051] Optionally, the device further includes an initial image generation module, configured to generate an initial texture image in a specified position; the initial image generation module includes:

[0052] The vehicle posture acquisition submodule is used to obtain the posture of the vehicle in each acquisition cycle;

[0053] A surround view image acquisition submodule is used to acquire a surround view image in each acquisition cycle; the surround view image is formed by stitching images acquired by at least four cameras installed on the vehicle;

[0054] A deformed image acquisition submodule is used to perform rendering and deformation processing on the surround view image to obtain a deformed texture image;

[0055] The initial image acquisition submodule is used to bind the posture of the vehicle and the deformed texture image to obtain the initial texture image.

[0056] Optionally, the deformed image acquisition submodule includes:

[0057] an intermediate image acquisition unit, configured to render the surround view image in a frame buffer manner to obtain an intermediate rendered image;

[0058] A deformed image acquisition unit is used to project the intermediate rendering image onto a preset bowl-shaped area for deformation processing to obtain the deformed texture image; wherein the shape of the bowl-shaped area matches the shape of the scene in which the vehicle is located.

[0059] Optionally, the real-time posture acquisition module includes:

[0060] A vehicle data acquisition submodule, configured to acquire inertial data collected by the vehicle's inertial sensor, the number of pulses collected by the vehicle's pulse sensor, the vehicle's previous position in a previous cycle, and a previous navigation angle;

[0061] A real-time posture calculation submodule is used to calculate the real-time posture of the vehicle in the current acquisition cycle based on a preset posture model, the inertial data, the pulse data, the previous position and the previous navigation angle; the real-time posture includes the current position and current navigation angle of the vehicle.

[0062] Optionally, the first image acquisition module includes:

[0063] The pose acquisition submodule is used to obtain the pose of the initial texture image of each frame;

[0064] A transformation matrix acquisition submodule, used to obtain the transformation matrix between the pose of the initial texture image of each frame and the real-time pose;

[0065] A target matrix acquisition submodule is used to obtain the rotation submatrix and translation submatrix of the transformation matrix of the initial texture image of each frame, and adjust the rotation submatrix and the translation submatrix to a target transformation matrix;

[0066] The first image acquisition submodule is used to render the initial texture image corresponding to the target transformation matrix, and obtain the first texture image corresponding to each frame of the initial texture image.

[0067] Optionally, the second image acquisition module includes:

[0068] A first image conversion submodule is configured to convert a first texture image corresponding to each frame of the initial texture image to the coordinate system of the vehicle, so that the bottom area of ​​the vehicle in the first texture image matches the area where the bottom of the vehicle passes;

[0069] The bottom image conversion submodule is used to obtain the bottom area image of the vehicle bottom area in each frame of the first texture image, obtain a first number of frames of bottom area images and use the bottom area images as the second texture image.

[0070] Optionally, the bottom image rendering submodule includes:

[0071] A color component acquisition unit is configured to acquire, for pixel points at the same position in the bottom area images of the first number of frames, a maximum value of each color component in the pixel points at the same position;

[0072] The color component updating unit is used to update the value of each color component to the corresponding maximum value to obtain a rendered image.

[0073] Optionally, the device further comprises:

[0074] The target image presentation module is used to present the target image in the bottom area of ​​the vehicle corresponding model.

[0075] According to a third aspect of an embodiment of the present disclosure, there is provided a vehicle, comprising: a vehicle body, and a memory and a processor provided on the vehicle body;

[0076] The memory is used to store a computer program executable by the processor;

[0077] The processor is configured to execute the computer program in the memory to implement the method according to the first aspect.

[0078] Optionally, it further includes at least 4 cameras; the at least 4 cameras are set at preset positions on the vehicle body to collect images in corresponding directions of the vehicle; the images collected by the at least 4 cameras are spliced ​​to obtain a surround view image, and the surround view image is used to obtain an initial texture image.

[0079] Optionally, an inertial sensor and a pulse sensor are further included; the inertial sensor is used to collect inertial data of the vehicle, and the pulse sensor is used to collect pulse data of the vehicle; the inertial data and the pulse data are used to calculate the posture of the vehicle.

[0080] Optionally, a positioning sensor is further included; the positioning sensor is used to collect the current position of the vehicle, and the current position is used to calculate the posture of the vehicle.

[0081] According to a fourth aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided. When an executable computer program in the storage medium is executed by a processor, the method described in the first aspect can be implemented.

[0082] The technical solutions provided by the embodiments of the present disclosure may have the following beneficial effects:

[0083] As can be seen from the above embodiments, the solution provided by the embodiments of the present disclosure can obtain the real-time posture of the vehicle and obtain the initial texture images of the first number of frames; then, based on the real-time posture and the initial texture images of each frame, the first texture image corresponding to the initial texture image of each frame is obtained; thereafter, the first texture image is converted to the coordinate system of the vehicle and the area corresponding to the bottom of the vehicle is obtained to obtain the second texture image; finally, the second texture images corresponding to the initial texture images of the first number of frames are mixed and rendered to obtain a target image representing the scene of the bottom area of ​​the vehicle. In this way, by obtaining the target image of the scene of the bottom area of ​​the vehicle, the present embodiment can monitor the road conditions at the bottom of the vehicle, expand the field of view, that is, reduce the blind spot of the field of view, which is conducive to improving the safety of unmanned vehicles.

[0084] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0085] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0086] Figure 1 The figure is a flowchart of an image processing method according to an exemplary embodiment.

[0087] Figure 2 The present invention is a flowchart showing a method for obtaining a real-time posture according to an exemplary embodiment.

[0088] Figure 3 The figure is a flowchart of obtaining an initial texture image according to an exemplary embodiment.

[0089] Figure 4 The flowchart of obtaining a first texture image is shown according to an exemplary embodiment.

[0090] Figure 5 The figure is a flowchart of obtaining a second texture image according to an exemplary embodiment.

[0091] Figure 6 is a block diagram of an image processing apparatus according to an exemplary embodiment.

[0092] Figure 7 is a block diagram of a vehicle according to an exemplary embodiment. DETAILED DESCRIPTION

[0093] Exemplary embodiments will be described in detail herein, with examples shown in the accompanying drawings. When the following description refers to the drawings, identical numbers in different drawings represent identical or similar elements, unless otherwise indicated. The exemplary embodiments described below do not represent all embodiments consistent with the present disclosure. Rather, they are merely examples of devices consistent with certain aspects of the present disclosure, as detailed in the appended claims. It should be noted that, unless there is a conflict, the features of the following embodiments and implementations may be combined with each other.

[0094] In order to solve the above technical problems, an embodiment of the present disclosure provides an image processing method that can be applied to a processor on a vehicle. The above processor can be a processor or controller of a vehicle's unmanned driving control system, which can be applied to scenarios with unmanned vehicles; the above processor can also be a processor or controller of a vehicle's assisted driving system, which can be applied to vehicles with assisted driving functions to replace the driver's driving in some scenarios. In other words, the above processor can at least have an image processing function, so as to obtain and provide corresponding target images for the unmanned driving control system or the driver, so as to expand the field of view and improve driving safety. For the convenience of description, the subsequent embodiments of the present disclosure all use the vehicle's processor as the execution subject to implement the above image processing method, but this does not constitute a limitation of the present disclosure.

[0095] Figure 1 FIG. 1 is a flow chart showing an image processing method according to an exemplary embodiment. Figure 1 , an image processing method, comprising steps 11 to 14.

[0096] In step 11, the real-time posture of the vehicle and a first number of frames of initial texture images are obtained.

[0097] In this embodiment, the vehicle's processor can obtain the vehicle's real-time posture, see Figure 2 , including step 21 and step 22.

[0098] In step 21 , the processor may obtain inertial data collected by an inertial sensor on the vehicle, pulse data collected by a pulse sensor of the vehicle, a previous position of the vehicle in a previous cycle, and a previous navigation angle.

[0099] Among them, the vehicle's inertial sensors may include accelerometers or gyroscopes, and their single, dual and / or three-axis combinations into inertial measurement units (IMUs) or attitude reference parameters (AHRS). The inertial sensor detects acceleration data in three spatial directions x, y, and z. The vehicle has a reference coordinate system (COG), and the inertial sensor has a coordinate system L. When the installation position of the inertial sensor on the vehicle body is known, the conversion relationship between the coordinate system L and the reference coordinate system COG can be determined according to the installation position of the inertial sensor and preset into the inertial sensor. Therefore, the inertial sensor can convert the detected acceleration data into the reference coordinate system in combination with the above-mentioned conversion relationship to obtain the vehicle's inertial data.

[0100] The vehicle's pulse sensor can include a magnetoelectric sensor, a Hall effect sensor, or a photoelectric sensor. Each time the wheel rotates, the pulse sensor generates a pulse. The pulse data from the pulse sensor can be used to detect the number of vehicle rotations. If the wheel's circumference is known, the vehicle's speed and other data can be determined.

[0101] The vehicle is also equipped with a positioning sensor, such as a Beidou positioning module, for collecting vehicle location data. In this embodiment, the positioning sensor collects location data according to a set period. Therefore, the period of this calculation can be regarded as the current period, and the period before the current period can be regarded as the previous period.

[0102] It is understood that after determining the data source, the processor can communicate with the corresponding sensor to obtain the corresponding data. For example, the processor communicates with an inertial sensor to obtain inertial data, or communicates with a pulse sensor to obtain pulse data. Of course, each sensor can store the collected data in a designated location, such as local memory, cache, or cloud, and the processor can read the data from the designated location. Ultimately, the processor can obtain inertial data, pulse data, previous position, and previous navigation angle.

[0103] In step 22, based on a preset posture model, the processor can calculate the real-time posture of the vehicle in the current acquisition cycle according to the inertial data, the pulse data, the previous position, and the previous navigation angle; the real-time posture includes the current position and the current navigation angle of the vehicle. The posture model can be expressed by the following expression (1):

[0104]

[0105] In expression (1), θ represents the navigation angle of the vehicle, x and y represent the horizontal and vertical coordinates of the vehicle respectively, ω represents the yaw angular velocity of the vehicle, and ω bis the yaw angular velocity of the vehicle at time t-1, Δt represents an acquisition cycle, ΔC represents the change in the pulse within a cycle, and K is a constant coefficient.

[0106] In this embodiment, the processor of the vehicle may obtain a first number of frames of initial texture images, where the initial texture images are pre-stored in a designated location, wherein the designated location stores a plurality of frames of initial texture images.

[0107] In one embodiment, the processor may designate the latest initial texture image in a position as the first frame, and select a first number of frame images in chronological order, thereby obtaining a first number of frames of initial texture images. The processing method is relatively simple and the processing efficiency is high.

[0108] In another embodiment, the processor may acquire a first number of image frames in an acquisition manner, including:

[0109] In one example, the time acquisition method is used, and the time data of each frame of the initial texture image is used in this method. The processor can use the latest initial texture image in the specified position as the first frame, and acquire the initial texture images of other frames according to the preset time interval to obtain the first number of frames of initial texture images. Among them, the time interval between two adjacent frames in the first number of frames of initial texture images exceeds the preset time interval. The range of the above preset time interval is 0.1s to 0.5s, which can be selected according to the specific scenario. In this way, the solution of this example acquires images collected at different positions through time intervals, thereby increasing the distinctiveness of the initial texture images, avoiding the initial texture images being too similar and affecting the subsequent effects, and can be applied to scenes with high and low vehicle speeds and is more effective in scenes with low vehicle speeds (such as less than 10km / h, which can be set).

[0110] In another example, the spatial sampling method is used to obtain the position data of each frame of the initial texture image. The processor can use the latest initial texture image in the specified position as the first frame, and obtain the initial texture images of other frames according to the preset distance threshold to obtain the first number of frames of initial texture images. Among them, the distance between two adjacent frames of the first number of frames of initial texture images exceeds the preset distance threshold. The range of the above-mentioned preset distance threshold is 1cm to 10cm, which can be selected according to the specific scene. In this way, the scheme of this example obtains images collected at different positions through the distance threshold, thereby increasing the distinctiveness of the initial texture image, avoiding the initial texture image being too similar and affecting the subsequent effect, and can be applied to scenes with high and low vehicle speeds and is more effective in scenes with high vehicle speeds (such as greater than 40km / h, which can be set).

[0111] In another example, a combination of temporal and spatial sampling is used. See the temporal and spatial approaches, which are not limited here. This example maximizes the distinctness of the initial texture image without losing key frames, making it suitable for various vehicle speed scenarios, i.e., with more relaxed speed requirements, and thus improving the accuracy of subsequent images.

[0112] In one embodiment, the processor may obtain the initial texture image and store it in a specified location, see Figure 3 , including steps 31 to 33.

[0113] In step 31, the processor can obtain the vehicle's posture in each acquisition cycle and obtain a surround view image for each acquisition cycle; the surround view image is composed of images acquired by at least four cameras installed on the vehicle. Figure 2 The contents of the illustrated embodiment will not be repeated here. The processor obtains the surround view image of each acquisition cycle, which means that the images acquired by at least 4 cameras in the same acquisition cycle are stitched together in sequence according to the order of the cameras (clockwise or counterclockwise). Taking 4 cameras set on a vehicle as an example, these 4 cameras are respectively facing the front, back, left and right of the vehicle, and the field of view of each camera can range from at least 90 degrees, so that the 4 cameras can cover a 360-degree range around the vehicle. In one example, the edges of the images acquired by two adjacent cameras of the 4 cameras are exactly the same edge, then the processor can stitch the 4 images together in a clockwise direction to obtain a surround view image. In another example, there is an overlapping part in the images acquired by two adjacent cameras of the 4 cameras; the processor can identify the overlapping part, and then stitch the 4 images together in a clockwise direction to obtain a surround view image.

[0114] In step 32, the processor may render and deform the surround view image to obtain a deformed texture image. In one example, the processor may render the surround view image using a frame buffer to obtain an intermediate rendered image. It is understood that in practical applications, processors typically render images using a frame buffer before displaying them. This step utilizes the frame buffer's ability to render images to render the surround view image, eliminating the need for a separate rendering function. The processor may then project the intermediate rendered image onto a pre-defined bowl-shaped area for deformation, obtaining a deformed texture image.

[0115] It should be noted that the bowl-shaped area is a reference model used in the deformation process, and the shape of the bowl-shaped area matches the shape of the scene the vehicle is in. The above-mentioned scene where the vehicle is located refers to a bowl-shaped area formed by at least four cameras capturing the surrounding scenery and the vehicle. The scenery captured by each camera forms the edge of the bowl and part of the bottom of the bowl, and the vehicle is located at the bottom of the bowl.

[0116] In step 33, the processor can bind the vehicle's position and the deformed texture image to obtain the initial texture image. In this step, the processor can bind the vehicle's position and the deformed texture image, ensuring a one-to-one correspondence between the vehicle's position and the deformed texture image within each acquisition cycle, thereby obtaining a single frame of initial texture image. The processor can then store the initial texture image at a designated location. Repeating steps 31 to 33, multiple frames of initial texture images can be stored at the designated location.

[0117] In step 12, a first texture image corresponding to each initial texture image of each frame is obtained according to the real-time posture and the initial texture image of each frame;

[0118] In this embodiment, the vehicle processor can obtain the first texture image corresponding to each frame of the initial texture image, see Figure 4 , including steps 41 to 44.

[0119] In step 41, the processor can obtain the pose of the initial texture image of each frame. The initial texture image includes the pose, so the processor can directly read it. In step 42, the processor can obtain the transformation matrix between the pose of the initial texture image of each frame and the real-time pose. In step 43, the processor can obtain the rotation submatrix and translation submatrix of the transformation matrix of the initial texture image of each frame, and adjust the rotation submatrix and the translation submatrix to the target transformation matrix. In step 44, the processor can render the initial texture image corresponding to the target transformation matrix to obtain the first texture image corresponding to the initial texture image of each frame. Considering that the transformation matrix of the initial texture image and the transformation matrix used for rendering have different formats, or that the same key content is located in different positions in the transformation matrix of the initial texture image and the transformation matrix used for rendering, steps 41 to 44 are added in this example to ensure that the content before and after rendering remains consistent, thereby improving rendering accuracy.

[0120] In step 13, the first texture image is converted to the coordinate system of the vehicle and the area corresponding to the bottom of the vehicle is obtained to obtain a second texture image;

[0121] In this embodiment, the vehicle processor can obtain the second texture image based on the first texture image, see Figure 5, including steps 51 to 52. In step 51, the processor can convert the first texture image corresponding to each frame of the initial texture image to the coordinate system where the vehicle is located (i.e., the reference coordinate system COG), so that the bottom area of ​​the vehicle in the first texture image matches the area where the bottom of the vehicle passes. In step 52, the processor can obtain the bottom area image of the bottom area of ​​the vehicle in each frame of the first texture image, obtain the bottom area image of a first number of frames and use the bottom area image as the second texture image. For example, the processor can obtain the coordinate data of the rectangular area of ​​the bottom of the vehicle. Since the first texture image is in the reference coordinate system, the bottom of the vehicle in the first texture image and the actual bottom of the vehicle are one-to-one corresponding, so the processor can cut out the bottom area image in the first texture image according to the coordinate data, and use the bottom area image as the second texture image.

[0122] In step 14, the second texture images corresponding to the first number of frames of initial texture images are mixed and rendered to obtain a target image representing the scene of the vehicle bottom area.

[0123] In this embodiment, the vehicle's processor can perform mixed rendering on the first number of frames and the second texture image to obtain a target image. Taking the MAX strategy as an example, for the pixel points at the same position in the bottom area image of the first number of frames, the processor can obtain the maximum value of each color component in the pixel points at the same position. The processor can then update the value of each color component to the corresponding maximum value to obtain the rendered image, namely the target image. The target image can have a uniform display brightness, avoiding the problem of uneven brightness caused by drawing multiple frames of images, which is conducive to improving the display effect.

[0124] In another embodiment, the processor may present the target image in a bottom area of ​​the corresponding vehicle model for user use.

[0125] At this point, the solution provided by the embodiment of the present disclosure can obtain the real-time posture of the vehicle and obtain the initial texture images of the first number of frames; then, based on the real-time posture and the initial texture images of each frame, the first texture image corresponding to the initial texture image of each frame is obtained; thereafter, the first texture image is converted to the coordinate system of the vehicle and the area corresponding to the bottom of the vehicle is obtained to obtain the second texture image; finally, the second texture images corresponding to the initial texture images of the first number of frames are mixed and rendered to obtain a target image representing the scene of the bottom area of ​​the vehicle. In this way, by obtaining the target image of the scene of the bottom area of ​​the vehicle, this embodiment can monitor the road conditions under the vehicle, expand the field of view, that is, reduce the blind spot of the field of view, which is conducive to improving the safety of unmanned vehicles.

[0126] Based on an image processing method provided in an embodiment of the present disclosure, an image processing device is also provided in an embodiment of the present disclosure. Figure 6, the device comprises:

[0127] Real-time posture acquisition module 61, used to obtain the real-time posture of the vehicle

[0128] An initial image acquisition module 62 is configured to acquire a first number of frames of initial texture images;

[0129] A first image acquisition module 63 is configured to acquire a first texture image corresponding to each initial texture image frame according to the real-time posture and each initial texture image frame;

[0130] A second image acquisition module 64 is configured to convert the first texture image into the coordinate system of the vehicle and acquire an area corresponding to the bottom of the vehicle to obtain a second texture image;

[0131] The target image acquisition module 65 is configured to perform mixed rendering on the second texture images corresponding to the first number of frames of initial texture images to obtain a target image representing the scene of the bottom area of ​​the vehicle.

[0132] In one embodiment, the initial image acquisition module includes:

[0133] A first acquisition submodule is configured to take the latest initial texture image at a specified position as a first frame, and acquire initial texture images of other frames at preset time intervals to obtain the first number of initial texture images;

[0134] The time interval between two adjacent frames in the first number of initial texture image frames exceeds the preset time interval.

[0135] In one embodiment, the initial image acquisition module includes:

[0136] A second acquisition submodule is configured to use the latest initial texture image stored in the designated position as the first frame, and acquire initial texture images of other frames according to a preset distance threshold to obtain the first number of initial texture images;

[0137] The distance between two adjacent frames of the first number of initial texture images exceeds the preset distance threshold.

[0138] In one embodiment, the apparatus further includes an initial image generation module for generating an initial texture image at a specified position; the initial image generation module includes:

[0139] The vehicle posture acquisition submodule is used to obtain the posture of the vehicle in each acquisition cycle;

[0140] A surround view image acquisition submodule is used to acquire a surround view image in each acquisition cycle; the surround view image is formed by stitching images acquired by at least four cameras installed on the vehicle;

[0141] A deformed image acquisition submodule is used to perform rendering and deformation processing on the surround view image to obtain a deformed texture image;

[0142] The initial image acquisition submodule is used to bind the posture of the vehicle and the deformed texture image to obtain the initial texture image.

[0143] In one embodiment, the deformed image acquisition submodule includes:

[0144] an intermediate image acquisition unit, configured to render the surround view image in a frame buffer manner to obtain an intermediate rendered image;

[0145] A deformed image acquisition unit is used to project the intermediate rendering image onto a preset bowl-shaped area for deformation processing to obtain the deformed texture image; wherein the shape of the bowl-shaped area matches the shape of the scene in which the vehicle is located.

[0146] In one embodiment, the real-time posture acquisition module includes:

[0147] A vehicle data acquisition submodule, configured to acquire inertial data collected by the vehicle's inertial sensor, the number of pulses collected by the vehicle's pulse sensor, the vehicle's previous position in a previous cycle, and a previous navigation angle;

[0148] A real-time posture calculation submodule is used to calculate the real-time posture of the vehicle in the current acquisition cycle based on a preset posture model, the inertial data, the pulse data, the previous position and the previous navigation angle; the real-time posture includes the current position and current navigation angle of the vehicle.

[0149] In one embodiment, the first image acquisition module includes:

[0150] The pose acquisition submodule is used to obtain the pose of the initial texture image of each frame;

[0151] A transformation matrix acquisition submodule, used to obtain the transformation matrix between the pose of the initial texture image of each frame and the real-time pose;

[0152] A target matrix acquisition submodule is used to obtain the rotation submatrix and translation submatrix of the transformation matrix of the initial texture image of each frame, and adjust the rotation submatrix and the translation submatrix to a target transformation matrix;

[0153] The first image acquisition submodule is used to render the initial texture image corresponding to the target transformation matrix, and obtain the first texture image corresponding to each frame of the initial texture image.

[0154] In one embodiment, the second image acquisition module includes:

[0155] A first image conversion submodule is configured to convert a first texture image corresponding to each frame of the initial texture image to the coordinate system of the vehicle, so that the bottom area of ​​the vehicle in the first texture image matches the area where the bottom of the vehicle passes;

[0156] The bottom image conversion submodule is used to obtain the bottom area image of the vehicle bottom area in each frame of the first texture image, obtain a first number of frames of bottom area images and use the bottom area images as the second texture image.

[0157] In one embodiment, the bottom image rendering submodule includes:

[0158] A color component acquisition unit is configured to acquire, for pixel points at the same position in the bottom area images of the first number of frames, a maximum value of each color component in the pixel points at the same position;

[0159] The color component updating unit is used to update the value of each color component to the corresponding maximum value to obtain a rendered image.

[0160] In one embodiment, the apparatus further comprises:

[0161] The target image presentation module is used to present the target image in the bottom area of ​​the vehicle corresponding model.

[0162] It should be noted that the apparatus shown in this embodiment matches the contents of the method embodiment, and reference may be made to the contents of the above method embodiment, which will not be repeated here.

[0163] The present disclosure also provides a vehicle, see Figure 7 , comprising: a vehicle body and a memory and a processor provided on the vehicle body;

[0164] The memory is used to store a computer program executable by the processor;

[0165] The processor is used to execute the computer program in the memory to implement the following Figures 1 to 5 The method shown.

[0166] In one embodiment, the above-mentioned vehicle also includes at least 4 cameras; the at least 4 cameras are set at preset positions on the vehicle body to capture images in corresponding directions of the vehicle; the images captured by the at least 4 cameras are spliced ​​to obtain a surround view image, and the surround view image is used to obtain an initial texture image.

[0167] In one embodiment, the above-mentioned vehicle also includes an inertial sensor and a pulse sensor; the inertial sensor is used to collect inertial data of the vehicle, and the pulse sensor is used to collect pulse data of the vehicle; the inertial data and the pulse data are used to calculate the posture of the vehicle.

[0168] In one embodiment, the above-mentioned vehicle further includes a positioning sensor; the positioning sensor is used to collect the current position of the vehicle, and the current position is used to calculate the posture of the vehicle.

[0169] It should be noted that the vehicle shown in this embodiment matches the content of the method embodiment. You can refer to the content of the above method embodiment and will not repeat it here.

[0170] In an exemplary embodiment, a computer-readable storage medium is also provided, such as a memory including instructions, and the above-mentioned executable computer program can be executed by a processor. The readable storage medium can be ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, optical data storage device, etc.

[0171] Other embodiments of the present disclosure will readily occur to those skilled in the art after considering the specification and practicing the disclosure herein. This disclosure is intended to cover any variations, uses, or adaptations that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The description and examples are to be considered as exemplary only, with the true scope and spirit of the present disclosure being indicated by the following claims.

[0172] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes can be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. An image processing method, characterized in that: The method comprises: Obtaining the real-time position and posture of the vehicle and obtaining a first number of frames of initial texture images; Acquire a first texture image corresponding to each initial texture image of each frame according to the real-time posture and the initial texture image of each frame; Converting the first texture image to the coordinate system of the vehicle and acquiring an area corresponding to the bottom of the vehicle to obtain a second texture image; Performing mixed rendering on the second texture images corresponding to the first number of frames of initial texture images to obtain a target image representing the scene of the bottom area of ​​the vehicle; Acquiring a first texture image corresponding to each initial texture image of each frame according to the real-time posture and the initial texture image of each frame, including: Binding the vehicle's posture and the deformed texture image to obtain an initial texture image; wherein the deformed texture image is obtained by rendering and deforming the vehicle's surround view image; The initial texture image is rendered according to the posture of the initial texture image and the real-time posture to obtain a first texture image corresponding to each frame of the initial texture image.

2. The method according to claim 1, characterized in that Obtaining a first number of frames of initial texture images, including: The latest initial texture image in the designated position is taken as the first frame, and initial texture images of other frames are acquired at preset time intervals to obtain the first number of initial texture images.

3. The method according to claim 1, characterized in that Obtaining a first number of frames of initial texture images, including: The latest initial texture image stored in the designated position is used as the first frame, and initial texture images of other frames are obtained through a preset distance threshold to obtain the first number of initial texture images.

4. The method according to claim 2, characterized in that The method further includes the step of generating an initial texture image in a specified position, specifically comprising: Obtaining the position and posture of the vehicle in each acquisition cycle, and obtaining a surround view image in each acquisition cycle; The surround view image is formed by stitching images collected by at least four cameras installed on the vehicle; Performing rendering and deformation processing on the surround view image to obtain a deformed texture image; The posture of the vehicle and the deformed texture image are bound to obtain the initial texture image.

5. The method according to claim 4, characterized in that Rendering and deforming the surround view image include: Rendering the surround view image in a frame buffer manner to obtain an intermediate rendered image; The intermediate rendering image is projected onto a preset bowl-shaped area for deformation processing to obtain the deformed texture image; wherein the shape of the bowl-shaped area matches the shape of the scene in which the vehicle is located.

6. The method according to claim 1, characterized in that Get the real-time position of the vehicle, including: Acquiring inertial data collected by an inertial sensor of the vehicle, the number of pulses collected by a pulse sensor of the vehicle, a previous position of the vehicle in a previous cycle, and a previous navigation angle; Based on a preset posture model, the real-time posture of the vehicle in the current acquisition cycle is calculated according to the inertial data, the pulse data, the previous position and the previous navigation angle; the real-time posture includes the current position and current navigation angle of the vehicle.

7. The method according to claim 1, characterized in that Acquiring a first texture image corresponding to each initial texture image of each frame according to the real-time posture and the initial texture image of each frame, including: Get the pose of the initial texture image of each frame; Obtaining a transformation matrix between the pose of each frame of the initial texture image and the real-time pose; Obtaining a rotation submatrix and a translation submatrix of a transformation matrix of an initial texture image of each frame and adjusting the rotation submatrix and the translation submatrix to a target transformation matrix; Rendering the initial texture image corresponding to the target transformation matrix to obtain a first texture image corresponding to each frame of the initial texture image.

8. The method according to claim 1, characterized in that Converting the first texture image to the coordinate system of the vehicle and acquiring an area corresponding to the bottom of the vehicle to obtain a second texture image includes: Converting a first texture image corresponding to each frame of the initial texture image to the coordinate system of the vehicle so that the bottom area of ​​the vehicle in the first texture image matches the area where the bottom of the vehicle passes; A bottom region image of the bottom region of the vehicle in each frame of the first texture image is acquired to obtain a first number of frames of bottom region images and use the bottom region images as the second texture image.

9. The method according to claim 8, characterized in that Performing mixed rendering on the bottom area images of the first number of frames to obtain a target image representing the scene of the bottom area of ​​the vehicle includes: For pixel points at the same position of the bottom area images of the first number of frames, obtaining the maximum value of each color component of the pixel points at the same position; The values ​​of the color components are updated to corresponding maximum values ​​to obtain a rendered image, and the rendered image is used as a target image representing the scene in the bottom area of ​​the vehicle.

10. The method according to claim 1, characterized in that The method further comprises: The target image is presented in a bottom area of ​​the vehicle corresponding model.

11. An image processing device, characterized in that: The device comprises: Real-time posture acquisition module, used to obtain the real-time posture of the vehicle; An initial image acquisition module, configured to acquire a first number of frames of initial texture images; A first image acquisition module is configured to acquire a first texture image corresponding to each initial texture image frame based on the real-time posture and the initial texture image frame; acquiring the first texture image corresponding to each initial texture image frame based on the real-time posture and the initial texture image frame comprises: binding the posture of the vehicle and the deformed texture image to obtain the initial texture image; rendering the initial texture image based on the posture of the initial texture image and the real-time posture to obtain the first texture image corresponding to each initial texture image frame; wherein the deformed texture image is obtained by rendering and deforming the surround view image of the vehicle; A second image acquisition module is used to convert the first texture image to the coordinate system of the vehicle and acquire an area corresponding to the bottom of the vehicle to obtain a second texture image; The target image acquisition module is used to perform mixed rendering on the second texture images corresponding to the first number of frames of initial texture images to obtain a target image representing the scene of the bottom area of ​​the vehicle.

12. The device according to claim 11, characterized in that The initial image acquisition module includes: The first acquisition submodule is configured to take the latest initial texture image in the designated position as the first frame, and acquire the initial texture images of other frames at preset time intervals to obtain the first number of initial texture images.

13. The device according to claim 11 or 12, characterized in that The initial image acquisition module includes: The second acquisition submodule is configured to use the latest initial texture image stored in the designated position as the first frame, and acquire initial texture images of other frames through a preset distance threshold to obtain the first number of initial texture images.

14. The device according to claim 12, characterized in that The device further includes an initial image generation module for generating an initial texture image in a specified position; the initial image generation module includes: A vehicle posture acquisition submodule is used to acquire the posture of the vehicle in each acquisition cycle; a surround view image acquisition submodule is used to acquire a surround view image in each acquisition cycle; the surround view image is composed of images acquired by at least four cameras installed on the vehicle; A deformed image acquisition submodule is used to perform rendering and deformation processing on the surround view image to obtain a deformed texture image; The initial image acquisition submodule is used to bind the posture of the vehicle and the deformed texture image to obtain the initial texture image.

15. The device according to claim 14, characterized in that The deformed image acquisition submodule includes: an intermediate image acquisition unit, configured to render the surround view image in a frame buffer manner to obtain an intermediate rendered image; A deformed image acquisition unit is used to project the intermediate rendering image onto a preset bowl-shaped area for deformation processing to obtain the deformed texture image; wherein the shape of the bowl-shaped area matches the shape of the scene in which the vehicle is located.

16. The device according to claim 11, characterized in that The real-time posture acquisition module includes: A vehicle data acquisition submodule, configured to acquire inertial data collected by the vehicle's inertial sensor, the number of pulses collected by the vehicle's pulse sensor, the vehicle's previous position in a previous cycle, and a previous navigation angle; A real-time posture calculation submodule is used to calculate the real-time posture of the vehicle in the current acquisition cycle based on a preset posture model, the inertial data, the pulse data, the previous position and the previous navigation angle; the real-time posture includes the current position and current navigation angle of the vehicle.

17. The device according to claim 11, characterized in that The first image acquisition module includes: The pose acquisition submodule is used to obtain the pose of the initial texture image of each frame; A transformation matrix acquisition submodule, used to obtain the transformation matrix between the pose of the initial texture image of each frame and the real-time pose; A target matrix acquisition submodule is used to obtain the rotation submatrix and translation submatrix of the transformation matrix of the initial texture image of each frame, and adjust the rotation submatrix and the translation submatrix to a target transformation matrix; The first image acquisition submodule is used to render the initial texture image corresponding to the target transformation matrix, and obtain the first texture image corresponding to each frame of the initial texture image.

18. The device according to claim 11, characterized in that The second image acquisition module includes: A first image conversion submodule is configured to convert a first texture image corresponding to each frame of the initial texture image to the coordinate system of the vehicle, so that the bottom area of ​​the vehicle in the first texture image matches the area where the bottom of the vehicle passes; The bottom image conversion submodule is used to obtain the bottom area image of the vehicle bottom area in each frame of the first texture image, obtain a first number of frames of bottom area images and use the bottom area images as the second texture image.

19. The device according to claim 18, characterized in that The bottom image rendering submodule includes: A color component acquisition unit is used to obtain the maximum value of each color component in the pixel points at the same position of the bottom area image of the first number of frames; a color component update unit is used to update the value of each color component to the corresponding maximum value to obtain a rendered image.

20. The device according to claim 11, characterized in that The device further includes: a target image presenting module, configured to present the target image in a bottom area of ​​the vehicle corresponding model.

21. A vehicle, characterized in that: include: A vehicle body and a memory and a processor provided on the vehicle body; The memory is used to store a computer program executable by the processor; The processor is configured to execute the computer program in the memory to implement the method according to any one of claims 1 to 10.

22. The vehicle according to claim 21, characterized in that It also includes at least 4 cameras; the at least 4 cameras are set at preset positions on the vehicle body to collect images in corresponding directions of the vehicle; the images collected by the at least 4 cameras are spliced ​​to obtain a surround view image, and the surround view image is used to obtain an initial texture image.

23. The vehicle according to claim 21, characterized in that It also includes an inertial sensor and a pulse sensor; the inertial sensor is used to collect inertial data of the vehicle, and the pulse sensor is used to collect pulse data of the vehicle; the inertial data and the pulse data are used to calculate the position and posture of the vehicle.

24. The vehicle according to claim 21, characterized in that It also includes a positioning sensor; the positioning sensor is used to collect the current position of the vehicle, and the current position is used to calculate the posture of the vehicle.

25. A computer-readable storage medium, characterized in that When the executable computer program in the storage medium is executed by a processor, the method according to any one of claims 1 to 10 can be implemented.

Images