Vehicle-mounted image processing method and device based on directed vector, vehicle domain controller and vehicle

Abstract

The invention discloses a directed vector-based vehicle-mounted image processing method and device, a vehicle domain controller and a vehicle, and the method comprises the steps: determining an overlapped region image according to an aerial view corresponding to an obtained vehicle-mounted look-around view; on the basis of a solving result of an error equation constructed for the overlapped area image, brightness equalization processing is carried out on the overlapped area image, and a primary uniform-color image is obtained; performing vector conversion processing on the brightness of the overlapped area of the primary uniform color image to obtain each brightness vector group; determining a real-time brightness result of the overlapped area image based on the brightness vector group and a preset brightness threshold value; and executing a brightness adjustment strategy matched with the real-time brightness result. According to the vehicle-mounted image processing method provided by the embodiment of the invention, the brightness adjustment strategy is scheduled through the real-time brightness result, the real-time response capability of the vehicle-mounted surround view system is improved, and continuous surround view images with balanced brightness can be rapidly adjusted and provided under the condition that the light condition is rapidly changed.

Description

Technical Field

[0001] The present invention relates to the field of intelligent driving technology, and in particular to a vehicle-mounted image processing method and device based on directed vectors, a vehicle domain controller and a vehicle. Background Art

[0002] In the intelligent driving industry, real-time output of the in-vehicle surround view is one of the key technologies to improve driving safety and driving experience. The in-vehicle panoramic imaging system integrates multiple cameras to achieve a 360-degree all-round view display, providing the driver with a panoramic view.

[0003] However, when the car is in a scene with drastic changes in light brightness, such as entering or exiting a tunnel or bridge, or when individual cameras are illuminated by strong light or blocked, the surround view may have uneven brightness, overexposure or darkness, which may affect the driver's visual experience and driving safety. Summary of the invention

[0004] The present invention provides a vehicle-mounted image processing method and device based on directed vectors, a vehicle domain controller and a vehicle, so as to solve the technical problem of uneven brightness and achieve the effect of outputting a surround view with balanced brightness under drastically changing lighting conditions.

[0005] In order to solve the above technical problems, an embodiment of the present invention provides a vehicle-mounted image processing method based on directed vectors, comprising: determining an overlapped area image according to a bird's-eye view corresponding to an acquired vehicle-mounted surround view;

[0006] Based on the solution result of the error equation constructed for the overlapped area image, the overlapped area image is subjected to brightness equalization processing to obtain a primary uniform color image;

[0007] Performing vector conversion processing on the brightness of the overlapping areas of the primary uniform color image to obtain each brightness vector group;

[0008] Determine the real-time brightness result of the overlapped area image based on the brightness vector group and a preset brightness threshold;

[0009] A brightness adjustment strategy matching the real-time brightness result is executed.

[0010] As one of the preferred solutions, the method of determining the overlapped area image according to the bird's-eye view corresponding to the acquired vehicle-mounted surround view includes:

[0011] Obtain each vehicle-mounted surround view collected by the vehicle-mounted camera;

[0012] Each of the vehicle-mounted surround views is preprocessed, and each of the preprocessed vehicle-mounted surround views is analyzed based on the selected field of view direction to determine an overlapped area image.

[0013] As one of the preferred solutions, the brightness adjustment strategy at least includes an over-brightness adjustment strategy and an over-darkness adjustment strategy;

[0014] The executing a brightness adjustment strategy matching the real-time brightness result includes:

[0015] Determining a first brightness threshold interval corresponding to the overbrightness adjustment strategy based on a first camera control instruction matching the first brightness threshold interval; and,

[0016] A second brightness threshold interval corresponding to the over-darkness adjustment strategy is determined based on a second camera control instruction that matches the second brightness threshold interval.

[0017] As one of the preferred solutions, the error equation is:

[0018]

[0019] in, and Represent the adjustment coefficients of image i and image j, channel∈(R,G,B), σ N represents the standard deviation of color error, σ g represents the standard deviation of the gain parameter, and Mean represents the average brightness of the overlapping area of ​​image i and image j.

[0020] As one of the preferred solutions, after executing the brightness adjustment strategy, the vehicle-mounted image processing method based on directed vectors further includes:

[0021] The Gauss-Seidel iteration method is used to solve the Poisson equation to perform Poisson fusion processing on the acquired real-time images to obtain the fused overlap area image.

[0022] As one of the preferred solutions, the vehicle-mounted image processing method based on directed vectors also includes:

[0023] Determine a non-overlapping area image based on the vehicle-mounted surround view and the overlapping area image;

[0024] The non-overlapping area and the fused overlapping area images are stitched together using a surround view stitching algorithm to generate a vehicle-mounted surround view to be displayed.

[0025] As one of the preferred solutions, the real-time brightness result is displayed in the form of a tree diagram.

[0026] Another embodiment of the present invention provides a vehicle-mounted image processing device based on a directed vector, comprising:

[0027] A preprocessing module, used for determining an overlapped area image according to the bird's-eye view corresponding to the acquired vehicle-mounted surround view;

[0028] A primary color equalization module, configured to perform brightness equalization processing on the overlapped area image based on the solution result of the error equation constructed for the overlapped area image, so as to obtain a primary color equalization image;

[0029] A conversion module, used for performing vector conversion processing on the brightness of the overlapping areas of the primary uniform color image to obtain each brightness vector group;

[0030] A processing module, used for determining a real-time brightness result of the overlapped area image based on the brightness vector group and a preset brightness threshold;

[0031] An execution module is used to execute a brightness adjustment strategy that matches the real-time brightness result.

[0032] Another embodiment of the present invention provides a vehicle domain controller, comprising a processor, a memory and a communication bus, wherein the processor and the memory communicate with an external vehicle-mounted camera via the communication bus;

[0033] The memory is used to store computer programs;

[0034] The processor is used to implement any of the above-mentioned vehicle-mounted image processing methods based on directed vectors when executing the computer program stored in the memory.

[0035] Another embodiment of the present invention provides a vehicle having a computer program, wherein when the vehicle executes the computer program, the vehicle implements any one of the above-mentioned vehicle-mounted image processing methods based on directed vectors.

[0036] Compared with the prior art, the embodiments of the present invention have the following advantages:

[0037] (1) The present invention covers all possible brightness distribution situations with brightness vector groups, and schedules brightness adjustment strategies through real-time brightness results, thereby improving the real-time response capability of the vehicle surround view system;

[0038] (2) When light conditions change rapidly, for example, when entering or exiting a tunnel or encountering strong light, the present invention can quickly adjust and provide a continuous and uniformly bright surround view image to better meet the needs in a dynamic environment. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 It is a schematic flow chart of a vehicle-mounted image processing method based on directed vectors in one embodiment of the present invention;

[0040] Figure 2 is a bird's-eye view of a vehicle-mounted surround view in one embodiment of the present invention;

[0041] Figure 3 is a schematic diagram of an image of an overlapped area of ​​a bird's-eye view of a vehicle-mounted surround view in one embodiment of the present invention;

[0042] Figure 4 is a directed acyclic graph constructed from overlapping regions in one embodiment of the present invention;

[0043] Figure 5 is a tree diagram showing real-time brightness results in one embodiment of the present invention;

[0044] Figure 6 It is a schematic diagram of a vehicle-mounted image processing device based on directed vectors in one embodiment of the present invention.

[0045] Reference numerals:

[0046] Among them, 11, pre-processing module; 12, initial color uniformity module; 13, conversion module; 14, processing module; 15, execution module. DETAILED DESCRIPTION

[0047] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. The purpose of providing these embodiments is to make the disclosure of the present invention more thorough and comprehensive. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0048] In the description of this application, the terms "first", "second", "third", etc. are used for descriptive purposes only and are not to be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first", "second", "third", etc. may explicitly or implicitly include one or more of the feature. In the description of this application, unless otherwise specified, "plurality" means two or more.

[0049] In the description of the present application, it should be noted that, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be a connection between the two elements. The terms "vertical", "horizontal", "left", "right", "upper", "lower" and similar expressions used herein are only for illustrative purposes, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention. The term "and / or" used herein includes any and all combinations of one or more related listed items. For those of ordinary skill in the art, the specific meanings of the above terms in this application can be understood according to specific circumstances.

[0050] In the description of this application, it should be noted that, unless otherwise defined, all technical and scientific terms used in the present invention have the same meaning as those commonly understood by those skilled in the art. The terms used in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. For those of ordinary skill in the art, the specific meanings of the above terms in this application can be understood by specific circumstances.

[0051] An embodiment of the present invention provides a vehicle-mounted image processing method based on a directed vector. For details, see Figure 1 , Figure 1 The figure is a schematic flow chart of a vehicle-mounted image processing method based on a directed vector in one embodiment of the present invention, comprising:

[0052] S1: determining an overlapped area image according to the bird's-eye view corresponding to the acquired vehicle-mounted surround view;

[0053] S2: Based on the solution result of the error equation constructed for the overlapped area image, the overlapped area image is subjected to brightness equalization processing to obtain a primary uniform color image;

[0054] S3: performing vector conversion processing on the brightness of the overlapping areas of the primary uniform color image to obtain each brightness vector group;

[0055] S4: Determine the real-time brightness result of the overlapped area image based on the brightness vector group and a preset brightness threshold;

[0056] S5: Execute a brightness adjustment strategy that matches the real-time brightness result.

[0057] In step S1, the vehicle-mounted surround view collected by the vehicle-mounted camera is obtained, each vehicle-mounted surround view is preprocessed, and each preprocessed vehicle-mounted surround view is analyzed based on the selected field of view direction to determine the overlap area image, and the image can be converted into a bird's-eye view of the corresponding format through projection transformation. The overlap area of ​​the four bird's-eye views is determined according to the actual field of view, such as Figure 2 As shown, Figure 2 It is a bird's-eye view of the vehicle-mounted surround view in one embodiment of the present invention, wherein the left front overlapping area is denoted as O1, the right front overlapping area is denoted as O2, the left rear overlapping area is denoted as O3, and the right rear overlapping area is denoted as O4.

[0058] Since each bird's-eye view contains two overlapping areas, the four bird's-eye views have eight overlapping area images. For example, the left side in the front view and the front side in the left view are overlapping areas, and the right side in the front view and the front side in the right view are overlapping areas. Figure 3 As shown, Figure 3 It is a schematic diagram of an image of an overlapped area of ​​a bird's-eye view of a vehicle-mounted surround view in one embodiment of the present invention.

[0059] Therefore, each overlapping area has two overlapping area images, and the two overlapping area images can construct a set of error equations.

[0060] The error equation is

[0061]

[0062] in, and Represent the adjustment coefficients of image i and image j, channel∈(R,G,B), σ N represents the standard deviation of color error, σ g represents the standard deviation of the gain parameter, σ N Set to 10.0, σ g Set to 0.1, Mean represents the average brightness of the overlapping area of ​​image i and image j.

[0063] It should be noted that Mean ij and Mean ji All belong to the same overlapping area, but Mean ij is the area in image i that overlaps with image j, Mean ji is the area in image j that overlaps with image i, Mean ij and Mean ji The values ​​are generally different.

[0064] The adjustment coefficient can be solved by using SVD singular value decomposition method, Gaussian iteration method or least square method, and the bird's-eye view image can be corrected by using the adjustment coefficient. By solving the constructed error equation, the color brightness error of the overlapped area image of the bird's-eye view image is minimized. The error equation links the four bird's-eye views, realizes the initial uniform color based on global color brightness, and obtains the initial uniform color image.

[0065] The brightness of the overlapped area of ​​the first uniform color image is vector-converted, and the adjacent overlapped areas are used to construct directed vectors V1, V2, and V3. For example, V1 indicates that it points from the left front to the right front, V2 indicates that it points from the right front to the left back, and V3 indicates that it points from the left back to the right back. Of course, this example is only one of the schematic diagrams, and the direction of the directed vector can also be other directions, such as the opposite direction to the above example. The brightness of the overlapped area is constructed into a directed acyclic graph according to the order of the directed vectors, such as Figure 4 As shown, Figure 4 It is a directed acyclic graph constructed by overlapping areas in one embodiment of the present invention.

[0066] Assume that the threshold of brightness difference acceptable to human eyes is T, and the difference between adjacent overlapping areas is D. If the absolute value of D is greater than T and is a positive number, it is represented by "1"; if the absolute value of D is less than T, it is represented by "0"; if the absolute value of D is greater than T and is a negative number, it is represented by "-1". According to the order and direction of the directed vectors V1, V2, and V3, the brightness difference values ​​of the overlapping areas are calculated in turn, and finally a real-time brightness result consisting of 1, 0, and -1 is obtained, and the real-time brightness result is displayed in the form of a tree diagram, as shown in the figure. Figure 5 As shown, Figure 5 A tree diagram showing real-time brightness results in one embodiment of the present invention.

[0067] When the brightness difference between O1 and O2 is 1, the real-time brightness results shown in the following table can be obtained.

[0068]

[0069] When the brightness difference value between O1 and O2 is 1 or -1, different real-time brightness results can be obtained, which will not be described here one by one.

[0070] According to all real-time brightness results, they can be summarized into 5 categories of brightness distribution:

[0071] (1) A single overlapping area is too bright or too dark;

[0072] (2) A single bird's-eye view is too bright or too dark;

[0073] (3) The overlapping areas of the diagonals are both too bright, or both too dark, or one is too bright and the other is too dark;

[0074] (4) Bird’s eye view from the opposite side Figure 1Too dark, too bright;

[0075] (5) A single overlapping area is too bright and one side of the bird's-eye view is too dark, or a single overlapping area is too dark and one side of the bird's-eye view is too bright.

[0076] According to the above real-time brightness results, the brightness adjustment strategy is as follows:

[0077] (1) The brightness of two adjacent bird's-eye views in the overlapping area that is too bright or too dark needs to be adjusted, and the brightness target value of the bird's-eye view is the average brightness value of the overlapping area;

[0078] (2) The bird's-eye view that is too bright or too dark needs to be adjusted. The brightness target value of the bird's-eye view is the average brightness of the overlapping area of ​​the bird's-eye view on the opposite side;

[0079] (3) All bird's-eye view images need to be adjusted, and the brightness target value of the bird's-eye view image is the average brightness value of all overlapping areas that are not too bright or too dark;

[0080] (4) The bird's-eye view that is too dark or too bright needs to be adjusted. The brightness target value of the bird's-eye view is the average brightness of all overlapping areas;

[0081] (5) All bird's-eye view images need to be adjusted, and the target brightness value of the bird's-eye view image is the average brightness of all overlapping areas.

[0082] That is, the brightness adjustment strategy includes at least an over-brightness adjustment strategy and an over-darkness adjustment strategy, which include:

[0083] Determine a first brightness threshold interval corresponding to the overbrightness adjustment strategy, and execute a first camera control instruction matching the first brightness threshold interval; determine a second brightness threshold interval corresponding to the overdarkness adjustment strategy, and execute a second camera control instruction matching the second brightness threshold interval.

[0084] After executing the brightness adjustment strategy, the acquired real-time images are subjected to Poisson fusion processing using the Gauss-Seidel iteration method to obtain the fused overlapped area image. At the same time, the non-overlapped area image is determined based on the vehicle-mounted surround view and the overlapped area image. The surround view stitching algorithm is used to stitch the non-overlapped area and the fused overlapped area image to generate the vehicle-mounted surround view to be displayed.

[0085] An embodiment of the present invention further provides a vehicle-mounted image processing device based on a directed vector. For details, see Figure 6 , Figure 6 The figure is a schematic flow chart of a vehicle-mounted image processing method based on a directed vector in one embodiment of the present invention, comprising:

[0086] A preprocessing module 11, used to determine an overlapped area image according to the bird's-eye view corresponding to the acquired vehicle-mounted surround view;

[0087] The primary color-equalizing module 12 is used to perform brightness equalization processing on the overlapped area images based on the solution result of the error equation constructed for the overlapped area images to obtain the primary color-equalizing image;

[0088] The conversion module 13 is used to perform vector conversion processing on the brightness of the overlapping areas of the primary uniform color image to obtain each brightness vector group;

[0089] The processing module 14 is used to determine the real-time brightness result of the overlapped area image based on the brightness vector group and the preset brightness threshold;

[0090] The execution module 15 is used to execute a brightness adjustment strategy that matches the real-time brightness result.

[0091] Compared with the prior art, the embodiments of the present invention have the following advantages:

[0092] (1) The present invention covers all possible brightness distribution situations with brightness vector groups, and schedules brightness adjustment strategies through real-time brightness results, thereby improving the real-time response capability of the vehicle surround view system;

[0093] (2) When light conditions change rapidly, for example, when entering or exiting a tunnel or encountering strong light, the present invention can quickly adjust and provide a continuous and uniformly bright surround view image to better meet the needs in a dynamic environment.

[0094] The above-mentioned embodiments only express several implementation methods of the present invention, and the description thereof is relatively specific and detailed, but it cannot be understood as limiting the scope of the patent of the present invention. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, which all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.

Claims

1. A vehicle-mounted image processing method based on directed vectors, characterized in that: include: Determine an overlapped area image according to the bird's-eye view corresponding to the acquired vehicle-mounted surround view; Based on the solution result of the error equation constructed for the overlapped area image, the overlapped area image is subjected to brightness equalization processing to obtain a primary uniform color image; Performing vector conversion processing on the brightness of the overlapping areas of the primary uniform color image to obtain each brightness vector group; Determine the real-time brightness result of the overlapped area image based on the brightness vector group and a preset brightness threshold; A brightness adjustment strategy matching the real-time brightness result is executed.

2. The vehicle-mounted image processing method based on directed vectors according to claim 1, characterized in that: The step of determining the overlapped area image according to the bird's-eye view corresponding to the acquired vehicle-mounted surround view includes: Obtain each vehicle-mounted surround view collected by the vehicle-mounted camera; Each of the vehicle-mounted surround views is preprocessed, and each of the preprocessed vehicle-mounted surround views is analyzed based on the selected field of view direction to determine an overlapped area image.

3. The vehicle-mounted image processing method based on directed vectors according to claim 1, characterized in that: The brightness adjustment strategy includes at least an over-brightness adjustment strategy and an over-darkness adjustment strategy; The executing a brightness adjustment strategy matching the real-time brightness result includes: Determine a first brightness threshold interval corresponding to the overbrightness adjustment strategy, and execute a first camera control instruction matching the first brightness threshold interval; and, Determine a second brightness threshold interval corresponding to the over-darkness adjustment strategy, and execute a second camera control instruction matching the second brightness threshold interval.

4. The vehicle-mounted image processing method based on directed vectors according to claim 1, characterized in that: The error equation is in, and Represent the adjustment coefficients of image i and image j, channel∈(R,G,B), σ N represents the standard deviation of color error, σ g represents the standard deviation of the gain parameter, and Mean represents the average brightness of the overlapping area of ​​image i and image j.

5. The vehicle-mounted image processing method based on directed vectors according to claim 1, characterized in that: After executing the brightness adjustment strategy, the vehicle-mounted image processing method based on directed vectors further includes: The real-time images acquired are processed by Poisson fusion using Gauss-Seidel iteration method to obtain the fused overlapping area image.

6. The vehicle-mounted image processing method based on directed vectors according to claim 5, characterized in that: The vehicle-mounted image processing method based on directed vectors also includes: Determine a non-overlapping area image based on the vehicle-mounted surround view and the overlapping area image; The non-overlapping area and the fused overlapping area images are stitched together using a surround view stitching algorithm to generate a vehicle-mounted surround view to be displayed.

7. The vehicle-mounted image processing method based on directed vectors according to claim 1, characterized in that: The real-time brightness result is displayed in the form of a tree diagram.

8. A vehicle-mounted image processing device based on directed vectors, characterized in that: include: A preprocessing module, used for determining an overlapped area image according to the bird's-eye view corresponding to the acquired vehicle-mounted surround view; A primary color equalization module, configured to perform brightness equalization processing on the overlapped area image based on the solution result of the error equation constructed for the overlapped area image, so as to obtain a primary color equalization image; A conversion module, used for performing vector conversion processing on the brightness of the overlapping areas of the primary uniform color image to obtain each brightness vector group; A processing module, used for determining a real-time brightness result of the overlapped area image based on the brightness vector group and a preset brightness threshold; An execution module is used to execute a brightness adjustment strategy that matches the real-time brightness result.

9. A vehicle domain controller, characterized in that: It includes a processor, a memory and a communication bus, wherein the processor and the memory communicate with an external vehicle-mounted camera via the communication bus; The memory is used to store computer programs; The processor is used to implement the vehicle-mounted image processing method based on directed vectors as described in any one of claims 1 to 7 when executing the computer program stored in the memory.

10. A vehicle, characterized in that: The vehicle has a computer program, wherein when the vehicle executes the computer program, it implements the vehicle-mounted image processing method based on directed vectors as described in any one of claims 1 to 7.

Images