Vehicle look-around image generation method and device, medium and equipment

Abstract

The invention provides a vehicle all-round view image generation method and device, a medium and equipment. The method comprises the following steps: converting a target video stream transmitted by a target camera into a top view video stream; performing image correction processing on the current frame image; carrying out image stabilization processing on the current frame image after image correction processing by adopting an image stabilization algorithm; performing color cast and brightness consistency correction on the current frame image after image stabilization processing; and according to the target mask image, splicing processing is performed on the current frame image after consistency correction so as to obtain a current spliced image, and the target mask image is a mask image corresponding to the target camera. The finally generated surround-view image has the characteristics of uniform color and uniform brightness, can adapt to airport environment requirements under different illumination conditions, provides clear and stable surround-view images for drivers, and significantly improves the driving safety and operation convenience of airport special vehicles in a complex environment.

Description

Technical Field

[0001] The present invention relates to the field of images, and in particular to a method, device, medium and equipment for generating a vehicle surround view image. Background Art

[0002] With economic development and increasing demand for airport operations, special vehicles are playing an increasingly important role in the airport environment. However, due to the diverse structures and sizes of special vehicles, as well as factors such as complex nighttime lighting and sudden climate changes, drivers face problems such as high jitter and blind spots in complex scenes when operating special vehicles. As an important auxiliary technology, the on-board surround view system can stitch images around the vehicle in real time through multiple cameras around the vehicle body, providing 360-degree environmental perception without blind spots. Improving the quality of stitched images has also become a difficult problem that technicians in this field continue to pay attention to. Summary of the Invention

[0003] The object of the present invention is to provide a method, device, medium and equipment for generating a vehicle surround view image to improve the above-mentioned problem.

[0004] In order to achieve the above objectives, the technical solutions adopted in the embodiments of the present invention are as follows: In a first aspect, an embodiment of the present invention provides a method for generating a vehicle surround view image, the method comprising: Converting a target video stream transmitted by a target camera into a bird's-eye view video stream, wherein the target camera is any one of a ring-shaped camera group arranged on the vehicle body; Performing image correction processing on a current frame image, wherein the current frame image is a first frame image in a bird's-eye view video stream transmitted by the target camera that has not yet been spliced; An image stabilization algorithm is used to perform image stabilization on the current frame image after image correction; Perform color deviation and brightness consistency correction on the current frame image after image stabilization processing; The current frame image after consistency correction is stitched according to the target mask image to obtain a current stitched image, wherein the target mask image is a mask image corresponding to the target camera.

[0005] Optionally, the step of converting the target video stream transmitted by the target camera into a bird's-eye view video stream includes: Performing distortion correction on the target video stream according to the intrinsic parameter coefficients of the target camera, wherein the intrinsic parameter coefficients include a focal length coefficient, a principal point coefficient, a radial distortion coefficient, and a tangential distortion coefficient; Constructing a mapping matrix according to the rotation matrix and translation vector of the target camera; The target video stream after distortion correction is converted into a bird's-eye view video stream based on the mapping matrix.

[0006] Optionally, the step of performing image correction processing on the current frame image includes: Extracting feature points within a feature area in the current frame image, wherein the feature area is an overlapping area between an image captured by the target camera and an image captured by an adjacent camera, and the feature points are pixels in the current frame image; Perform feature point matching on adjacent current frame images of the camera to obtain matching feature point pairs between adjacent current frame images of the camera; Get the average pixel distance between the matching feature point pairs between the adjacent current frame images of the camera; When the average value of the pixel distances between the matching feature point pairs is greater than a threshold, it is determined that there is a displacement between the adjacent current frame images of the cameras, and the average value of the pixel distances between the matching feature point pairs between the adjacent current frame images of the cameras is determined as the displacement coefficient between the adjacent current frame images of the cameras; The current frame image is corrected in combination with the displacement coefficient.

[0007] Optionally, the method further includes: adjusting the corresponding mapping matrix in combination with the displacement coefficient.

[0008] Optionally, the step of performing color cast and brightness consistency correction on the current frame image after image stabilization processing includes: Performing brightness anomaly and color shift detection on the current frame image after image stabilization to determine a normal frame image therein, wherein the normal frame is any current frame image that meets a preset condition, wherein the preset condition is that the brightness anomaly value is less than an anomaly threshold and the color shift value is less than a shift threshold; Color deviation and brightness consistency correction are performed on the abnormal frame image based on the normal frame image; wherein the abnormal frame image is a current frame image that does not meet a preset condition.

[0009] Optionally, the step of performing brightness anomaly and color shift detection on the current frame image after image stabilization to determine a normal frame image therein includes: Perform brightness anomaly and color shift detection on the current frame image after image stabilization processing to determine the brightness anomaly value and color shift value corresponding to the current frame image; Determining a comprehensive abnormal value of the current frame image according to the abnormal brightness value and the color offset value corresponding to the current frame image; The current frame image that meets the preset conditions and has the lowest comprehensive abnormality value is determined as a normal frame image.

[0010] Optionally, the method further comprises: generating an initial mask image corresponding to the target camera according to the vehicle body coordinate system and position information of the target camera relative to the center of the vehicle body; Gaussian blur processing is performed on the initial mask image to obtain a blurred target mask image.

[0011] In a second aspect, an embodiment of the present invention provides a vehicle surround view image generation device, the device comprising: a first processing unit, configured to convert a target video stream transmitted by a target camera into a bird's-eye view video stream, wherein the target camera is any one of a ring-shaped camera group arranged on the vehicle body; The first processing unit is further configured to perform image correction processing on a current frame image, wherein the current frame image is a first frame image in the bird's-eye view video stream transmitted by the target camera that has not yet been spliced; The first processing unit is further configured to perform image stabilization processing on the current frame image after image correction processing using an image stabilization algorithm; The first processing unit is further configured to perform color deviation and brightness consistency correction on the current frame image after image stabilization processing; The second processing unit is configured to perform a stitching process on the current frame image after consistency correction according to the target mask image to obtain a current stitched image, wherein the target mask image is a mask image corresponding to the target camera.

[0012] In a third aspect, an embodiment of the present invention provides a storage medium having a computer program stored thereon, which implements the above method when executed by a processor.

[0013] In a fourth aspect, an embodiment of the present invention provides an electronic device, comprising: a processor and a memory, wherein the memory is used to store one or more programs; when the one or more programs are executed by the processor, the above method is implemented.

[0014] Compared with the prior art, the embodiments of the present invention provide a vehicle surround view image generation method, device, medium and equipment, which converts the target video stream transmitted by the target camera into a bird's-eye view video stream, wherein the target camera is any camera in the annular camera group arranged on the vehicle body; performs image correction processing on the current frame image, wherein the current frame image is the first frame image in the bird's-eye view video stream transmitted by the target camera that has not yet been spliced; uses a stabilization algorithm to stabilize the current frame image after image correction processing; performs color deviation and brightness consistency correction on the current frame image after image stabilization processing; splices the current frame image after consistency correction according to the target mask image to obtain the current spliced ​​image, wherein the target mask image is the mask image corresponding to the target camera. The ultimately generated surround view image has the characteristics of uniform color and uniform brightness, can adapt to the airport environment requirements under different lighting conditions, provides the driver with a clear and stable surround view picture, and significantly improves the driving safety and operating convenience of airport special vehicles in complex environments.

[0015] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. It should be understood that the following drawings only illustrate certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without paying any creative work.

[0017] Figure 1 A schematic structural diagram of an electronic device provided by an embodiment of the present invention; Figure 2 One of the flow charts of the vehicle surround view image generation method provided in an embodiment of the present invention; Figure 3 A second flow chart of a method for generating a vehicle surround view image according to an embodiment of the present invention; Figure 4 The original image captured by the fisheye camera used for the surround view provided in the embodiment of the present invention; Figure 5 The image provided by the embodiment of the present invention is transformed into an image from a top-down perspective; Figure 6 The image before splicing and fusion provided by the embodiment of the present invention; Figure 7 The image after splicing and fusion provided by the embodiment of the present invention; Figure 8 The image before image correction processing provided by the embodiment of the present invention; Figure 9 The image after image correction provided by the embodiment of the present invention; Figure 10 The image before color cast and brightness consistency correction provided by the embodiment of the present invention; Figure 11 An image after color cast and brightness consistency correction provided by an embodiment of the present invention; Figure 12 A schematic diagram of the units of a vehicle surround view image generation device provided by an embodiment of the present invention.

[0018] In the figure: 10 - processor; 11 - memory; 12 - bus; 13 - communication interface; 301 - first processing unit; 302 - second processing unit. DETAILED DESCRIPTION

[0019] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions of the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings herein can be arranged and designed in various different configurations.

[0020] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention as claimed, but rather merely represents selected embodiments of the present invention. All other embodiments derived by persons of ordinary skill in the art based on the embodiments of the present invention without creative effort shall fall within the scope of protection of the present invention.

[0021] It should be noted that similar reference numerals and letters represent similar items in the following drawings. Therefore, once an item is defined in one drawing, it does not need to be further defined or explained in subsequent drawings. At the same time, in the description of the present invention, the terms "first", "second", etc. are used only to distinguish the description and should not be understood as indicating or implying relative importance.

[0022] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply the existence of any such actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or device comprising the element.

[0023] In the description of the present invention, it should be noted that the terms "upper", "lower", "inside", "outside", etc. indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, or are the orientations or positional relationships in which the inventive product is usually placed when in use. They are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation. Therefore, they should not be understood as limiting the present invention.

[0024] In the description of the present invention, it should also be noted that, unless otherwise expressly specified or limited, the terms "disposed" and "connected" should be understood in a broad sense. For example, they can refer to fixed connections, detachable connections, or integral connections; mechanical connections, or electrical connections; direct connections, indirect connections through an intermediate medium, or internal connections between two components. Those skilled in the art will understand the specific meanings of the above terms in the present invention based on the specific circumstances.

[0025] The following embodiments of the present invention are described in detail with reference to the accompanying drawings. In the absence of conflict, the following embodiments and features in the embodiments may be combined with each other.

[0026] Currently, there's no way to address the jitter caused by surround-view camera vibrations. The turbulence and vibrations during aircraft takeoff and landing, loading and unloading, and other operations can easily cause stitching errors and jitter in surround-view images. Current surround-view methods don't address the jitter-induced cracks in surround-view image stitching.

[0027] Currently, the conditions for correcting surround-view camera misalignment are quite stringent. The current surround-view algorithm requires lane lines to guide it in dealing with the misalignment problem caused by camera offset, and the correction algorithm has poor versatility.

[0028] Current surround view algorithms only address image equalization. They are typically designed for civilian vehicles, which are relatively small and operate in relatively simple environments. These algorithms typically address surround view image equalization solely through brightness adjustment. However, specialized airport vehicles often operate at night, where lighting color and brightness significantly impact surround view images. Furthermore, vehicles of varying sizes and lighting conditions can occlude surround view cameras, leading to uneven images.

[0029] In response to the above problems, an embodiment of the present invention provides a vehicle surround view image generation method, which may be a multi-path surround view image generation method for special vehicles at airports.

[0030] The embodiment of the present invention provides an electronic device, which can be a mobile phone device, a computer device, or a server device, such as an on-board computer in a special vehicle at an airport, or a back-end device that is connected to the on-board computer. Figure 1 , a schematic diagram of the structure of an electronic device. The electronic device includes a processor 10, a memory 11, and a bus 12. The processor 10 and the memory 11 are connected via the bus 12. The processor 10 is used to execute executable modules stored in the memory 11, such as computer programs.

[0031] Processor 10 can be an integrated circuit chip with signal processing capabilities. During implementation, each step of the vehicle surround view image generation method can be completed by hardware integrated logic circuits or software instructions within processor 10. The aforementioned processor 10 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0032] The memory 11 may include a high-speed random access memory (RAM), and may also include a non-volatile memory, such as at least one disk memory.

[0033] The bus 12 may be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, or an EISA (Extended Industry Standard Architecture) bus. Figure 1 Only one bidirectional arrow is used in the figure, but it does not mean that there is only one bus 12 or one type of bus 12.

[0034] The memory 11 is used to store programs, such as a program for the vehicle surround view image generation device. The vehicle surround view image generation device includes at least one software functional module, which can be stored in the memory 11 in the form of software or firmware, or embedded in the operating system (OS) of the electronic device. Upon receiving an execution instruction, the processor 10 executes the program to implement the vehicle surround view image generation method.

[0035] Possibly, the electronic device provided by the embodiment of the present invention further includes a communication interface 13. The communication interface 13 is connected to the processor 10 via a bus.

[0036] It should be understood that Figure 1 The structure shown is only a schematic diagram of a portion of the electronic device. The electronic device may also include Figure 1 More or fewer components than shown, or with Figure 1 Different configurations shown. Figure 1 Each component shown in the figure can be implemented by hardware, software or a combination thereof.

[0037] The vehicle surround view image generation method provided by the embodiment of the present invention can be applied to, but not limited to, Figure 1 For detailed procedures, please refer to the electronic equipment shown in Figure 2 The vehicle surround view image generation method includes: S11, S12, S13, S14 and S15, which are described in detail as follows.

[0038] S11, converting the target video stream transmitted by the target camera into a bird's-eye view video stream.

[0039] The target camera is any camera in a ring-shaped camera group arranged on the vehicle body, and the cameras in the ring-shaped camera group may be, but are not limited to, fisheye cameras.

[0040] Optionally, the annular camera group includes 4 or 6 fisheye cameras, which are arranged on the vehicle body and correspond to different orientations for collecting real-time images around the vehicle.

[0041] Optionally, the step of converting the target video stream transmitted by the target camera into a bird's-eye view video stream includes: S111, S112 and S113, which are described in detail as follows.

[0042] S111 , performing distortion correction on the target video stream according to the intrinsic parameter coefficients of the target camera.

[0043] The intrinsic parameter coefficients include focal length coefficient, principal point coefficient, radial distortion coefficient and tangential distortion coefficient.

[0044] Focal length factor: describes the focal length of the camera, usually in pixels, and is used to measure the degree of image zoom.

[0045] Principal point coefficient: describes the projection position of the origin of the image coordinate system in the camera coordinate system, usually in pixels, indicating the position of the center point of the image relative to the camera.

[0046] Radial distortion coefficient: describes the degree of radial distortion, expressed by parameters k1, k2, etc., and is used to correct barrel or pincushion distortion of the image.

[0047] Tangential distortion coefficient: describes the degree of tangential distortion, expressed by parameters d1, d2, etc., and is used to correct the asymmetric distortion of the image.

[0048] The intrinsic coefficients can be obtained during the camera calibration process, which is usually done using a calibration board or a specific scene to ensure the accuracy of the image under different viewing angles and scenes.

[0049] First, the target video stream is distorted using the intrinsic parameter coefficients of the target camera to remove the edge curvature of the fisheye lens and obtain a normal image.

[0050] S112: Construct a mapping matrix according to the rotation matrix and translation vector of the target camera.

[0051] Rotation Matrix: Describes the rotation relationship between the camera coordinate system and the world coordinate system. It is used to align the perspective of each camera to ensure consistent perspective between multiple images. The rotation matrix is ​​represented as a 3x3 matrix.

[0052] Translation Vector: Describes the translation relationship between the camera coordinate system and the world coordinate system. It is used to locate the specific position of each camera in the system. The translation vector is expressed as a 3x1 column vector.

[0053] The rotation matrix and translation vector are obtained through the camera calibration process and are used to convert the images acquired by each camera into a unified surround view coordinate system, thereby achieving accurate image stitching and fusion.

[0054] S113, converting the target video stream after distortion correction into a bird's-eye view video stream based on the mapping matrix.

[0055] Based on the mapping matrix, the target video stream after distortion correction is converted into a bird's-eye view video stream, so that the video streams of different cameras are aligned on the bird's-eye view plane, preparing for image fusion.

[0056] S12, performing image correction processing on the current frame image.

[0057] The current frame image is the first frame image in the bird's-eye view video stream transmitted by the target camera that has not yet been spliced.

[0058] Optionally, S12, the step of performing image correction processing on the current frame image, includes: S121, S122, S123, S124 and S125, which are specifically described as follows.

[0059] S121, extracting feature points in the feature area of ​​the current frame image.

[0060] The feature area is the overlapping area of ​​the image captured by the target camera and the image captured by the adjacent camera, and the feature point is the pixel point in the current frame image.

[0061] Optionally, a feature point recognition and matching model is used to extract key feature points in the feature area of ​​the current frame image and perform matching.

[0062] S122 , performing feature point matching on adjacent current frame images of the camera to obtain matching feature point pairs between adjacent current frame images of the camera.

[0063] S123: Obtain an average value of pixel distances between matching feature point pairs between adjacent current frame images of the camera.

[0064] S124, when the average value of the pixel distances in the matching feature point pairs is greater than the threshold, it is determined that there is a displacement between the adjacent current frame images of the cameras, and the average value of the pixel distances of the matching feature point pairs between the adjacent current frame images of the cameras is determined as the displacement coefficient between the adjacent current frame images of the cameras.

[0065] S125, performing image correction processing on the current frame image in combination with the displacement coefficient.

[0066] When it is detected that the camera has slightly moved and caused image misalignment, that is, when there is displacement between adjacent current frame images of the camera, the misalignment phenomenon is corrected by combining the displacement coefficient.

[0067] In an optional implementation, the vehicle surround view image generation method further includes: S126, as follows.

[0068] S126, adjusting the corresponding mapping matrix in combination with the displacement coefficient.

[0069] The corresponding transformation matrix is ​​calculated through the information (displacement coefficient) between the feature points, and the mapping matrix of the corresponding camera is modified based on the transformation matrix to correct the misalignment.

[0070] S13, using an image stabilization algorithm to perform image stabilization processing on the current frame image after the image correction processing.

[0071] Optionally, the image stabilization algorithm adopts a Kalman filter algorithm.

[0072] The Kalman filter algorithm is used to analyze the global motion of video frames, filter out high-frequency jitter, perform image translation and rotation transformations based on the filtered motion vector, align the image to a stable bird's-eye view, generate a stable video stream, and ensure image stability.

[0073] S14, performing color deviation and brightness consistency correction on the current frame image after the image stabilization processing.

[0074] Optionally, S14, the step of performing color cast and brightness consistency correction on the current frame image after image stabilization processing, includes: S141 and S142, as follows.

[0075] S141 , performing brightness anomaly and color shift detection on the current frame image after image stabilization processing to determine a normal frame image therein.

[0076] The normal frame is any current frame image that meets a preset condition, and the preset condition is that the abnormal brightness value is less than the abnormal threshold value, and the color offset value is less than the offset threshold value.

[0077] S142: Perform color deviation and brightness consistency correction on the abnormal frame image based on the normal frame image.

[0078] The abnormal frame image is a current frame image that does not meet the preset conditions.

[0079] To achieve brightness and color consistency in surround view images in airport environments, the system detects brightness anomalies and color shifts in each video stream within the video frame's color space, using normal frames as a reference for estimation. The system then adjusts the brightness and color deviations of abnormal frames using the normal frames as a benchmark. A harmonization algorithm then balances the brightness and color of each video stream, eliminating differences in brightness and color across the streams. This creates a uniform surround view image with uniform color and brightness, ensuring brightness and color consistency across multiple surround view channels.

[0080] S15 , performing a stitching process on the current frame image after consistency correction according to the target mask image to obtain a current stitched image, wherein the target mask image is a mask image corresponding to the target camera.

[0081] In the vehicle surround view image generation method provided in an embodiment of the present invention, the video stream is first converted into a bird's-eye view video stream, so that the video streams from different cameras are aligned on the bird's-eye view plane, preparing for image fusion. Real-time perception and correction of image jitter effectively reduces the image jitter and misalignment problems caused by the surround view images of airport special vehicles during operation, ensuring smooth and continuous image output while the vehicle is in motion. Even in the event of a minor collision or camera position shift, accurate image adjustment can be achieved, ensuring the driver's clear perception of the surrounding environment. In terms of image consistency, the brightness and color of each video image are adjusted in conjunction with a harmonization method, significantly improving the color and brightness consistency of each fisheye camera video in the airport environment. The resulting surround view image has the characteristics of uniform color and brightness, which can adapt to the requirements of the airport environment under different lighting conditions, provide the driver with a clear and stable surround view, and significantly improve the driving safety and operational convenience of airport special vehicles in complex environments.

[0082] Each step in the embodiment of the present invention can be processed by different independent threads respectively. Through multi-threaded processing, the waiting time of the system is reduced, and the processing speed and real-time performance of the video stream are improved.

[0083] Based on the foregoing, the present invention also provides an optional implementation for S141, which is described below. S141, performing brightness anomaly and color shift detection on the current frame image after image stabilization to determine a normal frame image, includes S141A, S141B, and S141C, which are described in detail below.

[0084] S141A, performing brightness anomaly and color shift detection on the current frame image after the image stabilization process to determine a brightness anomaly value and a color shift value corresponding to the current frame image.

[0085] S141B, determining a comprehensive abnormal value of the current frame image according to the abnormal brightness value and the color offset value corresponding to the current frame image.

[0086] S141C, determining the current frame image that meets the preset conditions and has the lowest comprehensive abnormality value as a normal frame image.

[0087] Please refer to Figure 3 In an optional implementation, the vehicle surround view image generation method further includes: S21 and S22, which are described in detail as follows.

[0088] S21 : Generate an initial mask image corresponding to the target camera according to the vehicle body coordinate system and the position information of the target camera relative to the center of the vehicle body.

[0089] S22, performing Gaussian blur processing on the initial mask image to obtain a blurred target mask image.

[0090] By generating a camera position mask and blurring the stitching edges, we reduce the seams when stitching images from different camera perspectives, smoothing boundary pixels and reducing edge artifacts during fusion. This ensures that the resulting surround view image has no noticeable stitching artifacts and presents a continuous and consistent image. This makes the surround view more coherent and consistent, improving the overall image perception.

[0091] Please refer to Figure 4 、 Figure 5 、 Figure 6 、 Figure 7 、 Figure 8 、 Figure 9 、 Figure 10 as well as Figure 11 , Figure 4 The original image captured by the fisheye camera used for the surround view provided in the embodiment of the present invention; Figure 5 The image provided by the embodiment of the present invention is transformed into an image from a top-down perspective; Figure 6 The image before splicing and fusion provided by the embodiment of the present invention; Figure 7 The image after splicing and fusion provided by the embodiment of the present invention; Figure 8 The image before image correction processing provided by the embodiment of the present invention; Figure 9 The image after image correction provided by the embodiment of the present invention; Figure 10 The image before color cast and brightness consistency correction provided by the embodiment of the present invention; Figure 11 This is an image after color cast and brightness consistency correction provided by an embodiment of the present invention.

[0092] See also Figure 12 , Figure 12 An embodiment of the present invention provides a vehicle surround view image generation device. Optionally, the vehicle surround view image generation device is applied to the electronic device described above.

[0093] The vehicle surround view image generating device includes: a first processing unit 301 and a second processing unit 302 .

[0094] The first processing unit 301 is configured to convert a target video stream transmitted by a target camera into a bird's-eye view video stream, wherein the target camera is any camera in a ring-shaped camera group arranged on the vehicle body; The first processing unit 301 is further configured to perform image correction processing on a current frame image, wherein the current frame image is the first frame image in the bird's-eye view video stream transmitted by the target camera that has not yet been spliced; The first processing unit 301 is further configured to perform image stabilization processing on the current frame image after image correction processing using an image stabilization algorithm; The first processing unit 301 is further configured to perform color deviation and brightness consistency correction on the current frame image after image stabilization processing; The second processing unit 302 is configured to perform stitching processing on the current frame image after consistency correction according to the target mask image to obtain a current stitched image, wherein the target mask image is a mask image corresponding to the target camera.

[0095] Optionally, the second processing unit 302 may execute the above S15, and the first processing unit 301 may execute other steps in the above method embodiment.

[0096] It should be noted that the vehicle surround view image generation device provided in this embodiment can execute the method flow shown in the above method flow embodiment to achieve the corresponding technical effects. For the sake of brevity, any part not mentioned in this embodiment can be referred to the corresponding content in the above embodiment.

[0097] An embodiment of the present invention further provides a storage medium storing computer instructions or programs that, when read and executed, execute the vehicle surround view image generation method of the above embodiment. The storage medium may include memory, flash memory, registers, or a combination thereof.

[0098] The following provides an electronic device, which can be a mobile phone device, a computer device, or a server device, such as a driving computer in a special vehicle at an airport, or a back-end device that is connected to the driving computer. Figure 1 As shown, the above-mentioned vehicle surround view image generation method can be implemented. Specifically, the electronic device includes: a processor 10, a memory 11, and a bus 12. The processor 10 may be a CPU. The memory 11 is used to store one or more programs. When the one or more programs are executed by the processor 10, the vehicle surround view image generation method of the above-mentioned embodiment is executed.

[0099] In summary, the embodiments of the present invention provide a vehicle surround view image generation method, device, medium and equipment, which converts the target video stream transmitted by the target camera into a bird's-eye view video stream, wherein the target camera is any camera in the annular camera group arranged on the vehicle body; performs image correction processing on the current frame image, wherein the current frame image is the first frame image in the bird's-eye view video stream transmitted by the target camera that has not yet been spliced; uses a stabilization algorithm to perform image stabilization processing on the current frame image after image correction processing; performs color deviation and brightness consistency correction on the current frame image after image stabilization processing; splices the current frame image after consistency correction according to the target mask image to obtain the current spliced ​​image, wherein the target mask image is the mask image corresponding to the target camera. The finally generated surround view image has the characteristics of uniform color and uniform brightness, can adapt to the airport environment requirements under different lighting conditions, provides the driver with a clear and stable surround view picture, and significantly improves the driving safety and operating convenience of airport special vehicles in complex environments.

[0100] The foregoing description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Those skilled in the art will readily appreciate that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention shall be included within the scope of protection of the present invention.

[0101] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above and that the invention can be embodied in other specific forms without departing from the spirit or essential characteristics of the invention. Therefore, the embodiments should be considered in all respects as illustrative and non-restrictive, and the scope of the invention is defined by the appended claims, not the foregoing description, and all variations within the meaning and range of equivalents of the claims are intended to be included therein. Any reference sign in a claim should not be construed as limiting the claim to which it relates.

Claims

1. A method for generating a vehicle surround view image, characterized in that: The method comprises: Converting a target video stream transmitted by a target camera into a bird's-eye view video stream, wherein the target camera is any one of a ring camera group arranged on a vehicle body; Performing image correction processing on a current frame image, wherein the current frame image is a first frame image in a bird's-eye view video stream transmitted by the target camera that has not yet been spliced; An image stabilization algorithm is used to perform image stabilization processing on the current frame image after image correction processing; Perform color deviation and brightness consistency correction on the current frame image after image stabilization processing; The current frame image after consistency correction is stitched according to the target mask image to obtain a current stitched image, wherein the target mask image is a mask image corresponding to the target camera.

2. The vehicle surround view image generation method according to claim 1, characterized in that: The step of converting the target video stream transmitted by the target camera into a bird's-eye view video stream comprises: Performing distortion correction on the target video stream according to the intrinsic parameter coefficients of the target camera, wherein the intrinsic parameter coefficients include a focal length coefficient, a principal point coefficient, a radial distortion coefficient, and a tangential distortion coefficient; Constructing a mapping matrix according to the rotation matrix and translation vector of the target camera; The distortion-corrected target video stream is converted into a bird's-eye view video stream based on the mapping matrix.

3. The vehicle surround view image generation method according to claim 2, characterized in that: The step of performing image correction processing on the current frame image includes: Extracting feature points in a feature area in the current frame image, wherein the feature area is an overlapping area of ​​an image captured by the target camera and an image captured by an adjacent camera, and the feature points are pixel points in the current frame image; Perform feature point matching on adjacent current frame images of the camera to obtain matching feature point pairs between adjacent current frame images of the camera; Get the average pixel distance of the matching feature point pairs between the adjacent current frame images of the camera; When the average pixel distance between the matching feature point pairs is greater than a threshold, it is determined that there is a displacement between the adjacent current frame images of the camera, and the average pixel distance between the matching feature point pairs between the adjacent current frame images of the camera is determined as the displacement coefficient between the adjacent current frame images of the camera; The current frame image is corrected by combining the displacement coefficient.

4. The vehicle surround view image generation method according to claim 3, characterized in that: The method further comprises: Adjust the corresponding mapping matrix in combination with the displacement coefficient.

5. The vehicle surround view image generation method according to claim 2, characterized in that: The step of performing color deviation and brightness consistency correction on the current frame image after the image stabilization process comprises: Perform brightness anomaly and color shift detection on the current frame image after the image stabilization process to determine a normal frame image therein, wherein the normal frame is any current frame image that meets a preset condition, and the preset condition is that the brightness anomaly value is less than the anomaly threshold, and the color shift value is less than the shift threshold; The abnormal frame image is corrected for color deviation and brightness consistency based on the normal frame image; wherein the abnormal frame image is a current frame image that does not meet a preset condition.

6. The vehicle surround view image generation method according to claim 5, characterized in that: The step of performing brightness anomaly and color shift detection on the current frame image after the image stabilization process to determine a normal frame image therein comprises: Perform brightness anomaly and color shift detection on the current frame image after image stabilization processing to determine the brightness anomaly value and color shift value corresponding to the current frame image; Determine a comprehensive abnormal value of the current frame image according to the brightness abnormal value and the color offset value corresponding to the current frame image; The current frame image that meets the preset conditions and has the lowest comprehensive abnormal value is determined as a normal frame image.

7. The vehicle surround view image generation method according to claim 1, characterized in that: The method further comprises: Generate an initial mask image corresponding to the target camera according to the vehicle body coordinate system and the position information of the target camera relative to the center of the vehicle body; The initial mask image is subjected to Gaussian blur processing to obtain a blurred target mask image.

8. A vehicle surround view image generating device, characterized in that: The device comprises: A first processing unit, configured to convert a target video stream transmitted by a target camera into a bird's-eye view video stream, wherein the target camera is any one of a ring camera group arranged on a vehicle body; The first processing unit is further used to perform image correction processing on the current frame image, wherein the current frame image is the first frame image in the bird's-eye view video stream transmitted by the target camera that has not yet been spliced; The first processing unit is further used to perform image stabilization processing on the current frame image after the image correction processing by using an image stabilization algorithm; The first processing unit is further used to perform color deviation and brightness consistency correction on the current frame image after the image stabilization processing; The second processing unit is used to perform stitching processing on the current frame image after consistency correction according to the target mask image to obtain a current stitching image, wherein the target mask image is a mask image corresponding to the target camera.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

10. An electronic device, characterized in that: include: A processor and a memory, the memory being used to store one or more programs; When the one or more programs are executed by the processor, the method according to any one of claims 1 to 7 is implemented.

Images