Vehicle all-round view image generation method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention provides a vehicle look-around image generation method and device, electronic equipment and a storage medium, and relates to the technical field of image processing. The method comprises the following steps: acquiring an initial look-around image of a vehicle at the current moment based on each image sensor; based on the image transformation matrix determined at the current moment, determining the offset of the initial looking-around image relative to the historical vehicle looking-around image; wherein the historical vehicle all-round view image is a vehicle all-round view image generated in the last period, and the historical vehicle all-round view image comprises a historical chassis image. And obtaining an initial spliced image according to the initial look-around image, the historical chassis image and the offset based on an ORB algorithm and an RANSAC algorithm. And preprocessing the initial spliced image, and fusing the preprocessed initial spliced image with the historical vehicle look-around image to obtain a current vehicle look-around image. According to the invention, the problem that the generation of the vehicle panoramic view has poor adaptability to the dynamic environment can be effectively solved.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and more specifically, to a method, apparatus, electronic device, and storage medium for generating vehicle surround view images. Background Technology

[0002] With the development of automotive intelligent technology, AVM (Around View Monitor) has become an important component of driver assistance systems. This system uses four fisheye cameras installed in the front, rear, left, and right directions of the vehicle to capture images, which are then stitched together to provide the driver with a 360° bird's-eye view of the vehicle's surroundings, effectively solving the blind spot problem inherent in traditional rearview mirrors.

[0003] However, the relevant technologies are poorly adapted to dynamic environments and have difficulty processing image shifts caused by vehicle movement in real time, which can lead to tearing or blurring of stitched images. Summary of the Invention

[0004] The present invention aims to, for example, provide a method, apparatus, electronic device, and storage medium for generating vehicle surround view images, which can at least partially solve the above-mentioned technical problems.

[0005] The embodiments of the present invention can be implemented as follows: In a first aspect, embodiments of the present invention provide a method for generating a vehicle surround view image, applied to a controller of a vehicle surround view image generation system. The vehicle surround view image generation system further includes multiple image sensors, each of which is respectively installed at preset positions around the vehicle. The controller is communicatively connected to each of the image sensors. The method includes: Based on each of the image sensors, an initial surround view image of the vehicle at the current moment is obtained; Based on the image transformation matrix determined at the current moment, the offset of the initial surround view image relative to the historical vehicle surround view image is determined; wherein, the historical vehicle surround view image is the vehicle surround view image generated in the previous cycle, and the historical vehicle surround view image includes the historical chassis image. An initial stitched image is obtained based on the ORB algorithm and the RANSAC algorithm, respectively, according to the initial surround view image, the historical chassis image, and the offset. The initial stitched image is preprocessed, and the preprocessed initial stitched image is fused with the historical vehicle surround view image to obtain the current vehicle surround view image.

[0006] Optionally, the vehicle surround view image generation system further includes a steering wheel angle sensor and a vehicle speed sensor, and the controller is communicatively connected to the steering wheel angle sensor and the vehicle speed sensor respectively; the method further includes a step of determining the image transformation matrix, which includes: The steering wheel angle of the vehicle at the current moment is obtained based on the steering wheel angle sensor; The vehicle speed sensor is used to obtain the wheel speed pulse count and tire unit pulse count of the vehicle, where the tire unit pulse count is the total number of pulses for one revolution of the tire. The image transformation matrix is ​​determined based on the steering wheel angle, the wheel speed pulse count, and the tire unit pulse count.

[0007] Optionally, determining the image transformation matrix based on the steering wheel angle, the wheel speed pulse count, and the tire unit pulse count includes: Based on the front wheel sway angle calculation formula, the front wheel sway angle of the vehicle is obtained according to the steering wheel angle and the preset transmission ratio. The front wheel sway angle calculation formula is as follows:

[0008] Wherein, α is the front wheel sway angle, β is the steering wheel angle, and i is the preset transmission ratio; Based on the turning radius calculation formula, the turning radius of the vehicle is obtained according to the vehicle width, wheelbase, and front wheel sway angle. The turning radius calculation formula is as follows:

[0009] Where W is the vehicle width, D L Where r is the wheelbase and r is the turning radius; Based on the yaw angle calculation formula, the yaw angle of the vehicle is obtained according to the tire diameter, the wheel speed pulse count, the tire unit pulse count, and the turning radius. The yaw angle calculation formula is as follows:

[0010] Among them, D r Where P is the tire diameter, N is the wheel speed pulse count, and θ is the tire unit pulse count; The image transformation matrix is ​​obtained based on the yaw angle.

[0011] Optionally, the image transformation matrix is ​​represented as:

[0012]

[0013]

[0014] Where T is the image transformation matrix.

[0015] Optionally, the step of obtaining the initial stitched image based on the ORB algorithm and the RANSAC algorithm, respectively, according to the initial surround view image, the historical chassis image, and the offset, includes: Based on the ORB algorithm, the first image features of the initial surround view image and the second image features of the historical chassis image are extracted respectively. Based on the offset, feature matching is performed between the first image features and the second image features to obtain a set of matching points; The matching point set is optimized based on the RANSAC algorithm to obtain multiple optimal matching pairs; Based on the weighted fusion formula, an initial stitched image is generated according to the optimal matching pair.

[0016] Optionally, generating the initial stitched image based on the optimal matching pair according to the weighted fusion formula includes: Substitute each of the optimal matching pairs into the weighted fusion formula to obtain the initial stitched image; The weighted fusion formula is as follows:

[0017]

[0018] Among them, w i For weights, d i The Euclidean distance between the matching points is given by σ, where σ is the scale parameter. , These are the two pixels of the optimal matching pair. The initial toroidal image, The historical chassis image, This refers to the initial stitched image.

[0019] Optionally, the preprocessing of the initial stitched image includes: The initial stitched image is subjected to grayscale conversion and histogram equalization processing respectively; The initial stitched image, after grayscale conversion and histogram equalization, undergoes color space conversion to complete the preprocessing of the initial stitched image.

[0020] Secondly, embodiments of the present invention provide a vehicle surround view image generation device, which is applied to a controller of a vehicle surround view image generation system. The vehicle surround view image generation system further includes multiple image sensors, each of which is installed at a preset position around the vehicle. The controller is communicatively connected to each of the image sensors. The vehicle surround view image generation device includes: An initial surround view image acquisition unit is used to acquire an initial surround view image of the vehicle at the current moment based on each of the image sensors. The offset determination unit is used to determine the offset of the initial surround view image relative to the historical vehicle surround view image based on the image transformation matrix determined at the current time; wherein, the historical vehicle surround view image is the vehicle surround view image generated in the previous cycle, and the historical vehicle surround view image includes a historical chassis image. The initial stitched image determination unit is used to obtain an initial stitched image based on the ORB algorithm and the RANSAC algorithm, respectively, according to the initial surround view image, the historical chassis image, and the offset. The vehicle surround view image processing unit is used to preprocess the initial stitched image and fuse the preprocessed initial stitched image with the historical vehicle surround view image to obtain the current vehicle surround view image.

[0021] Thirdly, embodiments of the present invention provide an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of any of the methods described above.

[0022] Fourthly, embodiments of the present invention provide a computer-readable storage medium, the computer-readable storage medium including a computer program, wherein the computer program, when executed, controls a server where the computer-readable storage medium is located to implement the steps of any of the methods described above.

[0023] The beneficial effects of the embodiments of the present invention include, for example: This invention determines the offset of the initial surround view image relative to the historical vehicle surround view image based on the image transformation matrix determined at the current moment. Then, based on the initial surround view image, the historical chassis image, and the offset, an initial stitched image is obtained, which in turn yields the current vehicle surround view image. By determining the offset of the initial surround view image relative to the historical vehicle surround view image and using this offset to generate the initial stitched image, the solution of this invention can effectively solve the problems of image offset, tearing, or blurring caused by vehicle movement. Attached Figure Description

[0024] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0025] Figure 1 A block diagram illustrating an electronic device according to an embodiment of the present invention; Figure 2 A flowchart illustrating the steps of a method for generating a vehicle surround view image according to an embodiment of the present invention; Figure 3 A system architecture diagram of a vehicle surround view image generation system provided in an embodiment of the present invention; Figure 4 This is a structural diagram of a vehicle surround view image generation device provided in an embodiment of the present invention.

[0026] Icons: 100 - Electronic device; 110 - Memory; 120 - Processor; 130 - Communication module; 300 - Vehicle surround view image generation device; 301 - Initial surround view image acquisition unit; 302 - Offset determination unit; 303 - Initial stitched image determination unit; 304 - Vehicle surround view image processing unit. Detailed Implementation

[0027] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0028] Therefore, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.

[0029] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0030] Furthermore, the terms "first" and "second" are used only to distinguish descriptions and should not be interpreted as indicating or implying relative importance.

[0031] It should be noted that, where there is no conflict, the features in the embodiments of the present invention can be combined with each other.

[0032] Existing panoramic surround view systems typically use four fisheye cameras as image acquisition devices, paired with an image processing host to stitch the images together. The camera hardware includes modules such as image sensors (e.g., CMOS chips), image signal processors (ISPs), and serial communication chips, transmitting image data to the host via coaxial cables. The system-on-chip (SOC) within the host executes algorithms for image calibration, distortion correction, and stitching, ultimately rendering and outputting a 2D or 3D surround view image through a graphics processing unit (GPU).

[0033] However, existing panoramic surround view systems are poorly adapted to dynamic environments while the vehicle is in motion, and have difficulty processing image shifts caused by vehicle movement in real time, resulting in tearing or blurring of stitched images.

[0034] Secondly, due to differences in the installation positions and angles of the cameras, existing panoramic surround view systems often exhibit ghosting and misalignment in the stitched images, especially noticeable in areas with three-dimensional obstacles. When the vehicle is in motion, the same object captured from different perspectives cannot completely overlap due to differences in projection, resulting in visual distortion in the synthesized image.

[0035] These problems severely impact the imaging quality and usability of panoramic surround view systems.

[0036] Based on the above, embodiments of the present invention provide a method, apparatus, electronic device, and storage medium for generating vehicle surround view images, which can effectively alleviate the above-mentioned technical problems.

[0037] Please refer to Figure 1 This is a block diagram of an electronic device 100 provided in this application. The electronic device 100 can be a data processing device, and this embodiment does not limit this. The electronic device 100 includes a memory 110, a processor 120, and a communication module 130. The memory 110, processor 120, and communication module 130 are electrically connected directly or indirectly to achieve data transmission or interaction. For example, these components can be electrically connected to each other through one or more communication buses or signal lines.

[0038] The memory 110 is used to store programs or data. The memory 110 may be, but is not limited to, random access memory (RAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), etc.

[0039] The processor 120 is used to read / write data or programs stored in memory and to perform corresponding functions.

[0040] The communication module 130 is used to establish a communication connection between the server and other communication terminals through the network, and to send and receive data through the network.

[0041] It should be understood that, Figure 1 The structure shown is only a schematic diagram of the electronic device 100. The electronic device 100 may also include components that are larger than... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown, for example, electronic device 100 may also include sensors, etc. Figure 1 The components shown can be implemented using hardware, software, or a combination thereof. This electronic device 100 can be integrated into other devices or configured as a standalone device.

[0042] Corresponding to electronic device 100, this embodiment of the invention provides a method for generating a vehicle surround view image, which can be applied to electronic device 100 and to a controller of a vehicle surround view image generation system. The vehicle surround view image generation system further includes multiple image sensors, each of which is installed at a preset position around the vehicle. The controller is communicatively connected to each of the image sensors. The method includes, as follows: Figure 2 The following steps are shown: Step S110: Based on each of the image sensors, acquire the initial surround view image of the vehicle at the current moment.

[0043] Step S120: Based on the image transformation matrix determined at the current moment, determine the offset of the initial surround view image relative to the historical vehicle surround view image. The historical vehicle surround view image is the vehicle surround view image generated in the previous cycle, and includes a historical chassis image.

[0044] Step S130: Based on the ORB algorithm and the RANSAC algorithm respectively, obtain the initial stitched image according to the initial surround view image, the historical chassis image and the offset.

[0045] Step S140: Preprocess the initial stitched image and fuse the preprocessed initial stitched image with the historical vehicle surround view image to obtain the current vehicle surround view image.

[0046] In step S110, an initial surround view image of the vehicle at the current moment is acquired based on each of the image sensors.

[0047] The image sensor in this embodiment of the invention can be a four-channel fisheye sensor. In practice, the four image sensors can be installed at preset positions around the vehicle. For example, two image sensors can be installed below the two rearview mirrors of the vehicle, and the other two image sensors can be installed directly below the front and rear license plates of the vehicle.

[0048] When the command to acquire surround view images is triggered (e.g., when the user puts the gear in reverse or presses the button corresponding to display the vehicle's panoramic surround view), the controller controls each image sensor to acquire the image captured at the current moment. These four images can be the initial surround view image of the vehicle at the current moment.

[0049] In step S120, based on the image transformation matrix determined at the current moment, the offset of the initial surround view image relative to the historical vehicle surround view image is determined. The historical vehicle surround view image is the vehicle surround view image generated in the previous cycle, and the historical vehicle surround view image includes a historical chassis image.

[0050] The image transformation matrix can be used to determine how much the initial surround view image acquired at the current moment deviates from the previously generated historical vehicle surround view image. The image transformation matrix can be determined in real time. Since the historical surround view image is a fully generated vehicle surround view image, it can also include the historical chassis image obtained by synthesizing and fusing the surround view image acquired in the previous cycle.

[0051] After obtaining the initial surround view image of the vehicle at the current moment, the historical vehicle surround view image and the initial surround view image are respectively input into the image transformation matrix. Then, the offset of the initial surround view image relative to the historical vehicle surround view image can be obtained. This offset is used to indicate how much the initial surround view image obtained at the current moment has deviated from the previously generated historical vehicle surround view image.

[0052] Optionally, the vehicle surround view image generation system further includes a steering wheel angle sensor and a vehicle speed sensor, and the controller is communicatively connected to the steering wheel angle sensor and the vehicle speed sensor, respectively. The method further includes a step of determining the image transformation matrix, which includes: The steering wheel angle of the vehicle at the current moment is obtained based on the steering wheel angle sensor.

[0053] The vehicle speed sensor acquires the wheel speed pulse count and tire unit pulse count of the vehicle, where the tire unit pulse count is the total number of pulses per revolution of the tire.

[0054] The image transformation matrix is ​​determined based on the steering wheel angle, the wheel speed pulse count, and the tire unit pulse count.

[0055] Please see Figure 3 This is an architecture diagram of a vehicle surround view image generation system. Each sensor is connected to the controller for communication. When any sensor needs to collect data, the controller can directly control that sensor to collect the corresponding data.

[0056] While acquiring the initial surround view image, the controller can obtain the current steering wheel angle based on the steering wheel angle sensor, and the number of wheel speed pulses and tire unit pulses based on the vehicle speed sensor, thereby determining the vehicle's real-time driving information at the current moment. The image transformation matrix for the current moment is determined using the steering wheel angle, wheel speed pulse count, and tire unit pulse count.

[0057] Optionally, determining the image transformation matrix based on the steering wheel angle, the wheel speed pulse count, and the tire unit pulse count includes: Based on the front wheel sway angle calculation formula, the front wheel sway angle of the vehicle is obtained according to the steering wheel angle and the preset transmission ratio. The front wheel sway angle calculation formula is as follows:

[0058] Wherein, α is the front wheel sway angle, β is the steering wheel angle, and i is the preset transmission ratio.

[0059] Based on the turning radius calculation formula, the turning radius of the vehicle is obtained according to the vehicle width, wheelbase, and front wheel sway angle. The turning radius calculation formula is as follows:

[0060] Where W is the vehicle width, D L Let r be the wheelbase and r be the turning radius.

[0061] Based on the yaw angle calculation formula, the yaw angle of the vehicle is obtained according to the tire diameter, wheel speed pulse count, tire unit pulse count, and turning radius. The yaw angle calculation formula is as follows:

[0062] Among them, D r Let P be the tire diameter, N be the wheel speed pulse count, θ be the tire unit pulse count, and θ be the yaw angle. The image transformation matrix is ​​obtained based on the yaw angle.

[0063] In one optional implementation, the controller can calculate the front wheel sway angle of the vehicle based on the formula for calculating the front wheel sway angle, using the ratio of the vehicle's steering wheel angle to a preset transmission ratio. After obtaining the front wheel sway angle, the controller substitutes the front wheel sway angle, the vehicle's width, and the wheelbase into the formula for calculating the turning radius to obtain the vehicle's turning radius. The vehicle width and wheelbase can be pre-stored in the controller's corresponding memory.

[0064] Once the turning radius is obtained, the controller combines the tire diameter, wheel speed pulse count, and tire unit pulse count, and substitutes these data into the yaw angle calculation formula to obtain the yaw angle. Similarly, the tire diameter can also be pre-stored in the corresponding memory of the controller.

[0065] Once the yaw angle is obtained, the image transformation matrix can be derived from it.

[0066] Optionally, the image transformation matrix is ​​represented as:

[0067]

[0068]

[0069] Where T is the image transformation matrix.

[0070] In one alternative implementation, the image transformation matrix T can be as shown above, wherein, ,and x and y are the horizontal and vertical coordinates of the image pixels, respectively.

[0071] In step S130, an initial stitched image is obtained based on the ORB algorithm and the RANSAC algorithm, respectively, according to the initial surround view image, the historical chassis image, and the offset.

[0072] The ORB (Oriented FAST and Rotated BRIEF) algorithm is a highly efficient feature detection and description algorithm. It employs an improved FAST keypoint detection and BRIEF descriptor extraction algorithm, controlling the number of feature points detected per frame to 800-1200, with feature point localization accuracy reaching sub-pixel level. The RANSAC (Random Sample Consensus) algorithm, on the other hand, establishes matching relationships based on the image transformation matrix and eliminates mismatches through a random sample consensus algorithm, achieving a matching accuracy of ≥95%.

[0073] After determining the offset based on the image transformation matrix, feature extraction and stitching processing can be performed on the initial surround view image and the historical chassis image of the historical vehicle surround view image using the ORB algorithm and the RANSAC algorithm respectively, thereby obtaining the initial stitched image.

[0074] Optionally, the step of obtaining the initial stitched image based on the ORB algorithm and the RANSAC algorithm, respectively, according to the initial surround view image, the historical chassis image, and the offset, includes: The ORB algorithm is used to extract the first image features of the initial surround view image and the second image features of the historical chassis image.

[0075] Based on the offset, feature matching is performed between the first image features and the second image features to obtain a set of matching points.

[0076] The matching point set is optimized based on the RANSAC algorithm to obtain multiple optimal matching pairs.

[0077] Based on the weighted fusion formula, an initial stitched image is generated according to the optimal matching pair.

[0078] Optionally, generating the initial stitched image based on the optimal matching pair according to the weighted fusion formula includes: Substitute each of the optimal matching pairs into the weighted fusion formula to obtain the initial stitched image.

[0079] The weighted fusion formula is as follows:

[0080]

[0081] Among them, w i For weights, d i The Euclidean distance between the matching points is given by σ, where σ is the scale parameter. , These are the two pixels of the optimal matching pair. The initial toroidal image, The historical chassis image, This refers to the initial stitched image.

[0082] The first and second image features can be the FAST keypoints and BRIEF descriptors of the initial surround view image and the historical chassis image, respectively. As an optional implementation, the first image features of the initial surround view image and the second image features of the historical chassis image can be extracted based on the ORB algorithm. Then, based on the determined offset of the initial surround view image relative to the historical vehicle surround view image, a matching relationship is established between the initial surround view image and the historical chassis image. The first and second image features are then matched to obtain a set of matching points. The matching point set is optimized using the RANSAC algorithm, eliminating erroneous matching points to obtain multiple optimal matching pairs. Each optimal matching pair is then input into a weighted fusion formula to obtain the initial stitched image.

[0083] In step S140, the initial stitched image is preprocessed, and the preprocessed initial stitched image is fused with the historical vehicle surround view image to obtain the current vehicle surround view image.

[0084] To make the vehicle surround view image more intuitive and clear to users, after obtaining the initial stitched image, the initial stitched image can be preprocessed, and then the preprocessed initial stitched image can be merged with the historical vehicle surround view image. Through 3D rendering and making the bottom of the vehicle transparent, the current vehicle surround view image is finally obtained and displayed on the vehicle's built-in display screen.

[0085] Optionally, the preprocessing of the initial stitched image includes: The initial stitched image is subjected to grayscale conversion and histogram equalization processing respectively.

[0086] The initial stitched image, after grayscale conversion and histogram equalization, undergoes color space conversion to complete the preprocessing of the initial stitched image.

[0087] After obtaining the initial stitched image, it can be converted to grayscale and histogram equalized, and then the color space can be converted to restore it to RGB space to complete the preprocessing of the initial stitched image.

[0088] Based on the same inventive concept, such as Figure 4 As shown in the figure, an embodiment of the present invention provides a vehicle surround view image generation device 300, a controller applied to a vehicle surround view image generation system, the vehicle surround view image generation system further including multiple image sensors, each of which is respectively installed at a preset position around the vehicle, and the controller is communicatively connected to each of the image sensors. The vehicle surround view image generation device 300 includes: The initial surround view image acquisition unit 301 is used to acquire the initial surround view image of the vehicle at the current moment based on each of the image sensors.

[0089] The offset determination unit 302 is used to determine the offset of the initial surround view image relative to the historical vehicle surround view image based on the image transformation matrix determined at the current time; wherein the historical vehicle surround view image is the vehicle surround view image generated in the previous cycle, and the historical vehicle surround view image includes a historical chassis image.

[0090] The initial stitched image determination unit 303 is used to obtain an initial stitched image based on the ORB algorithm and the RANSAC algorithm, respectively, according to the initial surround view image, the historical chassis image, and the offset.

[0091] The vehicle surround view image processing unit 304 is used to preprocess the initial stitched image and fuse the preprocessed initial stitched image with the historical vehicle surround view image to obtain the current vehicle surround view image.

[0092] Regarding the vehicle surround view image generation device 300 described above, the specific functions of each unit have been described in detail in the embodiments of the vehicle surround view image generation method provided in this specification, and will not be elaborated here.

[0093] Based on the same inventive concept, embodiments of this specification provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any of the methods in the aforementioned vehicle surround view image generation method.

[0094] The present invention has at least the following beneficial effects: 1. This invention determines the offset of the initial surround view image relative to the historical vehicle surround view image based on the image transformation matrix determined at the current moment. Then, based on the initial surround view image, the historical chassis image, and the offset, an initial stitched image is obtained, which in turn yields the current vehicle surround view image. Because the offset of the initial surround view image relative to the historical vehicle surround view image is determined and used to generate the initial stitched image, the solution of this invention can effectively solve the problems of image offset, tearing, or blurring caused by vehicle movement.

[0095] 2. By using the ORB and RANSAC algorithms, the matching accuracy between the initial surround view image and the historical chassis image was improved, while the matching point set was optimized, significantly reducing the computational complexity.

[0096] 3. By setting up a steering wheel angle sensor and a vehicle speed sensor, and determining the image transformation matrix based on the steering wheel angle, wheel speed pulse count, and tire unit pulse count, the stitching parameters are dynamically adjusted through a kinematic model, solving the problem that traditional static calibration cannot adapt to the dynamic driving of the vehicle.

[0097] 4. By preprocessing the initial stitched image and fusing the preprocessed initial stitched image with historical vehicle surround view images, the chassis area information of the current image is compensated by the historical vehicle surround view images, effectively eliminating the stepped shadow under the vehicle.

[0098] In the several embodiments provided by this invention, it should be understood that the disclosed apparatus and methods can also be implemented in other ways. The apparatus embodiments described above are merely illustrative; for example, the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments of the invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0099] In addition, the functional modules in the various embodiments of the present invention can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0100] If the aforementioned functions are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0101] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for generating a vehicle surround view image, characterized in that, A controller is applied to a vehicle surround view image generation system, the vehicle surround view image generation system further including multiple image sensors, each of the image sensors being installed at preset positions around the vehicle, and the controller being communicatively connected to each of the image sensors; the method includes: Based on each of the image sensors, an initial surround view image of the vehicle at the current moment is obtained; Based on the image transformation matrix determined at the current moment, the offset of the initial surround view image relative to the historical vehicle surround view image is determined; wherein, the historical vehicle surround view image is the vehicle surround view image generated in the previous cycle, and the historical vehicle surround view image includes the historical chassis image. An initial stitched image is obtained based on the ORB algorithm and the RANSAC algorithm, respectively, according to the initial surround view image, the historical chassis image, and the offset. The initial stitched image is preprocessed, and the preprocessed initial stitched image is fused with the historical vehicle surround view image to obtain the current vehicle surround view image.

2. The vehicle surround view image generation method as described in claim 1, characterized in that, The vehicle surround view image generation system further includes a steering wheel angle sensor and a vehicle speed sensor, and the controller is communicatively connected to the steering wheel angle sensor and the vehicle speed sensor respectively; the method further includes a step of determining the image transformation matrix, which includes: The steering wheel angle of the vehicle at the current moment is obtained based on the steering wheel angle sensor; The vehicle speed sensor is used to obtain the wheel speed pulse count and tire unit pulse count of the vehicle, where the tire unit pulse count is the total number of pulses for one revolution of the tire. The image transformation matrix is ​​determined based on the steering wheel angle, the wheel speed pulse count, and the tire unit pulse count.

3. The vehicle surround view image generation method as described in claim 2, characterized in that, Determining the image transformation matrix based on the steering wheel angle, the wheel speed pulse count, and the tire unit pulse count includes: Based on the front wheel sway angle calculation formula, the front wheel sway angle of the vehicle is obtained according to the steering wheel angle and the preset transmission ratio. The front wheel sway angle calculation formula is as follows: Wherein, α is the front wheel sway angle, β is the steering wheel angle, and i is the preset transmission ratio; Based on the turning radius calculation formula, the turning radius of the vehicle is obtained according to the vehicle width, wheelbase, and front wheel sway angle. The turning radius calculation formula is as follows: Where W is the vehicle width, D L Where r is the wheelbase and r is the turning radius; Based on the yaw angle calculation formula, the yaw angle of the vehicle is obtained according to the tire diameter, the wheel speed pulse count, the tire unit pulse count, and the turning radius. The yaw angle calculation formula is as follows: Among them, D r Where P is the tire diameter, N is the wheel speed pulse count, and θ is the tire unit pulse count; The image transformation matrix is ​​obtained based on the yaw angle.

4. The vehicle surround view image generation method as described in claim 3, characterized in that, The image transformation matrix is ​​represented as follows: Where T is the image transformation matrix.

5. The vehicle surround view image generation method as described in claim 1, characterized in that, The process of obtaining an initial stitched image based on the ORB algorithm and the RANSAC algorithm, respectively, according to the initial surround view image, the historical chassis image, and the offset, includes: Based on the ORB algorithm, the first image features of the initial surround view image and the second image features of the historical chassis image are extracted respectively. Based on the offset, feature matching is performed between the first image features and the second image features to obtain a set of matching points; The matching point set is optimized based on the RANSAC algorithm to obtain multiple optimal matching pairs; Based on the weighted fusion formula, an initial stitched image is generated according to the optimal matching pair.

6. The vehicle surround view image generation method as described in claim 5, characterized in that, The step of generating an initial stitched image based on the weighted fusion formula and the optimal matching pair includes: Substitute each of the optimal matching pairs into the weighted fusion formula to obtain the initial stitched image; The weighted fusion formula is as follows: Among them, w i For weights, d i The Euclidean distance between the matching points is given by σ, where σ is the scale parameter. , These are the two pixels of the optimal matching pair. The initial toroidal image, The historical chassis image, This refers to the initial stitched image.

7. The vehicle surround view image generation method as described in claim 1, characterized in that, The preprocessing of the initial stitched image includes: The initial stitched image is subjected to grayscale conversion and histogram equalization processing respectively; The initial stitched image, after grayscale conversion and histogram equalization, undergoes color space conversion to complete the preprocessing of the initial stitched image.

8. A vehicle surround view image generation device, characterized in that, A controller is applied to a vehicle surround view image generation system, the vehicle surround view image generation system further includes multiple image sensors, each of the image sensors is installed at a preset position around the vehicle, and the controller is communicatively connected to each of the image sensors; The vehicle surround view image generation device includes: An initial surround view image acquisition unit is used to acquire an initial surround view image of the vehicle at the current moment based on each of the image sensors. The offset determination unit is used to determine the offset of the initial surround view image relative to the historical vehicle surround view image based on the image transformation matrix determined at the current time; wherein, the historical vehicle surround view image is the vehicle surround view image generated in the previous cycle, and the historical vehicle surround view image includes a historical chassis image. The initial stitched image determination unit is used to obtain an initial stitched image based on the ORB algorithm and the RANSAC algorithm, respectively, according to the initial surround view image, the historical chassis image, and the offset. The vehicle surround view image processing unit is used to preprocess the initial stitched image and fuse the preprocessed initial stitched image with the historical vehicle surround view image to obtain the current vehicle surround view image.

9. An electronic device, characterized in that, include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the program, implements the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a computer program, which, when executed, controls the server where the computer-readable storage medium is located to implement the steps of the method according to any one of claims 1 to 7.

Images