Image processing method and device, vehicle, storage medium and program product

Abstract

The invention discloses an image processing method and device, a vehicle, a storage medium and a program product. Relates to the technical field of intelligent driving. According to the specific implementation scheme, the method comprises the steps that a processed current aerial view and a processed historical aerial view of a vehicle are obtained, and the visual range of the historical aerial view comprises a historical blind area of a chassis and a historical visual area outside the chassis; comparing the current aerial view with historical aerial views under different rotation and translation transformation values to obtain difference maps under different rotation and translation transformation values; selecting a target rotation and translation transformation value from the rotation and translation transformation values according to the pixel value of each difference image; and according to the target rotation translation transformation value, drawing a target area in the historical aerial view to a current blind area of the current aerial view. The target area of the historical aerial view is drawn to the blind area of the current aerial view, so that the dislocation phenomenon of the visible area and the blind area of the chassis is reduced, and the visualization of the blind area of the current chassis is realized.

Description

Technical Field

[0001] The present invention relates to the field of intelligent driving technology, and in particular to an image processing method, device, vehicle, storage medium, and program product. Background Art

[0002] Chassis transparency is also called blind spot filling. It mainly uses imaging technology to create a transparent chassis effect and visualize the chassis blind spots, thereby enhancing the panoramic view effect.

[0003] The current chassis transparency technology relies heavily on body signals provided by some motion sensors installed on the vehicle, such as wheel speed pulses and wheel angles. The vehicle's displacement vector is calculated based on these signals, and the chassis transparency is achieved through this displacement vector.

[0004] However, there will be errors between the results calculated from these vehicle body signals and the actual values, and these errors will gradually accumulate over time and mileage, causing misalignment between the transparent and non-transparent areas of the vehicle chassis. Summary of the invention

[0005] The present invention provides an image processing method, device, vehicle, storage medium, and program product, which reduce the misalignment of the visible area and blind area of ​​the vehicle chassis by drawing the target area of ​​the historical bird's-eye view to the blind area of ​​the current bird's-eye view, and realize the visualization of the blind area of ​​the current vehicle chassis.

[0006] According to a first aspect of the present invention, an image processing method is provided.

[0007] Acquire a processed current bird's-eye view of the vehicle and a processed historical bird's-eye view, wherein the visible range of the historical bird's-eye view includes a historical blind area of ​​the vehicle chassis and a historical visible area outside the vehicle chassis;

[0008] Comparing the current bird's-eye view with historical bird's-eye view images under different rotation and translation transformation values ​​to obtain difference images under different rotation and translation transformation values;

[0009] Selecting a target rotational translational transformation value from the rotational translational transformation values ​​according to the pixel values ​​of the difference images;

[0010] According to the target rotation and translation transformation value, the target area in the historical bird's-eye view is drawn to the current blind area of ​​the current bird's-eye view, the target area is the area in the historical bird's-eye view corresponding to the current blind area, and the target area includes part of the historical visible area.

[0011] According to a second aspect of the present invention, an image processing apparatus is provided.

[0012] An acquisition module, used to acquire a processed current bird's-eye view of the vehicle and a processed historical bird's-eye view, wherein the visible range of the historical bird's-eye view includes a historical blind area of ​​the vehicle chassis and a historical visible area outside the vehicle chassis;

[0013] A comparison module, used for comparing the current bird's-eye view with the historical bird's-eye view under different rotation and translation transformation values ​​to obtain a difference image under different rotation and translation transformation values;

[0014] A selection module, configured to select a target rotational translational transformation value from each of the rotational translational transformation values ​​according to the pixel values ​​of each of the difference images;

[0015] A drawing module is used to draw the target area in the historical bird's-eye view to the current blind area of ​​the current bird's-eye view according to the target rotation and translation transformation value, wherein the target area is the area in the historical bird's-eye view corresponding to the current blind area, and the target area includes part of the historical visible area.

[0016] According to a third aspect of the present invention, there is provided a vehicle, the vehicle comprising:

[0017] at least one processor; and

[0018] a memory communicatively connected to the at least one processor; wherein,

[0019] The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the image processing method described in any embodiment of the present invention.

[0020] According to a fourth aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the image processing method described in any embodiment of the present invention when executed.

[0021] According to a fifth aspect of the present invention, a computer program product is provided. The computer program product comprises a computer program. When the computer program is executed by a processor, the image processing method according to any one of the embodiments of the present invention is implemented.

[0022] The technical solution of the embodiment of the present invention obtains the historical bird's-eye view and the current bird's-eye view of the vehicle, compares the historical bird's-eye view with the current bird's-eye view under different rotational and translational transformation values, and finally draws the target area of ​​the historical bird's-eye view corresponding to the selected target rotational and translational transformation value to the blind spot area of ​​the current bird's-eye view, thereby realizing the visualization of the blind spot of the current vehicle chassis, wherein the target area used for drawing is transformed by the selected target rotational and translational transformation value, thereby reducing the misalignment phenomenon between the visible area and the blind spot area of ​​the vehicle chassis.

[0023] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0025] Figure 1A is a flowchart of an image processing method provided according to Embodiment 1 of the present invention;

[0026] Figure 1B It is a schematic diagram of a historical bird's-eye view and a current bird's-eye view at a certain moment when a vehicle is traveling according to the first embodiment of the present invention;

[0027] Figure 1C is a schematic diagram of a difference image between a current bird's-eye view and a historical bird's-eye view provided according to the first embodiment of the present invention;

[0028] Figure 1D is another schematic diagram of a difference image between a current bird's-eye view image and a historical bird's-eye view image provided according to the first embodiment of the present invention;

[0029] Figure 1E is a schematic diagram of a transparent vehicle chassis provided according to the first embodiment of the present invention;

[0030] Figure 2A is a flow chart of a method for obtaining a bird's-eye view provided according to Embodiment 2 of the present invention;

[0031] Figure 2B is a schematic diagram of a current bird's-eye view and a historical bird's-eye view provided according to the second embodiment of the present invention;

[0032] Figure 2C is a schematic diagram of the current bird's-eye view and the historical bird's-eye view before and after noise removal according to the second embodiment of the present invention;

[0033] Figure 3 is a structural schematic diagram of an image processing device provided according to Embodiment 3 of the present invention;

[0034] Figure 4 It is a block diagram of a vehicle provided according to a fourth embodiment of the present invention. DETAILED DESCRIPTION

[0035] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0036] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0037] Embodiment 1

[0038] Figure 1A A flowchart of an image processing method is provided for the first embodiment of the present invention. This embodiment is applicable to the case where a target area of ​​a historical bird's-eye view is drawn to a blind area of ​​a current bird's-eye view. The method can be executed by an image processing device, which can be implemented in the form of hardware and / or software, and can be configured in a vehicle. Figure 1A As shown, the method includes:

[0039] S110, obtaining a processed current bird's-eye view and a processed historical bird's-eye view of the vehicle, wherein the visible range of the historical bird's-eye view includes a historical blind area of ​​the vehicle chassis and a historical visible area outside the vehicle chassis.

[0040] In this embodiment, the bird's-eye view can be understood as an image obtained by looking down from a high place, usually showing the whole picture of an object in the form of a two-dimensional plane. The bird's-eye view of the vehicle can be captured by cameras around the vehicle body, and combined with the camera calibration data to form a bird's-eye view. The current bird's-eye view can be understood as the bird's-eye view of the vehicle at the current moment. The historical bird's-eye view can be understood as a bird's-eye view of a frame randomly selected at a historical moment during the movement of the vehicle. The visible range can be understood as the visible range. The visible range of the bird's-eye view refers to the size and boundaries of the area that can be clearly observed when looking down from a specific high angle. The chassis can be understood as a key component of a vehicle that is located at the bottom of the vehicle body. The historical blind area can be understood as the blind area blocked by the chassis in the historical bird's-eye view. The blind area can refer to an area that cannot be observed or is difficult to detect. The historical visible area can be understood as the visible area that is not blocked by the chassis in the historical bird's-eye view. The visible range may include a historical blind area and a historical visible area, wherein the historical blind area may be made transparent when blind area filling is performed on the historical bird's-eye view.

[0041] Specifically, when the vehicle is moving, the cameras around the vehicle body take images, which are combined with the camera calibration data to form a bird's-eye view. A certain frame of the bird's-eye view is selected and processed (such as blind spot filling) as a historical bird's-eye view. The historical bird's-eye view includes the historical blind spots blocked by the vehicle chassis and the historical visible areas not blocked by the vehicle chassis. Similarly, the current bird's-eye view is processed (such as denoising) to obtain the current bird's-eye view.

[0042] For example, Figure 1B Schematic diagram of the historical bird's-eye view and the current bird's-eye view at a certain moment. First, a bird's-eye view of a certain frame at a historical moment with blind area filling is selected, and the bird's-eye view is subjected to median filtering, Gaussian filtering and other fuzzy processing to filter noise, thereby obtaining a historical bird's-eye view 101, wherein the black area is the historical blind area 103, and the rest is the historical visible area 104. The bird's-eye view at the current moment is also subjected to median filtering, Gaussian filtering and other fuzzy processing to filter noise, thereby obtaining a current bird's-eye view 102.

[0043] S120: Compare the current bird's-eye view image with historical bird's-eye view images under different rotation and translation transformation values ​​to obtain difference images under different rotation and translation transformation values.

[0044] In this embodiment, the rotation and translation transformation values ​​can be understood as the two displacements of the vehicle in the horizontal direction along the length direction and along the width direction of the vehicle body when the vehicle moves, and the rotation angle generated when the vehicle moves. The difference image can be understood as an image obtained by taking the absolute value after pixel subtraction of two different frames of images.

[0045] Specifically, the historical bird's-eye view images obtained in S110 are transformed under different rotation and translation transformation values, and the multiple transformed historical bird's-eye view images are compared with the current bird's-eye view image, that is, a difference is performed to obtain a plurality of difference images under different rotation and translation transformation values.

[0046] Exemplarily, all rotational and translational transformation values ​​within a certain range are selected, the historical bird's-eye view images are subjected to corresponding rotational and translational transformations, and all transformed historical bird's-eye view images are differentiated from the current bird's-eye view, that is, pixel subtraction is performed and then the absolute value is taken to obtain a differential image under all rotational and translational transformation values ​​within a certain range.

[0047] Optionally, the rotational translation transformation value includes a rotation value and a translation value, and the current bird's-eye view is compared with the historical bird's-eye view under different rotational translation transformation values ​​to obtain a differential map under different rotational translation transformation values, including: traversing the rotation values ​​within the rotation range and the translation values ​​within the translation range; transforming the historical bird's-eye view based on the selected rotation value and the selected translation value to obtain a first transformed bird's-eye view; and differentiating the first transformed bird's-eye view and the current bird's-eye view to obtain a differential map under the corresponding rotational translation transformation value.

[0048] In this embodiment, the rotation range can be understood as all rotation angles generated when the vehicle moves within a certain period of time. The translation range can be understood as two displacements in the horizontal direction along the length direction of the vehicle body and along the width direction of the vehicle body when the vehicle moves within a certain period of time. The first transformed bird's-eye view can be understood as a set of rotation and translation transformation values ​​selected from all rotation ranges and translation ranges generated when the vehicle moves, and the historical bird's-eye view is transformed by the rotation and translation transformation values ​​to generate the bird's-eye view.

[0049] Specifically, the rotation value in the rotation range and the translation value in the translation range during the movement of the vehicle are obtained, the selected rotation value and translation value are used as a set of rotation and translation transformation values, and the historical bird's-eye view is transformed under the rotation and translation transformation value to obtain the first transformed bird's-eye view. The current bird's-eye view is differentiated from the first transformed bird's-eye view, and the obtained difference map is the difference map corresponding to the selected rotation and translation transformation value. Then, all rotation values ​​in the rotation range and all translation values ​​in the translation range during the movement of the vehicle are traversed, and the above steps are repeated to obtain the difference maps corresponding to different rotation and translation transformation values ​​for subsequent use.

[0050] Exemplarily, during the movement of the vehicle, the translation value includes two translation values ​​in the horizontal direction. The rotation value and the translation value (including the two translation values ​​in the horizontal direction) are used as a set of rotation and translation transformation values, and the historical bird's-eye view is transformed under the rotation and translation transformation value to obtain a first transformed bird's-eye view. The current bird's-eye view is differentiated from the first transformed bird's-eye view, that is, the two bird's-eye views are subjected to pixel subtraction and then the absolute value is taken to obtain a differential image. Among them, the two translation values ​​in the horizontal direction can be two components perpendicular to each other in the moving direction in a plane parallel to the ground, which can be regarded as the x-direction and the y-direction. For example, the x-direction can be the length direction of the vehicle. When the vehicle moves along the length direction of the vehicle body, it is a translation in the x-direction. The y-direction can be the width direction of the vehicle. When the vehicle moves along the width direction of the vehicle body, it is a translation in the y-direction.

[0051] Optionally, the rotation range is determined based on a rotation value used when the historical blind spot area is filled, and the translation range is determined based on a translation value used when the historical blind spot area is filled.

[0052] Specifically, during the movement of the vehicle, a set of rotation values ​​determined when multiple historical blind areas are filled is determined as a rotation range; or a rotation range is determined based on a rotation value determined when a historical blind area is filled, such as using the rotation value as a median to determine a rotation range of a set length. Similarly, a set of translation values ​​(including two translation values ​​in the horizontal direction) determined when multiple historical blind areas are filled is determined as a translation range; or a translation range is determined based on a translation value determined when a historical blind area is filled, such as using the translation value as a median to determine a translation range of a set length.

[0053] Optionally, the rotation range and the translation range are respectively associated with vehicle body signals.

[0054] In this embodiment, the vehicle body signal can be understood as a signal associated with the driving state of the vehicle, such as a wheel speed pulse and a wheel angle.

[0055] In each process of filling the current blind spot area, the values ​​of the rotation range and the translation range can be determined according to the vehicle body signal.

[0056] In one embodiment, the translation range can be determined based on the wheel speed pulses. The rotation range can be determined based on the wheel speed pulses and the wheel angle.

[0057] The specific determination method is not limited here. The larger the value corresponding to the vehicle body signal is, the larger the corresponding rotation range and translation range can be.

[0058] S130 . Selecting a target rotational translational transformation value from the rotational translational transformation values ​​according to the pixel values ​​of the difference images.

[0059] In this embodiment, the pixel value can be understood as the numerical information of each pixel in the image, and the pixel is the smallest unit of the image. The target rotational translation transformation value can be understood as a rotational translation value selected from multiple rotational translation values, which can make the historical bird's-eye view after the transformation based on the target rotational translation change value match (such as overlap) with the current bird's-eye view. The target rotational translation transformation value can be the target rotational translation transformation value corresponding to the minimum value selected from the sum of the pixel values ​​of multiple differential images.

[0060] Specifically, the sum of the pixel values ​​of the difference image under multiple different rotation and translation transformation values ​​is calculated, the minimum sum of the pixel values ​​of the difference image is selected, and the rotation and translation transformation value corresponding to the difference image is used as the target rotation and translation transformation value.

[0061] Exemplarily, the sum of the pixel values ​​of the differential image under all rotational and translational transformation values ​​during the movement of the vehicle is obtained, and the rotational and translational transformation value corresponding to the minimum sum of pixel values ​​is selected to obtain the target rotational and translational transformation value. Figure 1C , the left side is the simulated difference effect image 105, and the right side is the difference image 106. The brighter the pixel value, the larger the pixel value. Figure 1C There are obviously many diagonal lines in the image, which makes the pixel value of the whole image larger. Figure 1D , the left side is the simulated difference effect diagram 107, and the right side is the difference diagram 108. Similarly, the brighter the pixel value, the larger the pixel value. Figure 1D The brightness of the middle oblique line is not obvious, indicating that the pixel value of the whole image is moderate, so you can choose Figure 1D The rotation and translation change values ​​corresponding to the difference image are used as the target rotation and translation change values.

[0062] Optionally, based on the pixel values ​​of each of the differential images, a target rotational and translational transformation value is selected from each of the rotational and translational transformation values, including: for each of the differential images, determining the sum of the pixel values ​​in the differential image; selecting the minimum sum of pixel values ​​from the sums of the pixel values, and determining the rotational and translational transformation value corresponding to the selected sum of pixel values ​​as the target rotational and translational transformation value.

[0063] Specifically, the difference images corresponding to all rotational and translational transformation values ​​during the movement of the vehicle are obtained, the pixel values ​​of each difference image are summed, and the minimum sum of pixel values ​​is selected. The rotational and translational transformation value corresponding to the minimum sum of pixel values ​​is the target rotational and translational transformation value within the range.

[0064] Exemplarily, a differential image corresponding to all rotational and translational transformation values ​​during the movement of the vehicle is obtained, and the pixel values ​​of the differential image are obtained by introducing a programming language and an image processing library. The pixel values ​​of each differential image are summed, and the minimum sum of pixel values ​​is selected therefrom. The rotational and translational transformation value corresponding to the minimum sum of pixel values ​​is determined as the target rotational and translational transformation value during the movement of the vehicle.

[0065] S140. According to the target rotation and translation transformation value, draw the target area in the historical bird's-eye view to the current blind area of ​​the current bird's-eye view, wherein the target area is the area in the historical bird's-eye view corresponding to the current blind area, and the target area includes part of the historical visible area.

[0066] In this embodiment, the target area can be understood as consisting of a portion of the area in the historical bird's-eye view corresponding to the current blind area and a portion of the historical visible area. The current blind area can be understood as the blind area blocked by the vehicle chassis in the current bird's-eye view.

[0067] Specifically, according to the target rotation and translation transformation value selected in S130, the target area consisting of the area corresponding to the current blind area and a part of the historical visible area in the historical bird's-eye view is rotated and translated and drawn to the current blind area corresponding to the current bird's-eye view.

[0068] For example, Figure 1E This is a schematic diagram of the transparent chassis of the vehicle that has been realized in the prior art. It can be seen that there is a misalignment phenomenon in the image. According to the target rotation and translation transformation value corresponding to the minimum sum of pixel values ​​in a certain range, the target area in the historical bird's-eye view is rotated and translated and drawn to the current blind area corresponding to the current bird's-eye view, where the rotation and translation includes translation according to two translation values ​​in the horizontal direction and rotation according to the rotation angle generated when the vehicle moves.

[0069] Optionally, drawing the target area in the historical bird's-eye view to the current blind spot area of ​​the current bird's-eye view according to the target rotation and translation transformation value includes: transforming the historical bird's-eye view according to the target rotation and translation transformation value to obtain a second transformed bird's-eye view; and drawing the target area in the second transformed bird's-eye view to the current blind spot area of ​​the current bird's-eye view.

[0070] In this embodiment, the second transformed bird's-eye view can be understood as a bird's-eye view that can be used to implement blind spot filling. The second transformed bird's-eye view can be a bird's-eye view generated by transforming the historical bird's-eye view by the selected target rotation and translation transformation value.

[0071] Specifically, the target rotation and translation transformation value corresponding to the minimum sum of pixel values ​​during vehicle movement is obtained, the historical bird's-eye view is transformed according to the target rotation and translation transformation value to obtain a second transformed bird's-eye view, the target area corresponding to the current blind spot area in the second transformed bird's-eye view is obtained, and the target area is drawn to the current blind spot area of ​​the current bird's-eye view.

[0072] Exemplarily, a target rotation and translation transformation value corresponding to the minimum sum of pixel values ​​during vehicle movement is obtained, and the historical bird's-eye view is transformed according to the target rotation and translation transformation value, including translation according to two translation values ​​in the horizontal direction and rotation according to the rotation angle generated when the vehicle moves, to obtain a second transformed bird's-eye view. A target area consisting of an area corresponding to the current blind spot area and a portion of the historical visible area is obtained in the second transformed bird's-eye view, and the target area is drawn to the current blind spot area of ​​the current bird's-eye view.

[0073] The technical solution of the embodiment of the present invention obtains the historical bird's-eye view and the current bird's-eye view of the vehicle, compares the historical bird's-eye view with the current bird's-eye view under different rotational and translational transformation values, and finally draws the target area of ​​the historical bird's-eye view corresponding to the selected target rotational and translational transformation value to the blind spot area of ​​the current bird's-eye view, thereby realizing the visualization of the blind spot of the current vehicle chassis, wherein the target area used for drawing is transformed by the selected target rotational and translational transformation value, thereby reducing the misalignment phenomenon between the visible area and the blind spot area of ​​the vehicle chassis.

[0074] Embodiment 2

[0075] Figure 2A This is a flow chart of a method for obtaining a bird's-eye view provided in Embodiment 2 of the present invention. This embodiment is based on the method for obtaining a bird's-eye view in the above embodiment. Figure 2A As shown, the method includes:

[0076] S210: Obtain a historical bird's-eye view of the vehicle at a historical moment.

[0077] In this embodiment, the historical moment can be understood as a moment before the current moment when the vehicle is moving.

[0078] Specifically, at a certain historical moment during the movement of the vehicle, images are captured by cameras around the vehicle body, combined with camera calibration data to form a bird's-eye view, and processed (such as blind spot filling) as a historical bird's-eye view.

[0079] For example, Figure 2B The middle left image serves as a historical bird's eye view 201.

[0080] S220: Obtain the current bird's-eye view at the current moment.

[0081] Specifically, at the current moment, images are captured by cameras around the vehicle body and combined with camera calibration data to form a bird's-eye view, which is the current bird's-eye view.

[0082] For example, Figure 2B The image on the right side of the middle is used as the current bird's-eye view 202, and the bird's-eye view spliced ​​by the vehicle body camera and the camera calibration data at the current moment is used as the current bird's-eye view.

[0083] Optionally, obtaining the current bird's-eye view at the current moment includes: obtaining environmental images of the environment of the vehicle captured by multiple cameras of the vehicle; and splicing the environmental images to obtain the current bird's-eye view at the current moment.

[0084] Specifically, at the current moment, the environmental images of the vehicle's environment are captured by cameras around the vehicle body, and the multiple environmental images collected are spliced ​​in combination with the camera calibration data. The spliced ​​image is the current bird's-eye view at the current moment.

[0085] Exemplarily, at the current moment, cameras around the vehicle body capture environmental images of the vehicle's environment, including road conditions on which the vehicle body is located, and the captured environmental images are combined with camera calibration data to obtain the current bird's-eye view.

[0086] S230 , performing fuzzy processing on the historical bird's-eye view and the current bird's-eye view respectively to obtain the current bird's-eye view after noise filtering and the historical bird's-eye view after noise filtering.

[0087] In this embodiment, blur processing can be understood as a technology for processing images, the purpose of which is to reduce noise and prevent noise from affecting the final result. Blur processing can be completed by median filtering, mean filtering, Gaussian filtering, etc.

[0088] Specifically, the acquired historical bird's-eye view and the current bird's-eye view are subjected to fuzzy processing such as median filtering and Gaussian filtering to remove the influence of noise, thereby obtaining the historical bird's-eye view and the current bird's-eye view after the noise is filtered.

[0089] For example, the historical bird's-eye view and the current bird's-eye view are subjected to fuzzy processing such as median filtering and Gaussian filtering to obtain the current bird's-eye view after noise filtering and the historical bird's-eye view after noise filtering. Figure 2C , the left side is an unprocessed bird's-eye view image 203, and the right side is a bird's-eye view image 204 after fuzzy processing such as median filtering and Gaussian filtering to remove noise.

[0090] A method for obtaining a bird's-eye view provided in an embodiment of the present invention describes in detail the process of obtaining a historical bird's-eye view and a current bird's-eye view, and removes noise through fuzzy processing to obtain a historical bird's-eye view and a current bird's-eye view that can be used in an image processing method, which can be used in any embodiment of the present invention.

[0091] Embodiment 3

[0092] Figure 3 FIG. 1 is a schematic diagram of the structure of an image processing device provided in Embodiment 3 of the present invention. Figure 3 As shown, the device comprises:

[0093] The acquisition module 310 is used to acquire the processed current bird's-eye view of the vehicle and the processed historical bird's-eye view, wherein the visible range of the historical bird's-eye view includes the historical blind area of ​​the vehicle chassis and the historical visible area outside the vehicle chassis.

[0094] The comparison module 320 is used to compare the current bird's-eye view image with the historical bird's-eye view images under different rotation and translation transformation values ​​to obtain difference images under different rotation and translation transformation values.

[0095] The selection module 330 is used to select a target rotational translational transformation value from each of the rotational translational transformation values ​​according to the pixel value of each of the difference images.

[0096] The drawing module 340 is used to draw the target area in the historical bird's-eye view to the current blind spot area of ​​the current bird's-eye view according to the target rotation and translation transformation value, wherein the target area is the area in the historical bird's-eye view corresponding to the current blind spot area, and the target area includes part of the historical visible area.

[0097] The technical solution of the embodiment of the present invention completes the processing of the image through the mutual cooperation and collaborative work between the modules, obtains the historical bird's-eye view and the current bird's-eye view of the vehicle, and compares the historical bird's-eye view with the current bird's-eye view under different rotation and translation transformation values, and finally draws the target area of ​​the historical bird's-eye view corresponding to the selected target rotation and translation transformation value to the blind area of ​​the current bird's-eye view, thereby realizing the visualization of the blind area of ​​the current vehicle chassis, wherein the target area used for drawing is transformed by the selected target rotation and translation transformation value, thereby reducing the misalignment phenomenon of the visible area and the blind area of ​​the vehicle chassis.

[0098] In one embodiment, the rotation and translation transformation value includes a rotation value and a translation value, and the comparison module 320 is specifically used to:

[0099] Traverse the rotation values ​​within the rotation range and the translation values ​​within the translation range;

[0100] Transforming the historical bird's-eye view based on the selected rotation value and the selected translation value to obtain a first transformed bird's-eye view;

[0101] The first transformed bird's-eye view image and the current bird's-eye view image are differentiated to obtain a differential image under corresponding rotation and translation transformation values.

[0102] In one embodiment, the rotation range is determined based on the rotation value used when the historical blind spot area is filled, and the translation range is determined based on the translation value used when the historical blind spot area is filled.

[0103] In one embodiment, the rotation range and the translation range are respectively associated with vehicle body signals.

[0104] In one embodiment, the selection module 330 is specifically configured to:

[0105] For each of the difference images, determining the sum of the pixel values ​​in the difference image;

[0106] The smallest sum of pixel values ​​is selected from the sums of the pixel values, and the rotational translation transformation value corresponding to the selected sum of pixel values ​​is determined as the target rotational translation transformation value.

[0107] In one embodiment, the drawing module 340 is specifically configured to:

[0108] Transforming the historical bird's-eye view according to the target rotation and translation transformation value to obtain a second transformed bird's-eye view;

[0109] The target area in the second transformed bird's-eye view is drawn to the current blind area of ​​the current bird's-eye view.

[0110] In one embodiment, the acquisition module 310 includes:

[0111] A first acquisition unit, configured to acquire a historical bird's-eye view of the vehicle at a historical moment;

[0112] A second acquisition unit is used to acquire the current bird's-eye view at the current moment;

[0113] The processing unit is used to perform fuzzy processing on the historical bird's-eye view and the current bird's-eye view respectively to obtain the current bird's-eye view after noise filtering and the historical bird's-eye view after noise filtering.

[0114] In one embodiment, the second acquisition unit is specifically configured to:

[0115] Acquire environmental images of the environment in which the vehicle is located, collected by multiple cameras of the vehicle;

[0116] The environment images are stitched together to obtain a current bird's-eye view at the current moment.

[0117] Embodiment 4

[0118] According to an embodiment of the present invention, the present invention also provides a vehicle, a computer-readable storage medium, and a computer program product.

[0119] Figure 4It is a block diagram of a vehicle provided according to Embodiment 4 of the present invention, and the vehicle can implement the image processing method described in the embodiment of the present invention. The vehicle is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The vehicle can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0120] like Figure 4 As shown, the vehicle 410 includes at least one processor 411, and a memory connected to the at least one processor 411, such as a read-only memory (ROM) 412, a random access memory (RAM) 413, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 411 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 412 or the computer program loaded from the storage unit 418 to the random access memory (RAM) 413. In the RAM 413, various programs and data required for the operation of the vehicle 410 can also be stored. The processor 411, the ROM 412, and the RAM 413 are connected to each other via a bus 414. An input / output (I / O) interface 415 is also connected to the bus 414.

[0121] A number of components in the vehicle are connected to the I / O interface 415, including: an input unit 416, such as a keyboard, a mouse, etc.; an output unit 417, such as various types of displays, speakers, etc.; a storage unit 418, such as a disk, an optical disk, etc.; and a communication unit 419, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 419 allows the vehicle to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0122] The processor 411 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 411 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 411 performs the various methods and processes described above, such as an image processing method.

[0123] In some embodiments, the image processing method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 418. In some embodiments, part or all of the computer program may be loaded and / or installed on the vehicle 410 via the ROM 412 and / or the communication unit 419. When the computer program is loaded into the RAM 413 and executed by the processor 411, one or more steps of the image processing method described above may be performed. Alternatively, in other embodiments, the processor 411 may be configured to perform the image processing method in any other appropriate manner (e.g., by means of firmware).

[0124] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0125] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0126] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0127] To provide interaction with a user, the systems and techniques described herein can be implemented on a vehicle having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the vehicle. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0128] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0129] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0130] In some embodiments, the computer program product includes a computer program, and when the computer program is executed by a processor, the image processing method provided by the embodiment of the present invention is implemented.

[0131] The technical solution of the embodiment of the present invention obtains the historical bird's-eye view and the current bird's-eye view of the vehicle through an image processing method, device, vehicle, storage medium, and program product, and compares the historical bird's-eye view with the current bird's-eye view under different rotational and translational transformation values, and finally draws the target area of ​​the historical bird's-eye view corresponding to the selected target rotational and translational transformation value to the blind area of ​​the current bird's-eye view, thereby realizing the visualization of the blind area of ​​the current vehicle chassis, wherein the target area used for drawing is transformed by the selected target rotational and translational transformation value, thereby reducing the misalignment phenomenon between the visible area and the blind area of ​​the vehicle chassis.

[0132] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0133] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. An image processing method, characterized in that: include: Acquire a processed current bird's-eye view of the vehicle and a processed historical bird's-eye view, wherein the visible range of the historical bird's-eye view includes a historical blind area of ​​the vehicle chassis and a historical visible area outside the vehicle chassis; Comparing the current bird's-eye view image with historical bird's-eye view images under different rotation and translation transformation values ​​to obtain difference images under different rotation and translation transformation values; Selecting a target rotational translational transformation value from the rotational translational transformation values ​​according to the pixel values ​​of the difference images; According to the target rotation and translation transformation value, the target area in the historical bird's-eye view is drawn to the current blind area of ​​the current bird's-eye view, the target area is the area in the historical bird's-eye view corresponding to the current blind area, and the target area includes part of the historical visible area.

2. The method according to claim 1, characterized in that The rotation and translation transformation value includes a rotation value and a translation value, and the current bird's-eye view image is compared with the historical bird's-eye view image under different rotation and translation transformation values ​​to obtain a difference image under different rotation and translation transformation values, including: Traverse the rotation values ​​within the rotation range and the translation values ​​within the translation range; Transforming the historical bird's-eye view based on the selected rotation value and the selected translation value to obtain a first transformed bird's-eye view; The first transformed bird's-eye view image and the current bird's-eye view image are differentiated to obtain a differential image under corresponding rotation and translation transformation values.

3. The method according to claim 2, characterized in that The rotation range is determined based on a rotation value used when the historical blind spot area is filled, and the translation range is determined based on a translation value used when the historical blind spot area is filled.

4. The method according to claim 2, characterized in that: The rotation range and the translation range are respectively associated with vehicle body signals.

5. The method according to claim 1, characterized in that The step of selecting a target rotational translational transformation value from each of the rotational translational transformation values ​​according to the pixel values ​​of each of the difference images comprises: For each of the difference images, determining the sum of the pixel values ​​in the difference image; The smallest sum of pixel values ​​is selected from the sums of the pixel values, and the rotational translation transformation value corresponding to the selected sum of pixel values ​​is determined as the target rotational translation transformation value.

6. The method according to claim 1, characterized in that The step of drawing the target area in the historical bird's-eye view to the current blind area of ​​the current bird's-eye view according to the target rotation and translation transformation value comprises: Transforming the historical bird's-eye view according to the rotation and translation transformation values ​​corresponding to the target difference image to obtain a second transformed bird's-eye view; The target area in the second transformed bird's-eye view is drawn to the current blind area of ​​the current bird's-eye view.

7. The method according to claim 1, characterized in that The step of obtaining the current bird's-eye view processed by the vehicle and the processed historical bird's-eye view includes: Obtain a historical bird's-eye view of the vehicle at a historical moment; Get the current bird's-eye view of the current moment; The historical bird's-eye view and the current bird's-eye view are fuzzy processed respectively to obtain the current bird's-eye view after noise filtering and the historical bird's-eye view after noise filtering.

8. The method according to claim 7, characterized in that The step of obtaining the current bird's-eye view at the current moment includes: Acquire environmental images of the environment in which the vehicle is located, collected by multiple cameras of the vehicle; The environment images are stitched together to obtain a current bird's-eye view at the current moment.

9. An image processing device, characterized in that: include: An acquisition module, used to acquire a processed current bird's-eye view of the vehicle and a processed historical bird's-eye view, wherein the visual range of the historical bird's-eye view includes a historical blind area of ​​the vehicle chassis and a historical visible area outside the vehicle chassis; A comparison module, used for comparing the current bird's-eye view with the historical bird's-eye view under different rotation and translation transformation values ​​to obtain a difference image under different rotation and translation transformation values; A selection module, configured to select a target rotational translational transformation value from each of the rotational translational transformation values ​​according to the pixel values ​​of each of the difference images; A drawing module is used to draw the target area in the historical bird's-eye view to the current blind area of ​​the current bird's-eye view according to the target rotation and translation transformation value, wherein the target area is the area in the historical bird's-eye view corresponding to the current blind area, and the target area includes part of the historical visible area.

10. A vehicle, characterized in that: The vehicle comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can perform the image processing method according to any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the image processing method according to any one of claims 1 to 8 when executed.

12. A computer program product, characterized in that The computer program product comprises a computer program, which, when executed by a processor, implements the image processing method according to any one of claims 1 to 8.

Images