Image processing method and device, computer equipment and computer readable storage medium

By combining the surround view camera and the rear view camera to acquire images to realize the functions of the electronic rearview mirror camera and/or the side rearview camera, the cost increase in the prior art caused by additional installation of the camera is solved, reducing hardware and debugging costs.

CN118945494APending Publication Date: 2024-11-12TCL YUXIN ZHIXING TECHNOLOGY (NINGBO) CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411082487.2
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-06
Publication Date
2024-11-12

AI Technical Summary

Technical Problem

Existing vehicles require additional installation of side rearview cameras and/or electronic rearview mirror cameras when acquiring rearview images, resulting in increased hardware costs and commissioning costs.

Method used

By combining the surround view camera and the rearview camera, images are captured to realize the functions of the electronic rearview mirror camera and/or side rearview camera by utilizing the extreme detection distance of the surround view camera and the resolution of the rearview camera, avoiding additional camera installation.

Benefits of technology

The functions of electronic rearview mirror cameras and/or side rearview cameras are realized, reducing hardware costs and debugging costs.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118945494A_ABST
    Figure CN118945494A_ABST
Patent Text Reader

Abstract

The invention discloses an image processing method and device, computer equipment and a computer readable storage medium, and the method comprises the steps: determining a limit detection distance of a surround-view camera and a second resolution of a rear-view camera based on a first resolution and a horizontal visual angle of the surround-view camera; acquiring a first image through a surround-view camera and acquiring a second image through a rear-view camera meeting a second resolution ratio; and determining a rearview image based on the limit detection distance, the first image and the second image. By adopting the method provided by the invention, the functions of the electronic rearview mirror camera and / or the side rearview mirror camera can be realized by combining the surround-view camera and the rearview camera, and the hardware cost and the debugging cost caused by additionally installing the side rearview mirror camera and / or the electronic rearview mirror camera are / is avoided.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image processing method, apparatus, computer equipment and computer-readable storage medium. Background Art

[0002] In order to obtain rear-view images, existing vehicles need to separately install side rear-view cameras and / or electronic rear-view mirror cameras. This design method requires additional side rear-view cameras and / or electronic rear-view mirror cameras, which not only increases the hardware cost of the vehicle, but also increases the debugging cost of the side rear-view cameras and / or electronic rear-view mirror cameras. Summary of the invention

[0003] The embodiments of the present application provide an image processing method, apparatus, computer device and computer-readable storage medium, which can combine a surround view camera and a rear view camera to realize the functions of an electronic rearview mirror camera and / or a side rearview camera, thereby avoiding the hardware cost and debugging cost caused by the additional installation of a side rearview camera and / or an electronic rearview mirror camera.

[0004] The technical solution adopted by the present invention to solve the problem is as follows:

[0005] In one aspect, the present application provides an image processing method, comprising:

[0006] Determining a limit detection distance of the surround-view camera and a second resolution of the rear-view camera based on a first resolution and a horizontal viewing angle of the surround-view camera;

[0007] The first image is acquired by a surround-view camera and the second image is acquired by a rear-view camera meeting a second resolution;

[0008] A rear view image is determined based on the limit detection distance, the first image, and the second image.

[0009] In some embodiments of the present application, determining the limit detection distance of the surround view camera and the second resolution of the rear view camera based on the first resolution and horizontal viewing angle of the surround view camera includes:

[0010] Determine a limit detection distance of the surround view camera based on a first resolution and a horizontal viewing angle of the surround view camera;

[0011] Based on the limit detection distance, determine the horizontal viewing angle range of the rearview camera;

[0012] A second resolution of the rear view camera is determined based on the horizontal viewing angle range and the detection distance range of the rear view camera.

[0013] In some embodiments of the present application, determining the horizontal viewing angle range of the rearview camera based on the limit detection distance includes:

[0014] Determining a first horizontal distance, a first vertical distance, and a second vertical distance based on a digital model file of the vehicle, a first theoretical installation position of a surround view camera, a second theoretical installation position of a rear view camera, and a third theoretical installation position of a side rear view camera;

[0015] Based on the first horizontal distance, determining a detection field length range of the rearview camera;

[0016] Determine a rearview detection distance range based on the first vertical distance, the second vertical distance and the limit detection distance;

[0017] Based on the detection field of view length range and the rearview detection distance range, the horizontal viewing angle range of the rearview camera is determined.

[0018] In some embodiments of the present application, determining a rear view image based on the limit detection distance, the first image, and the second image includes:

[0019] Performing conversion processing on the first image to obtain a converted first image;

[0020] Performing conversion processing on the second image to obtain a converted second image;

[0021] Within the limit detection distance, determining the converted first image as a rear-view image;

[0022] Outside the limit detection distance, the converted second image is determined as a rearview image.

[0023] In some embodiments of the present application, performing conversion processing on the first image to obtain the converted first image includes:

[0024] Obtaining distortion parameters of the surround view camera and distances from pixels in the first image to an image center point of the first image;

[0025] Performing distortion correction on the pixel points in the first image based on the distortion parameter and the distance between the pixel points in the first image and the image center point of the first image to obtain a first image after distortion correction;

[0026] The distortion-corrected first image is converted from the image coordinate system to the camera coordinate system to obtain a third image;

[0027] The third image is transformed from the camera coordinate system to the world coordinate system to obtain a transformed first image.

[0028] In some embodiments of the present application, converting the distortion-corrected first image from an image coordinate system to a camera coordinate system to obtain a third image includes:

[0029] Obtaining the length of the pixel point in the first image after distortion correction on the imaging plane and the coordinates of the origin of the image coordinate system in the camera coordinate system;

[0030] The pixel points in the first image after distortion correction are converted based on the length of the pixel points in the first image after distortion correction on the imaging plane and the coordinates of the origin of the image coordinate system in the camera coordinate system to obtain a third image.

[0031] In some embodiments of the present application, converting the third image from the camera coordinate system to the world coordinate system to obtain the converted first image includes:

[0032] Get the depth value of the point in the world coordinate system, the intrinsic parameter matrix and extrinsic parameter matrix of the surround camera;

[0033] The pixel points in the third image are converted based on the depth value, the intrinsic parameter matrix and the extrinsic parameter matrix to obtain a converted first image.

[0034] In a second aspect, an embodiment of the present invention further provides an image processing device, including:

[0035] An information determination module, configured to determine a limit detection distance of the surround view camera and a second resolution of the rear view camera based on a first resolution and a horizontal viewing angle of the surround view camera;

[0036] An image acquisition module, configured to acquire a first image through a surround view camera and a second image through a rear view camera meeting a second resolution;

[0037] The image processing module is used to determine a rear view image based on the limit detection distance, the first image and the second image.

[0038] In a third aspect, the present application further provides a computer device, the computer device comprising:

[0039] one or more processors;

[0040] Memory; and

[0041] One or more application programs, wherein the one or more application programs are stored in the memory and configured to be executed by the processor to implement any image processing method in the first aspect.

[0042] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, and the computer program is loaded by a processor to execute the steps in any one of the image processing methods in the first aspect.

[0043] The beneficial effects of the present invention are as follows: by collecting a first image with a surround-view camera and collecting a second image with a rear-view camera that meets a second resolution, the rear-view image is determined based on the maximum detection distance of the surround-view camera, the first image and the second image. The wide viewing angle and close-range detection of the surround-view camera can be combined with the conventional viewing angle and long-range detection of the rear-view camera to realize the functions of an electronic rearview mirror camera and / or a side rearview camera, thereby avoiding the hardware cost and debugging cost caused by the additional installation of side rearview cameras and / or electronic rearview mirror cameras. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0045] Figure 1 is a schematic diagram of a flow chart of an image processing system provided by an embodiment of the present invention;

[0046] Figure 2 is a flowchart of an embodiment of an image processing method provided by an embodiment of the present invention;

[0047] Figure 3 is a flowchart of a specific embodiment of determining the limit detection distance and the second resolution provided by an embodiment of the present invention;

[0048] Figure 4 is a flowchart of a specific embodiment of determining a rear-view image provided by an embodiment of the present invention;

[0049] Figure 5 is a principle block diagram of an image processing device provided by an embodiment of the present invention;

[0050] Figure 6 It is a schematic diagram of the structure of a computer device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0051] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of this application.

[0052] In the description of this application, the terms "first", "second", "third", etc. are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, features defined as "first", "second", "third", etc. may explicitly or implicitly include one or more of the features. In the description of this application, "plurality" means two or more, unless otherwise clearly and specifically defined.

[0053] In this application, the word "exemplary" is used to mean "used as an example, illustration, or description." Any embodiment described in this application as "exemplary" is not necessarily to be construed as being preferred or advantageous over other embodiments. The following description is given to enable any technician in the field to implement and use the present application. In the following description, details are listed for the purpose of explanation. It should be understood that a person of ordinary skill in the art can recognize that the present application can be implemented without using these specific details. In other instances, well-known structures and processes will not be elaborated in detail to avoid obscuring the description of the present application with unnecessary details. Therefore, the present application is not intended to be limited to the embodiments shown, but is consistent with the widest scope consistent with the principles and features disclosed in the present application.

[0054] It should be noted that since the method of the embodiment of the present application is executed in a computer device, the processing objects of each computer device exist in the form of data or information. For example, time is actually time information. It can be understood that if size, quantity, position, etc. are mentioned in subsequent embodiments, they are all corresponding data for processing by the computer device. The details will not be repeated here.

[0055] The embodiments of the present application provide an image processing method, apparatus, computer device, and computer-readable storage medium, which are described in detail below.

[0056] See also Figure 1 , Figure 1 Schematic diagram of a scene of an image processing system provided in an embodiment of the present application. The image processing system may include a computer device 100, in which an image processing device is integrated, such as Figure 1 Computer equipment in.

[0057] In the embodiment of the present application, the computer device 100 is mainly used to determine the maximum detection distance of the surround-view camera and the second resolution of the rear-view camera based on the first resolution and horizontal viewing angle of the surround-view camera; collect a first image by the surround-view camera and collect a second image by the rear-view camera that meets the second resolution; determine the rear-view image based on the maximum detection distance, the first image and the second image, and can combine the surround-view camera and the rear-view camera to realize the functions of the electronic rearview mirror camera and / or the side rearview camera, thereby avoiding the hardware cost and debugging cost caused by the additional installation of the side rearview camera and / or the electronic rearview mirror camera.

[0058] In the embodiment of the present application, the computer device 100 may be an independent server, or a server network or server cluster composed of servers. For example, the computer device 100 described in the embodiment of the present application includes but is not limited to a computer, a network host, a single network server, a plurality of network server sets or a cloud server composed of a plurality of servers. The cloud server is composed of a large number of computers or network servers based on cloud computing.

[0059] It is understandable that the computer device 100 used in the embodiment of the present application may be a device including both receiving and transmitting hardware, that is, a device having receiving and transmitting hardware capable of performing two-way communication on a two-way communication link. Such a device may include: a cellular or other communication device having a single-line display or a multi-line display or a cellular or other communication device without a multi-line display. The computer device 100 may specifically be a desktop terminal or a mobile terminal, and the computer device 100 may specifically be a central control device of a vehicle, which is not limited in this embodiment.

[0060] Those skilled in the art will understand that Figure 1 The application environment shown in the figure is only one application scenario of the present application solution and does not constitute a limitation on the application scenario of the present application solution. Other application environments may also include Figure 1 More or less computer equipment as shown in Figure 1 Only one computer device is shown in the figure. It can be understood that the image processing system can also include one or more other services, which are not specifically limited here.

[0061] In addition, if Figure 1 As shown, the image processing system may further include a memory 200 for storing data, such as image data, such as a first image, a second image, etc., such as resolution information, such as a first resolution, a second resolution, etc.

[0062] It should be noted that Figure 1The scene diagram of the image processing system shown is only an example. The image processing system and scene described in the embodiment of the present application are intended to more clearly illustrate the technical solution of the embodiment of the present application, and do not constitute a limitation on the technical solution provided in the embodiment of the present application. Ordinary technicians in this field can know that with the evolution of the image processing system and the emergence of new business scenarios, the technical solution provided in the embodiment of the present application is also applicable to similar technical problems.

[0063] First, an image processing method is provided in an embodiment of the present application. The executor of the image processing method is an image processing device, and the image processing device is applied to a computer device. The image processing method includes: determining the maximum detection distance of the surround-view camera and the second resolution of the rear-view camera based on the first resolution and horizontal viewing angle of the surround-view camera; collecting a first image by the surround-view camera and collecting a second image by the rear-view camera that meets the second resolution; and determining the rear-view image based on the maximum detection distance, the first image and the second image.

[0064] like Figure 2 FIG. 2 is a flow chart of an embodiment of an image processing method in an embodiment of the present application. The image processing method may include the following steps S201 to S203, which are as follows:

[0065] Step S201: Determine the limit detection distance of the surround view camera and the second resolution of the rear view camera based on the first resolution and horizontal viewing angle of the surround view camera.

[0066] In a specific embodiment, the surround-view camera can be selected according to actual needs, and the first resolution and horizontal viewing angle are the resolution and horizontal viewing angle of the surround-view camera selected by the user. For example, when the user selects a surround-view camera with a resolution of 3 million pixels and a horizontal viewing angle of 196°, the first resolution of the surround-view camera is 3 million pixels and the horizontal viewing angle is 196°.

[0067] Furthermore, the limit detection distance of the surround-view camera is the limit distance that the surround-view camera can detect. For example, the limit detection distance of the surround-view camera is 9m, which indicates that the surround-view camera can detect images within a distance of 9m. The second resolution is the resolution that the rear-view camera needs to meet, which is determined based on the first resolution and horizontal viewing angle of the surround-view camera. In this embodiment, the second resolution of the rear-view camera is determined based on the first resolution and horizontal viewing angle of the surround-view camera, and the rear-view camera is selected based on the determined second resolution, which can ensure that the detection field of view of the rear-view camera covers the field of view within the detection distance limit of the surround-view camera.

[0068] Step S202: capturing a first image through a surround view camera and capturing a second image through a rear view camera that meets a second resolution.

[0069] This embodiment determines the maximum detection distance of the surround-view camera and the second resolution of the rear-view camera based on the first resolution and horizontal viewing angle of the surround-view camera. By capturing the first image with the surround-view camera and selecting a rear-view camera that meets the second resolution to capture the second image, it can be ensured that the detection field of view of the rear-view camera covers the field of view within the detection distance limit of the surround-view camera.

[0070] Step S203: determining a rear view image based on the limit detection distance, the first image and the second image.

[0071] In a specific embodiment, the rearview image may include a side rearview image and / or an electronic rearview mirror image. This embodiment determines the rearview image based on the limit detection distance, the first image and the second image, and can simulate the image of the side rearview camera and / or the image of the electronic rearview mirror camera in combination with the first image captured by the surround view camera and the second image captured by the rearview camera. The vehicle does not need to additionally install a side rearview camera and / or an electronic rearview mirror camera to realize the functions of the side rearview camera and / or the electronic rearview mirror camera, thereby avoiding the hardware cost and debugging cost caused by the additional installation of the side rearview camera and / or the electronic rearview mirror camera.

[0072] In a specific implementation, referring to Figure 3 As shown, in step S201, determining the limit detection distance of the surround view camera based on the first resolution and horizontal viewing angle of the surround view camera may include steps S301 to S303, which are specifically as follows:

[0073] Step S301: Determine the limit detection distance of the surround view camera based on the first resolution and horizontal viewing angle of the surround view camera.

[0074] In a specific embodiment, based on the first resolution and horizontal viewing angle of the surround view camera, the step of determining the limit detection distance of the surround view camera specifically includes: obtaining preset first mapping information; the first mapping information represents the mapping relationship between the first resolution, the horizontal viewing angle and the limit detection distance; based on the first resolution, the horizontal viewing angle and the first mapping information of the surround view camera, determining the limit detection distance of the surround view camera. For example, in the first mapping information, 3 million pixels and a horizontal viewing angle of 196° correspond to a limit detection distance of 9m. When the first resolution of the surround view camera is 3 million pixels and the horizontal viewing angle is 196°, the limit detection distance of the surround view camera is determined to be 9m.

[0075] Step S302: Determine the horizontal viewing angle range of the rear view camera based on the limit detection distance.

[0076] In a specific embodiment, the step of determining the horizontal viewing angle range of the rearview camera based on the limit detection distance specifically includes: determining the detection field of view length range and the rearview detection distance range of the rearview camera based on the limit detection distance; determining the horizontal viewing angle range of the rearview camera based on the detection field of view length range and the rearview detection distance range.

[0077] In a specific embodiment, the detection field of view length range of the rearview camera is the horizontal length range of the detection field of view of the rearview camera. Based on the limit detection distance, the steps of determining the detection field of view length range of the rearview camera and the rearview detection distance range specifically include: determining the first horizontal distance, the first vertical distance and the second vertical distance based on the digital model file of the vehicle, the first theoretical installation position of the surround view camera, the second theoretical installation position of the rearview camera and the third theoretical installation position of the side rearview camera; determining the detection field of view length range of the rearview camera based on the first horizontal distance; determining the rearview detection distance range based on the first vertical distance, the second vertical distance and the limit detection distance.

[0078] The first horizontal distance represents the horizontal distance between the optical center of the rear view camera and the optical center of the side rear view camera, the first vertical distance represents the vertical distance between the optical center of the rear view camera and the optical center of the side rear view camera, and the second vertical distance represents the vertical distance between the optical center of the surround view camera and the optical center of the side rear view camera. When determining the first horizontal distance, the first vertical distance, and the second vertical distance based on the digital model file of the vehicle, the first theoretical installation position of the surround view camera, the second theoretical installation position of the rear view camera, and the third theoretical installation position of the side rear view camera, the 3D dimension drawings of each camera can be assembled onto the 3D dimension drawing of the vehicle based on the first theoretical installation position, the second theoretical installation position, and the third theoretical installation position, and then the first horizontal distance, the first vertical distance, and the second vertical distance can be measured by 3D software.

[0079] In a specific implementation example, the process of determining the detection field of view length range can be expressed as: Y>=2*(Y1+3), where Y represents the detection field of view length, and Y1 represents the first horizontal distance.

[0080] In a specific implementation example, based on the first vertical distance, the second vertical distance and the limit detection distance, the step of determining the rearview detection distance range specifically includes: subtracting the first vertical distance from the second vertical distance to obtain the third vertical distance; subtracting the limit detection distance from the third vertical distance to obtain the rearview detection distance range. The process of determining the rearview detection distance range can be expressed as: d<(S-(L1-L2)), d represents the rearview detection distance, S represents the limit detection distance, L1 represents the first vertical distance, and L2 represents the second vertical distance.

[0081] It should be noted that, in this embodiment, the second horizontal distance, the fourth vertical distance and the fifth vertical distance can also be determined based on the digital model file of the vehicle, the first theoretical installation position of the surround view camera, the second theoretical installation position of the rear view camera and the fourth theoretical installation position of the electronic rear view mirror camera, and the detection field length range and the rear view detection distance range of the rear view camera can be determined based on the second horizontal distance, the fourth vertical distance, the fifth vertical distance and the limit detection distance. When the detection field length range and the rear view detection distance range of the rear view camera are determined based on the second horizontal distance, the fourth vertical distance, the fifth vertical distance and the limit detection distance, the rear view image finally determined is the electronic rear view mirror image; when the detection field length range and the rear view detection distance range of the rear view camera are determined based on the first horizontal distance, the first vertical distance, the second vertical distance and the limit detection distance, the rear view image finally determined is the side rear view image.

[0082] Among them, the second horizontal distance represents the horizontal distance between the optical center of the rearview camera and the optical center of the electronic rearview mirror camera, the fourth vertical distance represents the vertical distance between the optical center of the rearview camera and the optical center of the electronic rearview mirror camera, and the fifth vertical distance represents the vertical distance between the optical center of the surround view camera and the optical center of the electronic rearview mirror camera. The process of determining the second horizontal distance, the fourth vertical distance and the fifth vertical distance based on the digital model file of the vehicle, the first theoretical installation position of the surround view camera, the second theoretical installation position of the rear view camera and the fourth theoretical installation position of the electronic rearview mirror camera is the same as the process of determining the first horizontal distance, the first vertical distance and the second vertical distance based on the digital model file of the vehicle, the first theoretical installation position of the surround view camera, the second theoretical installation position of the rear view camera and the third theoretical installation position of the side rearview camera. For details, reference can be made to the process of determining the first horizontal distance, the first vertical distance and the second vertical distance based on the digital model file of the vehicle, the first theoretical installation position of the surround view camera, the second theoretical installation position of the rear view camera and the third theoretical installation position of the side rearview camera. To avoid repetition, this application will not repeat it here.

[0083] In a specific embodiment, the process of determining the horizontal viewing angle range of the rearview camera based on the detection field of view length range and the rearview detection distance range can be expressed as: tg(c / 2)=Y / 2d, wherein tg() represents a tangent function, c represents the horizontal viewing angle, Y represents the detection field of view length, and d represents the rearview detection distance range. The horizontal viewing angle range calculated by the above formula can retain an error range, for example, retain an error range of +3°.

[0084] Step S303: determining a second resolution of the rear view camera based on the horizontal viewing angle range and the detection distance range of the rear view camera.

[0085] In a specific embodiment, based on the horizontal viewing angle range and the detection distance range of the rear-view camera, the step of determining the second resolution of the rear-view camera specifically includes: obtaining preset second mapping information; the second mapping information represents the mapping relationship between the horizontal viewing angle, the detection distance range of the rear-view camera and the second resolution; based on the horizontal viewing angle range and the detection distance range of the rear-view camera and the second mapping information, determining the second resolution of the rear-view camera. For example, in the second mapping information, the detection distance range of the rear-view camera is greater than 30m, the horizontal viewing angle of the rear-view camera is 70°, and the second resolution corresponds to 5 million pixels. When the detection distance range of the rear-view camera is greater than 30m and the horizontal viewing angle of the rear-view camera is 70°, the second resolution of the rear-view camera is determined to be 5 million pixels.

[0086] In a specific implementation, referring to Figure 4 As shown, in step S203, determining the rear view image based on the limit detection distance, the first image and the second image may include steps S401 to S404, which are specifically as follows:

[0087] Step S401: convert the first image to obtain a converted first image.

[0088] In a specific embodiment, converting the first image refers to converting the first image from an image coordinate system to a world coordinate system, and the steps of converting the first image to obtain the converted first image specifically include: performing distortion correction on the first image to obtain the distortion-corrected first image; converting the distortion-corrected first image from the image coordinate system to the camera coordinate system to obtain the third image; and converting the third image from the camera coordinate system to the world coordinate system to obtain the converted first image.

[0089] In a specific embodiment, the step of performing distortion correction on the first image to obtain the distortion-corrected first image specifically includes: obtaining the distortion parameters of the surround-view camera and the distance from the pixel points in the first image to the image center point of the first image; performing distortion correction on the pixel points in the first image based on the distortion parameters and the distance from the pixel points in the first image to the image center point of the first image to obtain the distortion-corrected first image.

[0090] The distortion parameters of the surround view camera include radial distortion parameters and tangential distortion parameters of the surround view camera. The distortion parameters of the surround view camera can be obtained by the camera supplier through internal calibration during manufacturing. The process of performing distortion correction on the pixels in the first image can be expressed as: (x, y) represents the coordinates of the pixel point in the first image, (x distorted ,y distorted) represents the coordinates of (x, y) after distortion correction, k1, k2, k3 represent radial distortion parameters, p1, p2 represent tangential distortion parameters, and r represents the distance from the pixel point (x, y) to the image center point of the first image.

[0091] In a specific embodiment, the step of converting the first image after distortion correction from the image coordinate system to the camera coordinate system to obtain the third image specifically includes: obtaining the length of the pixel point in the first image after distortion correction on the imaging plane and the coordinates of the origin of the image coordinate system in the camera coordinate system; based on the length of the pixel point in the first image after distortion correction on the imaging plane and the coordinates of the origin of the image coordinate system in the camera coordinate system, converting the pixel point in the first image after distortion correction to obtain the third image.

[0092] The length of the pixel point in the first image after distortion correction on the imaging plane includes the physical length dx of the pixel point in the first image after distortion correction on the imaging plane in the x direction and the physical length dy of the pixel point in the first image after distortion correction on the imaging plane in the y direction, and the units of dx and dy are mm. The process of converting the pixel points in the first image after distortion correction can be expressed as: (x distorted ,y distorted ) represents the coordinates of the pixel point in the first image after distortion correction, (u,v) represents (x distorted ,y distorted ) is the coordinate after transformation, dx represents the pixel point (x distorted ,y distorted ) in the x direction of the imaging surface, dy represents the physical length of the pixel point (x distorted ,y distorted ) is the physical length in the y direction of the imaging surface, and (u0,v0) represents the coordinates of the origin of the image coordinate system in the camera coordinate system.

[0093] In a specific embodiment, the step of converting the third image from the camera coordinate system to the world coordinate system to obtain the converted first image specifically includes: obtaining the depth value of the point in the world coordinate system, the intrinsic parameter matrix and the extrinsic parameter matrix of the surround camera; converting the pixel points in the third image based on the depth value, the intrinsic parameter matrix and the extrinsic parameter matrix to obtain the converted first image.

[0094] Among them, the intrinsic parameter matrix of the surround view camera can be obtained by the camera supplier through the intrinsic parameter calibration during manufacturing, and the extrinsic parameter matrix of the surround view camera can be obtained by a variety of methods. For example, the extrinsic parameter matrix of the surround view camera can be obtained by external parameter calculation based on the 3D digital model file of the vehicle and the theoretical installation positions of the surround view camera and the rear view camera on the vehicle. The extrinsic parameter matrix of the surround view camera can also be obtained by installing the surround view camera and the rear view camera on the vehicle and calibrating them by the existing Zhang Zhengyou calibration method, which is not limited in this embodiment.

[0095] In a specific embodiment, the process of converting the pixels in the third image based on the depth value, the intrinsic parameter matrix and the extrinsic parameter matrix can be expressed as: Among them, Z c represents the depth value of the point in the world coordinate system, (u, v) represents the coordinates of the pixel point in the third image, and dx represents the pixel point (x) in the first image after distortion correction. distorted ,y distorted ) in the x direction of the imaging surface, dy represents the physical length of the pixel point (x distorted ,y distorted ) is the physical length in the y direction of the imaging surface, (u0,v0) represents the coordinates of the origin of the image coordinate system in the camera coordinate system, f represents the focal length of the surround camera, represents the internal parameter matrix, represents the external parameter matrix, R represents the rotation matrix of the surround camera, T represents the translation matrix of the surround camera, (X w , Y w , Z w ) represents the coordinates of (u,v) in the camera coordinate system.

[0096] Step S402: convert the second image to obtain a converted second image.

[0097] In a specific embodiment, converting the second image refers to converting the second image from an image coordinate system to a world coordinate system, and the steps of converting the second image to obtain a converted second image specifically include: performing distortion correction on the second image to obtain a distortion-corrected second image; converting the distortion-corrected second image from the image coordinate system to a camera coordinate system to obtain a fourth image; and converting the fourth image from the camera coordinate system to a world coordinate system to obtain a converted second image.

[0098] Among them, the process of performing distortion correction on the second image to obtain the distortion-corrected second image is the same as the process of performing distortion correction on the first image to obtain the distortion-corrected first image. Specifically, please refer to the process of performing distortion correction on the first image to obtain the distortion-corrected first image. To avoid repetition, this application will not go into details here.

[0099] Furthermore, the second image after distortion correction is converted from the image coordinate system to the camera coordinate system to obtain the fourth image, which is the same as the step of converting the first image after distortion correction from the image coordinate system to the camera coordinate system to obtain the third image. Specifically, refer to the process of converting the first image after distortion correction from the image coordinate system to the camera coordinate system to obtain the third image. To avoid repetition, this application will not go into details here.

[0100] In a specific embodiment, the process of converting the fourth image from the camera coordinate system to the world coordinate system to obtain the converted second image is the same as the process of converting the third image from the camera coordinate system to the world coordinate system to obtain the converted first image. For details, please refer to the process of converting the third image from the camera coordinate system to the world coordinate system to obtain the converted first image. To avoid repetition, this application will not go into details here.

[0101] Step S403: within the limit detection distance, determine the converted first image as a rearview image.

[0102] Step S404: beyond the limit detection distance, determine the converted second image as a rearview image.

[0103] In this embodiment, within the limit detection distance of the surround view camera, the converted first image is determined as the rear view image, and outside the limit detection distance of the surround view camera, the converted second image is determined as the rear view image. The wide viewing angle and short-range detection of the surround view camera and the conventional viewing angle and long-range detection of the rear view camera can be combined to realize the functions of the electronic rearview mirror camera and / or the side rearview camera, avoiding the hardware cost and debugging cost caused by the additional installation of the side rearview camera and / or the electronic rearview mirror camera.

[0104] In order to better implement the image processing method in the embodiment of the present application, based on the image processing method, an image processing device is also provided in the embodiment of the present application, such as Figure 5 As shown, the image processing device 700 includes:

[0105] An information determination module 710, configured to determine a limit detection distance of the surround view camera and a second resolution of the rear view camera based on a first resolution and a horizontal viewing angle of the surround view camera;

[0106] An image acquisition module 720 is configured to acquire a first image through a surround view camera and a second image through a rear view camera meeting a second resolution;

[0107] The image processing module 730 is used to determine a rear view image based on the limit detection distance, the first image and the second image.

[0108] In an embodiment of the present application, a first image is captured by a surround-view camera and a second image is captured by a rear-view camera that meets a second resolution, and a rear-view image is determined based on the maximum detection distance of the surround-view camera, the first image, and the second image. The wide viewing angle and close-range detection of the surround-view camera can be combined with the conventional viewing angle and long-range detection of the rear-view camera to realize the functions of an electronic rearview mirror camera and / or a side rearview camera, thereby avoiding the hardware cost and debugging cost caused by the additional installation of side rearview cameras and / or electronic rearview mirror cameras.

[0109] In some embodiments of the present application, the information determination module 710 determines the limit detection distance of the surround view camera and the second resolution of the rear view camera based on the first resolution and horizontal viewing angle of the surround view camera, including:

[0110] Determine a limit detection distance of the surround view camera based on a first resolution and a horizontal viewing angle of the surround view camera;

[0111] Based on the limit detection distance, determine the horizontal viewing angle range of the rearview camera;

[0112] A second resolution of the rear view camera is determined based on the horizontal viewing angle range and the detection distance range of the rear view camera.

[0113] In some embodiments of the present application, the information determination module 710 determines the horizontal viewing angle range of the rear view camera based on the limit detection distance, including:

[0114] Determining a first horizontal distance, a first vertical distance, and a second vertical distance based on a digital model file of the vehicle, a first theoretical installation position of the surround view camera, a second theoretical installation position of the rear view camera, and a third theoretical installation position of the side rear view camera;

[0115] Based on the first horizontal distance, determining a detection field length range of the rearview camera;

[0116] Determining the rearview detection distance range based on the first vertical distance, the second vertical distance and the limit detection distance;

[0117] Based on the detection field of view length range and the rearview detection distance range, the horizontal viewing angle range of the rearview camera is determined.

[0118] In some embodiments of the present application, the image processing module 730 determines the rear view image based on the limit detection distance, the first image and the second image, including:

[0119] Performing conversion processing on the first image to obtain a converted first image;

[0120] Performing conversion processing on the second image to obtain a converted second image;

[0121] Within the limit detection distance, determining the converted first image as a rear-view image;

[0122] Outside the limit detection distance, the converted second image is determined as a rearview image.

[0123] In some embodiments of the present application, the image processing module 730 performs conversion processing on the first image to obtain a converted first image, including:

[0124] Obtaining a distortion parameter of the surround view camera and a distance from a pixel point in the first image to an image center point of the first image;

[0125] Performing distortion correction on the pixel points in the first image based on the distortion parameter and the distance between the pixel points in the first image and the image center point of the first image to obtain a distortion-corrected first image;

[0126] The distortion-corrected first image is converted from the image coordinate system to the camera coordinate system to obtain a third image;

[0127] The third image is transformed from the camera coordinate system to the world coordinate system to obtain a transformed first image.

[0128] In some embodiments of the present application, the image processing module 730 converts the first image after distortion correction from the image coordinate system to the camera coordinate system to obtain a third image, including:

[0129] Obtaining the length of the pixel point in the first image after distortion correction on the imaging plane and the coordinates of the origin of the image coordinate system in the camera coordinate system;

[0130] The pixel points in the first image after distortion correction are converted based on the length of the pixel points in the first image after distortion correction on the imaging plane and the coordinates of the origin of the image coordinate system in the camera coordinate system to obtain a third image.

[0131] In some embodiments of the present application, the image processing module 730 converts the third image from the camera coordinate system to the world coordinate system to obtain the converted first image, including:

[0132] Get the depth value of the point in the world coordinate system, the intrinsic parameter matrix and extrinsic parameter matrix of the surround camera;

[0133] The pixel points in the third image are converted based on the depth value, the intrinsic parameter matrix and the extrinsic parameter matrix to obtain a converted first image.

[0134] The embodiment of the present application further provides a computer device, which integrates any one of the image processing devices provided in the embodiment of the present application, and the computer device includes:

[0135] one or more processors;

[0136] Memory; and

[0137] One or more applications, wherein the one or more applications are stored in the memory and are configured to execute, by the processor, the steps of the image processing method in any of the above-mentioned image processing method embodiments.

[0138] The present application also provides a computer device that integrates any image processing device provided in the present application. Figure 6 As shown, it shows a schematic diagram of the structure of the computer device involved in the embodiment of the present application, specifically:

[0139] The computer device may include one or more processing core processors 801, one or more computer-readable storage media memories 802, a power supply 803, an input unit 804 and other components. Those skilled in the art will appreciate that Figure 6 The computer device structure shown in the figure does not constitute a limitation on the computer device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently. Among them:

[0140] The processor 801 is the control center of the computer device. It uses various interfaces and lines to connect various parts of the entire computer device. By running or executing software programs and / or modules stored in the memory 802 and calling data stored in the memory 802, it executes various functions of the computer device and processes data, thereby monitoring the computer device as a whole. Optionally, the processor 801 may include one or more processing cores; preferably, the processor 801 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 801.

[0141] The memory 802 can be used to store software programs and modules. The processor 801 executes various functional applications and data processing by running the software programs and modules stored in the memory 802. The memory 802 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 802 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices. Accordingly, the memory 802 may also include a memory controller to provide the processor 801 with access to the memory 802.

[0142] The computer device also includes a power supply 803 for supplying power to each component. Preferably, the power supply 803 can be logically connected to the processor 801 through a power management system, so that the power management system can manage charging, discharging, power consumption and other functions. The power supply 803 can also include one or more DC or AC power supplies, recharging systems, power failure detection circuits, power converters or inverters, power status indicators and other arbitrary components.

[0143] The computer device may further include an input unit 804, which may be used to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal input related to user settings and function control.

[0144] Although not shown, the computer device may further include a display unit, etc., which will not be described in detail herein. Specifically in this embodiment, the processor 801 in the computer device will load the executable files corresponding to the processes of one or more application programs into the memory 802 according to the following instructions, and the processor 801 will run the application programs stored in the memory 802, thereby realizing various functions, as follows:

[0145] Determining a limit detection distance of the surround-view camera and a second resolution of the rear-view camera based on a first resolution and a horizontal viewing angle of the surround-view camera;

[0146] The first image is acquired by a surround-view camera and the second image is acquired by a rear-view camera meeting a second resolution;

[0147] A rear view image is determined based on the limit detection distance, the first image, and the second image.

[0148] A person of ordinary skill in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be completed by instructions, or by controlling related hardware through instructions. The instructions may be stored in a computer-readable storage medium and loaded and executed by a processor.

[0149] To this end, the embodiment of the present application provides a computer-readable storage medium, which may include: a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, etc. A computer program is stored thereon, and the computer program is loaded by a processor to execute the steps in any image processing method provided in the embodiment of the present application. For example, the computer program can be loaded by a processor to execute the following steps:

[0150] Determining a limit detection distance of the surround-view camera and a second resolution of the rear-view camera based on a first resolution and a horizontal viewing angle of the surround-view camera;

[0151] The first image is acquired by a surround-view camera and the second image is acquired by a rear-view camera meeting a second resolution;

[0152] A rear view image is determined based on the limit detection distance, the first image, and the second image.

[0153] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the detailed description of other embodiments above, and will not be repeated here.

[0154] In specific implementation, the above units or structures can be implemented as independent entities, or can be arbitrarily combined to be implemented as the same or several entities. The specific implementation of the above units or structures can refer to the previous method embodiments, which will not be repeated here.

[0155] The specific implementation of the above operations can be found in the previous embodiments, which will not be described in detail here.

[0156] The above is a detailed introduction to an image processing method, device, computer equipment and computer-readable storage medium provided in the embodiments of the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for technical personnel in this field, according to the ideas of the present application, there will be changes in the specific implementation methods and application scopes. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. An image processing method, characterized in that: include: Determining a limit detection distance of the surround-view camera and a second resolution of the rear-view camera based on a first resolution and a horizontal viewing angle of the surround-view camera; The surround view camera collects a first image and the rear view camera that meets the second resolution collects a second image; A rear view image is determined based on the limit detection distance, the first image, and the second image.

2. The method according to claim 1, characterized in that The determining, based on the first resolution and horizontal viewing angle of the surround view camera, the limit detection distance of the surround view camera and the second resolution of the rear view camera comprises: Determining a limit detection distance of the surround view camera based on a first resolution and a horizontal viewing angle of the surround view camera; Based on the limit detection distance, determining the horizontal viewing angle range of the rearview camera; A second resolution of the rear view camera is determined based on the horizontal viewing angle range and the detection distance range of the rear view camera.

3. The method according to claim 2, characterized in that The determining the horizontal viewing angle range of the rearview camera based on the limit detection distance includes: Determining a first horizontal distance, a first vertical distance, and a second vertical distance based on a digital model file of the vehicle, a first theoretical installation position of the surround view camera, a second theoretical installation position of the rear view camera, and a third theoretical installation position of the side rear view camera; Based on the first horizontal distance, determining a detection field length range of the rearview camera; Determining the rearview detection distance range based on the first vertical distance, the second vertical distance and the limit detection distance; Based on the detection field of view length range and the rearview detection distance range, a horizontal viewing angle range of the rearview camera is determined.

4. The method according to claim 1, characterized in that: The determining of the rear view image based on the limit detection distance, the first image and the second image includes: Performing conversion processing on the first image to obtain a converted first image; Performing conversion processing on the second image to obtain a converted second image; Within the limit detection distance, determining the converted first image as the rearview image; Outside the limit detection distance, the converted second image is determined as the rearview image.

5. The method according to claim 4, characterized in that The converting the first image to obtain a converted first image includes: Obtaining a distortion parameter of the surround view camera and a distance from a pixel point in the first image to an image center point of the first image; Performing distortion correction on the pixel points in the first image based on the distortion parameter and the distance between the pixel points in the first image and the image center point of the first image to obtain a distortion-corrected first image; Converting the first image after distortion correction from an image coordinate system to a camera coordinate system to obtain a third image; The third image is converted from the camera coordinate system to a world coordinate system to obtain a converted first image.

6. The method according to claim 5, characterized in that The step of converting the first image after distortion correction from an image coordinate system to a camera coordinate system to obtain a third image includes: Obtaining the length of a pixel point in the first image after distortion correction on an imaging plane and the coordinates of the origin of the image coordinate system in the camera coordinate system; The pixel points in the first image after distortion correction are converted based on the length of the pixel points in the first image after distortion correction on the imaging plane and the coordinates of the origin of the image coordinate system in the camera coordinate system to obtain a third image.

7. The method according to claim 5, characterized in that The step of converting the third image from the camera coordinate system to a world coordinate system to obtain a converted first image includes: Obtaining the depth value of the point in the world coordinate system, the intrinsic parameter matrix and the extrinsic parameter matrix of the surround view camera; The pixel points in the third image are converted based on the depth value, the intrinsic parameter matrix and the extrinsic parameter matrix to obtain a converted first image.

8. An image processing device, characterized in that: include: An information determination module, configured to determine a limit detection distance of the surround view camera and a second resolution of the rear view camera based on a first resolution and a horizontal viewing angle of the surround view camera; An image acquisition module, configured to acquire a first image through the surround view camera and a second image through the rear view camera meeting the second resolution; An image processing module is used to determine a rear view image based on the limit detection distance, the first image and the second image.

9. A computer device, characterized in that: The computer device comprises: one or more processors; Memory; and One or more application programs, wherein the one or more application programs are stored in the memory and are configured to be executed by the processor to implement the image processing method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and the computer program is loaded by a processor to execute the steps in the image processing method according to any one of claims 1 to 7.