Vehicle-mounted camera multiplexing system, vehicle and method

Abstract

The invention provides a vehicle-mounted camera multiplexing system, a vehicle, a method, equipment and a computer readable storage medium, and the system comprises a camera which is arranged on the side surface or the periphery of a vehicle body and is configured to obtain a first image around the vehicle body, the angular resolution of the edge field of view of the camera is greater than or equal to the angular resolution of the center field of view; and the image processing module is in communication connection with the camera, and the image processing module is configured to process the first image to obtain a side view area image and an all-round view area image. In this way, one vehicle-mounted camera on each side is used for replacing a plurality of cameras on the side face or the periphery of a vehicle body, so that the cost is greatly reduced, images with higher angular resolution of an edge view field are obtained, and the image definition is greatly improved.

Description

Technical Field

[0001] The present disclosure generally relates to the field of vehicle-mounted cameras, and particularly to a vehicle-mounted camera multiplexing system, a vehicle, a method, a device, and a computer-readable storage medium. Background Art

[0002] With the development of the vehicle field, vehicle-mounted cameras are increasingly applied in the vehicle field. Especially in an autonomous driving system, vehicle-mounted cameras become key components for obtaining external information. To achieve the purpose of safe driving, the requirements for vehicle-mounted cameras are also getting higher and higher. For example, higher requirements are also put forward for the angular resolution, dynamic range, frame rate, etc. of vehicle-mounted cameras. However, with the continuous increase in the number of vehicle-mounted cameras installed on vehicles and the continuous improvement of imaging clarity, the requirements for computing power are also increasing. To make vehicle-mounted cameras achieve better performance, the cost will be very high. On the contrary, to reduce the cost, the performance of vehicle-mounted cameras will deteriorate, and there is no way to balance the cost and performance of vehicle-mounted cameras. Summary of the Invention

[0003] In a first aspect of the present disclosure, a vehicle-mounted camera multiplexing system is provided. The system includes: a camera, which is disposed on the side or around the vehicle body and is configured to acquire a first image around the vehicle body, and the angular resolution of the edge field of view of the camera is greater than or equal to the angular resolution of the central field of view; and an image processing module, which is communicatively connected to the camera and is configured to process the first image to obtain a side view area image and a surround view area image.

[0004] In a second aspect of the present disclosure, a vehicle is provided. The vehicle includes a vehicle body; and the vehicle-mounted camera multiplexing system as described in the first aspect, and the vehicle-mounted camera multiplexing system is installed on the side or around the vehicle body.

[0005] In a third aspect of the present disclosure, a vehicle-mounted camera multiplexing method is provided. The method includes: acquiring a first image around the vehicle body by a camera disposed on the side or around the vehicle body, and the angular resolution of the edge field of view of the camera is greater than or equal to the angular resolution of the central field of view; and processing the first image to obtain a side view area image and a surround view area image.

[0006] In a fourth aspect of the present disclosure, an electronic device is provided. The device includes at least one processing unit; and at least one memory, and the at least one memory is coupled to the at least one processing unit and stores instructions for execution by the at least one processing unit. When the instructions are executed by the at least one processing unit, the device executes the method of the third aspect.

[0007] In a fifth aspect of the present disclosure, a computer-readable storage medium is provided. A computer program is stored on the computer-readable storage medium, and the computer program can be executed by a processor to implement the method of the third aspect.

[0008] It should be understood that the content described in this part of the present disclosure is not intended to limit the key features or important features of the embodiments of the present disclosure, nor is it used to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] In conjunction with the accompanying drawings and with reference to the following detailed description, the above and other features, advantages, and aspects of the embodiments of the present disclosure will become more apparent. In the drawings, the same or similar reference numerals represent the same or similar elements, where:

[0010] Figure 1 A schematic diagram of a conventional vehicle-mounted camera is shown;

[0011] Figure 2 A schematic diagram of an exemplary environment in which embodiments of the present disclosure can be implemented is shown;

[0012] Figure 3 A schematic diagram showing the relationship between the field of view angle and the angular resolution of an eagle-eye camera according to some embodiments of the present disclosure is shown;

[0013] Figure 4 A schematic diagram of a vehicle-mounted camera multiplexing system according to some embodiments of the present disclosure is shown;

[0014] Figure 5 A schematic diagram of a first image captured by a camera according to some embodiments of the present disclosure is shown;

[0015] Figure 6 A schematic diagram of a schematic diagram showing an image displayed on a display according to some embodiments of the present disclosure is shown;

[0016] Figure 7 A flowchart of a process of vehicle-mounted camera multiplexing according to some embodiments of the present disclosure is shown; and

[0017] Figure 8 A block diagram of a device capable of implementing multiple embodiments of the present disclosure is shown. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0018] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. On the contrary, these embodiments are provided to more thoroughly and completely understand the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the protection scope of the present disclosure.

[0019] It should be noted that the title of any section / subsection provided herein is not restrictive. Various embodiments are described throughout this document, and any type of embodiment can be included under any section / subsection. In addition, the embodiments described in any section / subsection can be combined with any other embodiments described in the same section / subsection and / or different section / subsections in any manner.

[0020] In the description of the embodiments of the present disclosure, the term "including" and its like terms should be understood as open inclusion, that is, "including but not limited to". The term "based on" should be understood as "at least partially based on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The term "some embodiments" should be understood as "at least some embodiments". There may also be other explicit and implicit definitions hereinafter. The terms "first", "second", etc. may refer to different or the same objects. There may also be other explicit and implicit definitions hereinafter.

[0021] The embodiments of the present disclosure may involve the user's data, data acquisition and / or use, etc. These aspects all comply with the corresponding laws, regulations and related provisions. In the embodiments of the present disclosure, all data collection, acquisition, processing, processing, forwarding, use, etc. are carried out on the premise that the user is aware and confirms. Accordingly, when implementing the embodiments of the present disclosure, the type, scope of use, usage scenario, etc. of the data or information that may be involved should be informed to the user and the user's authorization should be obtained in an appropriate manner according to the relevant laws and regulations. The specific informing and / or authorization methods may vary according to the actual situation and application scenario, and the scope of the present disclosure is not limited in this regard.

[0022] For the convenience of description, several concepts and terms are first defined herein.

[0023] Field of view, the field of view represents the maximum range that a camera can observe, usually expressed in terms of an angle. The larger the field of view, the larger the observation range. The field of view can also be referred to as the "field of view angle". The field of view angle refers to the angular range in the horizontal, vertical or diagonal direction that a camera can capture, usually expressed by the horizontal field of view angle, vertical field of view angle and diagonal field of view angle. The larger the field of view angle, the wider the captured picture.

[0024] Central field of view, the area located at the center position in the camera's field of view, and also the most important part in camera imaging. The range of the central field of view can be determined according to the maximum field of view. For example, if the maximum field of view is 180°, the range of the central field of view may be 0° to + / -30°, 0° to + / -50°, 0° to + / -90°, etc. If the maximum field of view is 200°, the range of the central field of view may be 0° to + / -20°, 0° to + / -40°, 0° to + / -60°, 0° to + / -110°, etc.

[0025] Edge field of view, the area located at the edge part in the camera's field of view, which generally occupies the peripheral part of the field of view relative to the central field of view. The range of the edge field of view is determined according to the maximum field of view and the range of the central field of view. For example, if the maximum field of view is 180°, the range of the central field of view may be 0° to + / -50°, then the range of the edge field of view is + / -(50 - 90)°. If the maximum field of view is 200°, the range of the central field of view may be 0° to + / -60°, then the range of the edge field of view is + / -(60 - 100)°.

[0026] Angular resolution, angular resolution (PPD, Pixels Per Degree), also known as spatial angular resolution or angular resolution, refers to the amount of information stored within a unit angle range in an image, usually represented by the number of pixel points. Specifically, it refers to the number of pixel points filled within an average of every 1° included angle in the field of view angle. The larger the angular resolution, the more pixel points are filled within the image of the unit angle area, and the clearer the image, but the larger the storage space occupied by the image.

[0027] Modulation Transfer Function (MTF), generally, the contrast of the output image of an optical system is always worse than that of the input image, and this change amount of contrast has a close relationship with the spatial frequency characteristics. The ratio of the contrast of the output image to the contrast of the input image is called the Modulation Transfer Function (MTF), that is, the definition of MTF is: MTF = contrast of the output image / contrast of the input image. Since the contrast of the output image is always less than that of the input image, the MTF value is between 0 - 1.

[0028] As briefly mentioned before, the existing in-vehicle cameras have a high cost, and there is no balance between the cost and performance of in-vehicle cameras. In traditional solutions, multiple cameras are set at different positions on the side or around the vehicle body. Since the field of view ranges of each camera are different, an image with a larger field of view can be obtained finally.

[0029] Figure 1 Fig. 100 shows a schematic diagram of a traditional in-vehicle camera. As Figure 1As shown, in general, in order to achieve safe driving, multiple cameras with different field of view ranges can be set around the vehicle body, such as Figure 1 the front side view camera 110, rear side view camera 120, electronic rearview mirror camera 130, and surround view camera 140 set on the side of the vehicle body. According to different vehicle models, the positions of these cameras are different. For example, the position of the front side view camera can be set on the B-pillar, a vertical support structure between the front and rear doors of the vehicle, or on both sides of the roof, or in front of the rearview mirror bracket. Figure 1 The installation position of the front side view camera 110 in Figure 1 is only an example. Similarly, the installation positions of the rear side view camera 120, electronic rearview mirror camera 130, and surround view camera 140 are only examples, and the installation positions can be adjusted according to different vehicle models.

[0030] The inventor found that there is a field of view overlap between the front side view camera 110, rear side view camera 120, and electronic rearview mirror camera 130 and the surround view camera 140 within a specific field of view angle range. For example, the field of view angles of the front side view camera 110 and rear side view camera 120 are about 90° - 100°, using a 3 - 5M resolution camera. The field of view angle of the electronic outside rearview mirror camera 130 is usually greater than 65°, and the field of view angle of the surround view camera 140 is usually greater than 180°, using a 1.5 - 3M resolution camera. Due to the wider field of view, the angular resolution (or pixel density) of the traditional surround view camera 140 is lower than that of the front side view camera 110, rear side view camera 120, and electronic rearview mirror camera 130. The surround view camera 140 can be a fish-eye camera. A fish-eye camera is a special type of wide-angle camera with a field of view angle usually exceeding 180 degrees, and it produces a significant barrel distortion effect, that is, it produces an obvious outward bending effect at the image edge. Specifically, as the field of view angle increases, the angular resolution gradually decreases, making the pixel density of the edge field of view lower than that of the central field of view. Therefore, the angular resolution of the edge field of view is lower than that of the central field of view. Such a design is to achieve a 360° display screen. Usually, the image of the edge field of view is subjected to distortion processing, projective transformation, and / or cropping before splicing. Therefore, a high requirement is not placed on the angular resolution of the edge field of view.

[0031] However, the traditional solution uses multiple cameras to obtain images with a large field of view, resulting in high costs. Moreover, due to the low angular resolution of the edge field of view of the surround view camera, which is usually lower than that of the central field of view, the angular resolution of the edge field of view image is low, and the modulation transfer function (MTF) of the edge field of view is also very low, resulting in poor image clarity of the edge field of view.

[0032] Embodiments of the present disclosure propose a vehicle-mounted camera multiplexing solution. According to various embodiments of the present disclosure, a vehicle-mounted camera multiplexing system includes a camera. The camera is disposed on the side or around the vehicle body, and is configured to acquire a first image around the vehicle body. The angular resolution of the edge field of view of the camera is greater than or equal to the angular resolution of the central field of view. And an image processing module, which is communicatively connected to the camera and is configured to process the first image to obtain a side view area image and a surround view area image. Further, the modulation transfer function (MTF) value of the edge field of view of the camera is relatively high, approaching 0.7 to 0.8 times the MTF value of the central field of view.

[0033] In this way, the vehicle-mounted camera multiplexing system uses one vehicle-mounted camera to replace multiple cameras on the side of the vehicle body, such as replacing the front side view camera, the rear side view camera, the surround view camera, the electronic rearview mirror camera, etc., thereby significantly reducing costs. More importantly, since the angular resolution of the edge field of view of the camera is greater than or equal to the angular resolution of the central field of view, and the relatively high modulation transfer function of the edge field of view, the angular resolution of the edge field of view of the camera provided by the embodiments of the present disclosure is higher than that of the edge field of view of the surround view camera in the traditional solution, so as to obtain an image with a higher angular resolution in the edge field of view, and significantly improve the image clarity.

[0034] Some exemplary embodiments of the present disclosure will be described below with continued reference to the drawings.

[0035] Figure 2 A schematic diagram of an exemplary environment 200 in which embodiments of the present disclosure can be implemented is shown.

[0036] As Figure 2 shown, in the environment 200, the vehicle includes a vehicle body 210 and a vehicle-mounted camera multiplexing system installed on the side or around the vehicle body. The vehicle-mounted camera multiplexing system includes cameras 220, 221 and an image processing module (not shown in the figure).

[0037] The camera 220 can be disposed on the side or around the vehicle body 210. As an example, the camera 220 can be disposed on the side of the vehicle body 210, such as at a position near the rearview mirror, or at a position near the door handle, etc. It can be understood that the camera 220 can also be disposed at other positions of the vehicle body, such as the rear and the front of the vehicle body 210. The scope of the present disclosure is not specifically limited in this regard. The camera 220 is configured to obtain a first image around the vehicle body 210. In this embodiment, one camera 220 is used to replace the conventional multiple cameras to obtain an image with a larger field of view. For example, the field of view of the camera 220 is greater than or equal to 180°, and the camera 220 obtains an image with a field of view greater than or equal to 180° as the first image. Since the camera 220 captures an image of the surrounding environment on one side of the vehicle body 210, similarly, a camera 221 can also be installed on the other side of the vehicle body 210 to capture an image of the surrounding environment on the other side of the vehicle body 210. The field of view of the camera 221 is greater than or equal to 180°.

[0038] The angular resolution of the edge field of view of the camera 220 and the camera 221 is greater than or equal to the angular resolution of the central field of view. As an example, Figure 2 the edge field of view may include a side front field of view 222, a first side rear field of view 223, and a second side rear field of view 225, and the second side rear field of view 225 is smaller than the first side rear field of view 223. The central field of view may be the central part of the panoramic field of view 224, that is, the field of view angle obtained by subtracting the side front field of view 222 and the first side rear field of view 223 from the field of view angle of the panoramic field of view 224 is determined as the field of view angle of the central field of view. Considering that the present disclosure uses the camera 220 or the camera 221 to replace the existing multiple cameras, the range of the edge field of view of the camera 220 or the camera 221 can be the same as the field of view angle range of the conventional side front view camera 110 and the side rear view camera 120, or the same as the field of view angle range of the electronic outside rearview mirror camera 130. For example, the field of view of the side front field of view 222 can be the same as the field of view of the conventional side front view camera 110, the field of view of the first side rear field of view 223 can be the same as the field of view of the conventional side rear view camera 120, and the field of view of the second side rear field of view 225 can be the same as the field of view of the conventional electronic outside rearview mirror camera 130. The range of the central field of view can be determined based on the range of the edge field of view. It can be understood that the above ranges of the edge field of view and the central field of view are only examples, and can also be other angular ranges. The scope of the present disclosure is not specifically limited in this regard.

[0039] In some embodiments, cameras 220 and 221 may be cameras composed of optical lenses. The maximum field of view (FOV) of the optical lenses generally satisfies: FOV ≥ 180°, which is beneficial to achieving wide-angle characteristics, so as to obtain more scene information and meet the requirements of large-range detection. The image height IH corresponding to the maximum field of view, the effective focal length f of the optical lens, and the radian value θ of the maximum half field of view of the optical lens satisfy: 1.10 < (IH / 2) / (f×θ). Meeting this range can achieve a large positive F-θ distortion. Ultra-wide-angle lenses usually adopt F-θ distortion, making the image height linearly related to the field of view. Positive distortion is also known as pincushion distortion. The actual image height increases faster than the ideal image height as the field of view increases, that is, the magnification increases with the increase of the field of view. This distortion makes the edge part of the imaged object magnified more than the central part, resulting in the shape of the image presenting a pincushion, so the proportion of the edge field of view in the picture is higher than that of the central field of view in the picture. Such a design is beneficial to achieving the ultra-wide-angle characteristics of the optical lens, and at the same time can effectively increase the proportion of the edge field of view of the optical lens in the entire image plane, improve the angular resolution of the edge field of view, so as to achieve that the angular resolution of the edge field of view is greater than or equal to the angular resolution of the central field of view. According to such an optical lens design, the images captured by cameras 220 and 221 usually present a pincushion shape, and the angular resolution of the edge field of view is greater than or equal to the angular resolution of the central field of view, and the angular resolution of the edge field of view is higher, effectively improving the image clarity.

[0040] In some embodiments, the modulation transfer function value of the edge field of view of the camera is close to the modulation transfer function value of the central field of view.

[0041] Cameras 220 and 221 may be cameras that can meet the characteristic that the angular resolution of the edge field of view is greater than or equal to the angular resolution of the central field of view, and meet the requirement that the modulation transfer function (MTF) value of the edge field of view of the camera is close to the modulation transfer function value of the central field of view. For example, the modulation transfer function (MTF) value of the edge field of view is close to 0.7 - 0.8 times the modulation transfer function (MTF) value of the central field of view. The specific implementation is achieved through optical design according to the given specifications.

[0042] In some embodiments, the field of view of the camera is greater than or equal to 180°. As the field of view angle of the camera increases, the angular resolution increases or remains unchanged to meet the characteristic that the angular resolution of the edge field of view is greater than or equal to that of the central field of view. As an example, such a camera can be an eagle-eye camera. An eagle-eye camera is a wide-angle camera different from a fish-eye camera. It has an extremely wide field of view. The field of view of the eagle-eye camera is usually greater than or equal to 180°, and it will produce a significant pincushion distortion effect, that is, there will be an obvious inward bending effect at the image edge. Specifically, as the field of view angle of the camera increases, the angular resolution increases or remains unchanged, so that the angular resolution of the edge field of view is greater than or equal to that of the central field of view. That is to say, the pixel density of the edge field of view is greater than or equal to that of the central field of view. Figure 3 FIG. shows a schematic diagram of the relationship between the field of view angle and the angular resolution of the eagle-eye camera provided by the embodiment of the present disclosure. As Figure 3 shown, as the field of view angle of the eagle-eye camera increases from 0° to 120°, the corresponding angular resolution gradually increases from the initial angular resolution and is not lower than the initial angular resolution. In one embodiment, the distribution area of the maximum angular resolution can be determined by the application scenario. The distribution area of the maximum angular resolution is usually within the range of + / -(60°-90°) of the field of view angle of the camera.

[0043] It can be understood that Figure 3 only some data are exemplified. Although the variation relationship of larger viewing angles and angular resolutions is not shown, it still follows the characteristic that as the field of view angle of the camera increases, the angular resolution increases or remains unchanged.

[0044] Preferably, the angular resolution of the maximum field of view of camera 220 or camera 221 is greater than or equal to 1.2 times that of the central field of view, effectively improving the image clarity of the edge field of view. It can be understood that the relationship between the angular resolution of the maximum field of view and the angular resolution of the central field of view can also be other multiple relationships, and the present disclosure does not make specific limitations in this regard. Figure 4 FIG. shows a schematic diagram of the vehicle-mounted camera multiplexing system according to the embodiment of the present disclosure. As Figure 4 shown, the vehicle-mounted camera multiplexing system further includes an image processing module 410. The image processing module 410 is communicatively connected to the camera 220 and the camera 221. The image processing module 410 is configured to perform first image processing on the camera 220 or the camera 221 to obtain a side view area image and a surround view area image.

[0045] Alternatively, the image processing module 410 may be integrated with the camera 220 in the same electronic device, or the image processing module 410 may be separately provided independently of the camera 220 for processing the images sent by the camera 220. Alternatively, another image processing module may be integrated with the camera 221 in the same electronic device, or the other image processing module may be separately provided independently of the camera 221, and the other image processing module is used for processing the images sent by the camera 221. Alternatively, an image processing module is communicatively connected to the camera 220 and the camera 221, and such an image processing module is used for processing the images sent by the camera 220 and the camera 221. It can be understood that the setting manner of the image processing module and the communication connection manner with the camera may also be other manners, and the scope of the present disclosure in this regard is not specifically limited.

[0046] Figure 5 Schematic diagram showing a first image captured by a camera according to an embodiment of the present disclosure. In combination with Figure 2 , Figure 4 , Figure 5 The process of the image processing module 410 obtaining the side view area image and the panoramic view area image by processing the first image sent by the camera 220 or the camera 221 is introduced.

[0047] First, the image processing module 410 may determine a side front view area corresponding to the side front view field, a first side rear view area corresponding to the first side rear view field, a second side rear view area corresponding to the second side rear view field, and a panoramic view area corresponding to the panoramic view field from the first image sent by the camera 220 or the camera 221.

[0048] Figure 2 The side front view field 222, the first side rear view field 223, the second side rear view field 225, and the panoramic view field 224 are shown, where the first side rear view field 223 is larger than the second side rear view field 225. In combination with Figure 5 as shown, Figure 2 the side front view field 222 in Figure 2 corresponds to the side front view area 520 in the first image 510, Figure 2 the first side rear view field 223 in Figure 5 corresponds to the first side rear view area 530,

[0049] Then, the image processing module 410 performs projection correction on the side front view area 520, the first side rear view area 530, the second side rear view area (not shown in the figure), and the surround view area 540 respectively, to obtain a side front view image, a first side rear view image, a second side rear view image, and a surround view image. In some embodiments, the side view area images may include the side front view image, the first side rear view image, and the second side rear view image. The field of view of the first side rear view image is larger than that of the second side rear view image. It can be understood that the side view area images may also include the side front view image and the first side rear view image, which are within the protection scope of the present disclosure.

[0050] The transformation method of the projection correction is to divide the first image 510 into grids to form a grid map. Figure 5 The side front view area 520, the first side rear view area 530, and the surround view area 540 after grid division are exemplified in []. The points in the grid map are calibrated in the camera coordinate system, converted from the camera coordinate system to the vehicle coordinate system, and then transformed from the vehicle coordinate system to the world coordinate system. Through such coordinate transformation, the correction matrix is determined, and the first image 510 is projected and corrected based on the correction matrix to obtain a distortion-free side front view image, a first side rear view image, a second side rear view image, and a surround view area image, which is convenient for direct viewing by the human eye. The specific process of the correction can refer to various existing correction methods and will not be described in detail here.

[0051] In some embodiments, the image processing module 410 may obtain the images captured by the cameras 220 and 221 respectively. The field of view corresponding to each image is the field of view on one side of the vehicle body, that is, the field of view includes 0° to 180° or greater than 180°. The image processing module 410 may perform projection correction on the surround view areas in the two images captured by the cameras 220 and 221 respectively to obtain the surround view area images on the two sides of the vehicle body. The image processing module 410 may splice the surround view area images on the two sides of the vehicle body, or the surround view area images on the two sides of the vehicle body and the surround view area images in the front and rear of the vehicle body into a distortion-free panoramic image, which is convenient for direct viewing by the human eye. The panoramic field of view corresponding to the panoramic image refers to a 360° field of view. The side front view field 222, the first side rear view field 223, and the second side rear view field 225 are all smaller than the panoramic field of view ( Figure 2 not shown in []).

[0052] In some embodiments, the image processing module 410 may acquire images of the field of view on both sides of the vehicle body captured by the cameras 220 and 221. The field of view range of each image is the field of view range on one side of the vehicle body, including 0° to 180° or greater than 180°. First, the image processing module 410 may splice the images of the fields of view on both sides of the vehicle body and the images of the front and rear fields of view of the vehicle body. The panoramic field of view corresponding to the spliced image refers to a 360° field of view. Then, a panoramic area corresponding to the panoramic field of view is determined from the spliced image, and the panoramic area is projection-corrected to obtain a distortion-free panoramic image for direct viewing by the human eye.

[0053] Continuing to refer Figure 4 , in some embodiments, the image processing module 410 may include a serializer 411. The input end of the serializer 411 is communicatively connected to the output ends of the cameras 220 and 221, and the output end of the serializer 411 is communicatively connected to the intelligent driving system 420, the electronic outside rearview mirror system 430, and the surround view system 440 respectively. The serializer 411 may send the side front view image and the first side rear view image to the intelligent driving system 420, and may also send the first image captured by the camera 220 or the camera 221 to the intelligent driving system 420; the serializer 411 may also send the first side rear view image and / or the second side rear view image to the electronic outside rearview mirror system 430; the serializer 411 may also send the surround view area image and / or the panoramic image, or the first image to the surround view system 440. In addition, the serializer may also be configured to send the first image to a relevant domain control system for the relevant domain control system to process the first image.

[0054] In some embodiments, the in-vehicle camera multiplexing system may further include an intelligent driving system 420, an electronic outside rearview mirror system 430, a surround view system 440, and a display 450. Specifically, the intelligent driving system 420 may receive and process the side front view image and the first side rear view image. For example, the intelligent driving system 420 may perform image recognition and other processing processes based on the side front view image and the first side rear view image, such as recognizing lane lines, obstacles, etc. on the side front view image and the first side rear view image. The intelligent driving system 420 may also send the side front view image and the first side rear view image, or the first image to the display 450 for display. In addition, the intelligent driving system 420 may also receive and process the first image captured by the camera 220 or the camera 221. The electronic outside rearview mirror system 430 may receive the first side rear view image and / or the second side rear view image, and send the first side rear view image and / or the second side rear view image to the display 450 for display. The surround view system 440 may receive the surround view area image and / or the panoramic image, and send the surround view area image and / or the panoramic image to the display 450 for display. In addition, the in-vehicle camera multiplexing system may further include a relevant domain control system, and the relevant domain control system may receive and process the first image.

[0055] In some embodiments, the display 450 is configured to display at least one of the following: a side front view image, a first side rear view image, a second side rear view image, a surround view area image, and a panoramic image. It can be understood that the display 450 can display any one or more of the above.

[0056] For example, Figure 6 illustrates a schematic diagram of the display 450 of the embodiments of the present disclosure displaying an image. As Figure 6 shown, the cameras 220 and 221 on both sides of the vehicle body 210 respectively capture images with a field of view range greater than or equal to 180° on both sides of the vehicle body 210, namely the first image 610 and the second image 620. The image processing module 410 can splice the first image 610 and the second image 620 to obtain an image with a panoramic field of view. The display 450 can display the side front view image 650 and the first side rear view image 660 corresponding to the first side of the vehicle body 210 sent by the intelligent driving system 420, and the side front view image 670 and the first side rear view image 680 corresponding to the second side of the vehicle body 210. The display 450 can display the second side rear view image 630 corresponding to the first side of the vehicle body 210 sent by the electronic outside rearview mirror system, the second side rear view image 640 corresponding to the second side of the vehicle body 210, and / or the first side rear view image 660 corresponding to the first side of the vehicle body 210 sent by the electronic outside rearview mirror system and the first side rear view image 680 corresponding to the second side of the vehicle body 210. The display 450 can display the 360° panoramic image 690 sent by the surround view system 440. Figure 6 These images in are only simplified schematic diagrams, and the actually captured images may include more objects.

[0057] The process of using the in-vehicle camera multiplexing system of the embodiments of the present disclosure to capture and display images on both sides of the vehicle body 210 is introduced above. Since the angular resolution of the edge field of view of the cameras 220 and 221 is greater than or equal to the angular resolution of the central field of view, the side front view image 650 and the first side rear view image 660 corresponding to the first side of the vehicle body 210, the side front view image 670 and the first side rear view image 680 corresponding to the second side of the vehicle body 210, and the second side rear view image 630 and the second side rear view image 640 can be determined as side view area images corresponding to the edge field of view. The resolution of these side view area images is higher than that of the edge resolution of traditional surround view cameras and is close to the resolution of images captured by traditional side view cameras, greatly improving the image clarity of the side view area images compared with traditional surround view cameras. It should be noted that the resolution of the side view area images, surround view area images, and panoramic images generated by the in-vehicle camera multiplexing system of this solution can be customized.

[0058] An embodiment of the present disclosure further provides a vehicle. The in-vehicle camera multiplexing system is installed on the side or around the vehicle body 210, and no cameras corresponding to the side front view area, side rear view area, surround view area, or panoramic area are additionally configured on the side or around the vehicle body 210. The in-vehicle camera multiplexing system uses one in-vehicle camera 220 (or camera 221) on each side to replace multiple cameras on the traditional vehicle body side, thereby greatly reducing costs. More importantly, since the angular resolution of the edge field of view of the camera is greater than or equal to the angular resolution of the central field of view, the angular resolution of the edge field of view of the camera provided by the embodiment of the present disclosure is higher than that of the edge field of view in the traditional surround view solution. Moreover, the modulation transfer function (MTF) value of the edge field of view of the camera is close to 0.7 to 0.8 times the modulation transfer function (MTF) value of the central field of view, thereby obtaining an image with a higher angular resolution in the edge field of view, greatly improving the image clarity, and obtaining an angular resolution and image clarity close to those of a traditional side view camera.

[0059] Example process

[0060] Figure 7 The flowchart shows the process of in-vehicle camera multiplexing according to some embodiments of the present disclosure. The following will be combined with Figure 4 and Figure 7 Describe process 700.

[0061] In block 710, a first image around the vehicle body 210 is obtained by the camera 220 disposed on the side or around the vehicle body 210, and the angular resolution of the edge field of view of the camera 220 is greater than or equal to the angular resolution of the central field of view.

[0062] In some embodiments, the field of view range of the camera is greater than or equal to 180°, and as the field of view angle of the camera 210 increases, the angular resolution increases or remains unchanged.

[0063] In some embodiments, the angular resolution of the maximum field of view is greater than or equal to 1.2 times the angular resolution of the central field of view.

[0064] In some embodiments, the side view area image includes a side front view image, a first side rear view image, and a second side rear view image, and the field of view range of the first side rear view image is greater than the field of view range of the second side rear view image.

[0065] In block 720, the image processing module 410 processes the first image to obtain a side view area image and a surround view area image.

[0066] In some embodiments, the image processing module 410 determines a side front view area corresponding to a side front view field, a first side rear view area corresponding to a first side rear view field, a second side rear view area corresponding to a second side rear view field, a surround view area corresponding to a surround view field, and / or a panoramic area corresponding to a panoramic view field from a first image, wherein the first side rear view field is larger than the second side rear view field, and the side front view field, the first side rear view field, and the second side rear view field are all smaller than the panoramic view field;

[0067] The image processing module 410 may perform projection correction on the side front view area, the first side rear view area, the second side rear view area, the surround view area, and / or the panoramic area respectively to obtain a side front view image, a first side rear view image, a second side rear view image, a surround view area image, and a panoramic image.

[0068] In some embodiments, the image processing module 410 sends the surround view area image and / or the panoramic image to the surround view system respectively, sends the side front view image and the first side rear view image to the intelligent driving system 420, and sends the first side rear view image and / or the second side rear view image to the electronic outside rearview mirror system 430; the intelligent driving system receives and processes the side front view image and the first side rear view image, the electronic outside rearview mirror system receives the first side rear view image and / or the second side rear view image, and the surround view system 440 receives the surround view area image and / or the panoramic image.

[0069] In some embodiments, the display 450 displays at least one of the following: the side front view image, the first side rear view image, the second side rear view image, the surround view area image, and the panoramic image.

[0070] Example device

[0071] Figure 8 The block diagram of an electronic device 800 in which one or more embodiments of the present disclosure can be implemented is shown. It should be understood that, Figure 8 The illustrated electronic device 800 is merely exemplary and should not constitute any limitation to the functions and scopes of the embodiments described herein. Figure 8 The illustrated electronic device 800 can be used to implement the electronic device 800 of the method for multiplexing vehicle-mounted cameras of the present disclosure.

[0072] As Figure 8As shown, the electronic device 800 is in the form of a general-purpose electronic device. The components of the electronic device 800 may include, but are not limited to, one or more processors or processing units 810, a memory 820, a storage device 830, one or more communication units 840, one or more input devices 850, and one or more output devices 860. The processing unit 810 may be an actual or virtual processor and is capable of performing various processes according to the programs stored in the memory 820. In a multi-processor system, multiple processing units execute computer-executable instructions in parallel to improve the parallel processing ability of the electronic device 800.

[0073] The electronic device 800 generally includes multiple computer storage media. Such media can be any accessible media that can be obtained by the electronic device 800, including but not limited to volatile and non-volatile media, removable and non-removable media. The memory 820 may be a volatile memory (such as registers, caches, random access memory (RAM)), a non-volatile memory (such as read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory), or some combination thereof. The storage device 830 may be a removable or non-removable medium and may include machine-readable media, such as flash drives, magnetic disks, or any other medium that can be used to store information and / or data (such as training data for training) and can be accessed within the electronic device 800.

[0074] The electronic device 800 may further include additional removable / non-removable, volatile / non-volatile storage media. Although not shown in Figure 8 , a disk drive for reading from or writing to a removable, non-volatile magnetic disk (such as a "floppy disk") and an optical disk drive for reading from or writing to a removable, non-volatile optical disk may be provided. In these cases, each drive may be connected to a bus (not shown) by one or more data media interfaces. The memory 820 may include a computer program product 825 having one or more program modules configured to execute the various methods or actions of the various embodiments of the present disclosure.

[0075] The communication unit 840 enables communication with other electronic devices through a communication medium. Additionally, the functions of the components of the electronic device 800 may be implemented by a single computing cluster or multiple computer machines that are capable of communicating through a communication connection. Thus, the electronic device 800 may operate in a networked environment using a logical connection with one or more other servers, network personal computers (PCs), or another network node.

[0076] The input device 850 can be one or more input devices, such as a mouse, a keyboard, a trackball, etc. The output device 860 can be one or more output devices, such as a display, a speaker, a printer, etc. The electronic device 800 can also communicate with one or more external devices (not shown) as needed through the communication unit 840. The external devices such as a storage device, a display device, etc., communicate with one or more devices that enable a user to interact with the electronic device 800, or communicate with any device that enables the electronic device 800 to communicate with one or more other electronic devices (e.g., a network card, a modem, etc.). Such communication can be performed via an input / output (I / O) interface (not shown).

[0077] According to an exemplary implementation of the present disclosure, there is provided a computer-readable storage medium having computer-executable instructions stored thereon, wherein the computer-executable instructions are executed by a processor to implement the method described above. According to an exemplary implementation of the present disclosure, there is also provided a computer program product, the computer program product being tangibly stored on a non-transitory computer-readable medium and including computer-executable instructions, and the computer-executable instructions being executed by a processor to implement the method described above.

[0078] Aspects of the present disclosure are described herein with reference to the flowcharts and / or block diagrams of methods, apparatuses, devices, and computer program products implemented according to the present disclosure. It should be understood that each block of the flowcharts and / or block diagrams, and the combinations of blocks in the flowcharts and / or block diagrams, can be implemented by computer-readable program instructions.

[0079] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine such that when these instructions are executed by the processing unit of the computer or other programmable data processing device, a device is produced that implements the functions / acts specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause a computer, a programmable data processing device, and / or other devices to work in a specific manner. Thus, the computer-readable medium storing the instructions includes a manufacture, which includes instructions for implementing various aspects of the functions / acts specified in one or more blocks of the flowchart and / or block diagram.

[0080] The computer-readable program instructions can be loaded onto a computer, other programmable data processing device, or other device, such that a series of operation steps are performed on the computer, other programmable data processing device, or other device to produce a computer-implemented process, so that the instructions executed on the computer, other programmable data processing device, or other device implement the functions / acts specified in one or more blocks of the flowchart and / or block diagram.

[0081] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various implementations of the present disclosure. In this regard, each block in the flowchart or block diagram may represent a module, a segment of a program, or a portion of an instruction, which contains one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions noted in the blocks may occur in a different order than noted in the accompanying drawings. For example, two consecutive blocks may actually be executed substantially in parallel, or they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented by a dedicated hardware-based system that performs the specified functions or actions, or by a combination of dedicated hardware and computer instructions.

[0082] The various implementations of the present disclosure have been described above. The above description is exemplary, not exhaustive, and is not limited to the disclosed implementations. Many modifications and variations will be apparent to those of ordinary skill in the art in the field of the present technology without departing from the scope and spirit of the described implementations. The choice of terms used herein is intended to best explain the principles of the implementations, the practical application, or the improvement of the technology in the market, or to enable other ordinary skilled persons in the field of the present technology to understand the various implementation manners disclosed herein.

Claims

1. A vehicle camera multiplexing system, comprising: A camera, the camera is arranged on the side or around the vehicle body, the camera is configured to obtain a first image around the vehicle body, and the angular resolution of the edge field of view of the camera is greater than or equal to the angular resolution of the central field of view; as well as An image processing module is communicatively connected to the camera, and is configured to process the first image to obtain a side view area image and a surround view area image.

2. The vehicle-mounted camera multiplexing system according to claim 1, wherein the field of view of the camera is greater than or equal to 180°, and as the field of view angle of the camera increases, the angular resolution increases or remains unchanged.

3. The vehicle-mounted camera multiplexing system according to claim 2, wherein the angular resolution of the maximum field of view of the camera is greater than or equal to 1.2 times the angular resolution of the central field of view.

4. The vehicle-mounted camera multiplexing system according to claim 1, wherein the side view area image includes a side front view image, a first side rear view image, and a second side rear view image, and the field of view of the first side rear view image is larger than the field of view of the second side rear view image.

5. The vehicle-mounted camera multiplexing system according to claim 4, wherein the vehicle-mounted camera multiplexing system further includes an intelligent driving system, and the intelligent driving system is configured to receive and process the side front view image, the first side rear view image, or the first image sent by the image processing module.

6. The vehicle-mounted camera multiplexing system according to claim 4, wherein the vehicle-mounted camera multiplexing system further comprises an electronic exterior rearview mirror system, and the electronic exterior rearview mirror system is configured to receive the first side rearview image and / or the second side rearview image sent by the image processing module.

7. The vehicle-mounted camera multiplexing system according to claim 4, wherein the vehicle-mounted camera multiplexing system further comprises a surround view system, and the surround view system is configured to receive the surround view area image and / or panoramic image sent by the image processing module.

8. The vehicle-mounted camera multiplexing system according to any one of claims 4 to 7, wherein the vehicle-mounted camera multiplexing system further comprises a display, and the display is configured to display at least one of the following: the side front view image, the first side rear view image, the second side rear view image, the surround view area image, and the panoramic image.

9. The vehicle camera multiplexing system according to any one of claims 4 to 7, wherein the image processing module comprises a serializer, an input end of the serializer is communicatively connected to an output end of the camera, and an output end of the serializer is communicatively connected to the intelligent driving system, the electronic exterior rearview mirror system, and the surround view system respectively; The serializer is configured to send the surround view area image and / or the panoramic image, or the first image to the surround view system, send the side front view image, the first side rear view image, or the first image to the intelligent driving system, and send the first side rear view image and / or the second side rear view image to the electronic exterior rearview mirror system; the serializer can also be configured to send the first image to a related domain control system, and the related domain control system processes the first image.

10. The vehicle camera multiplexing system according to claim 7, wherein the image processing module is further configured to: determine from the first image a side front view area corresponding to the side front view field, a first side rear view area corresponding to the first side rear view field, a second side rear view area corresponding to the second side rear view field, a surround view area corresponding to the surround view field, and / or a panoramic view area corresponding to the panoramic view field; wherein the first side rear view field is larger than the second side rear view field, and the side front view field, the first side rear view field, and the second side rear view field are all smaller than the panoramic view field; The image processing module is also configured to perform projection correction on the side front view area, the first side rear view area, the second side rear view area, the surrounding view area and / or the panoramic area, respectively, to obtain the side front view image, the first side rear view image, the second side rear view image, the surrounding view area image and / or the panoramic image. 11 . The vehicle-mounted camera multiplexing system according to claim 1 , wherein the modulation transfer function value of the edge field of view of the camera is close to the modulation transfer function value of the center field of view.

12. A vehicle comprising: Car body; as well as The vehicle-mounted camera multiplexing system as described in any one of claims 1-11, wherein the vehicle-mounted camera multiplexing system is installed on the side or around the vehicle body.

13. The vehicle according to claim 12, wherein no cameras corresponding to a side front view area, a side rear view area, a surround view area or a panoramic view area are additionally configured on the side or around the vehicle body.

14. A vehicle camera multiplexing method, comprising: A first image around the vehicle body is acquired by a camera arranged on the side or around the vehicle body, wherein an angular resolution of an edge field of view of the camera is greater than or equal to an angular resolution of a central field of view; as well as The first image is processed to obtain a side view area image and a surround view area image. 15 . The method according to claim 14 , wherein the field of view of the camera is greater than or equal to 180°, and as the field of view of the camera increases, the angular resolution increases or remains unchanged.

16. The method of claim 15, wherein the angular resolution of the maximum field of view is greater than or equal to 1.2 times the angular resolution of the central field of view.

17. The method according to claim 14, wherein the side viewing area image comprises a side front viewing image, a first side rear viewing image, and a second side rear viewing image, and a field of view of the first side rear viewing image is greater than a field of view of the second side rear viewing image.

18. The method according to claim 15, further comprising: Determine from the first image a side front viewing area corresponding to the side front viewing field, a first side rear viewing area corresponding to the first side rear viewing field, a second side rear viewing area corresponding to the second side rear viewing field, a surround viewing area corresponding to the surround viewing field, and / or a panoramic area corresponding to the panoramic viewing field, wherein the first side rear viewing field is larger than the second side rear viewing field, and the side front viewing field, the first side rear viewing field, and the second side rear viewing field are all smaller than the panoramic viewing field; Projection correction is performed on the side front view area, the first side rear view area, the second side rear view area, the surrounding view area and / or the panoramic area respectively to obtain the side front view image, the first side rear view image, the second side rear view image, the surrounding view area image and the panoramic image.

19. The method according to claim 18, further comprising: sending the surround view area image and / or the panoramic image to a surround view system, sending the side front view image, the first side rear view image, or the first image to an intelligent driving system, and sending the first side rear view image and / or the second side rear view image to an electronic exterior rearview mirror system; The intelligent driving system receives and processes the side front view image and the first side rear view image, the electronic exterior rearview mirror system receives the first side rear view image and / or the second side rear view image, and the surround view system receives the surround view area image and / or the panoramic image; At least one of the following is displayed on a display: the side front view image, the first side rear view image, the second side rear view image, the surround view area image and / or the panoramic image.

20. An electronic device, comprising: at least one processing unit; as well as At least one memory, the at least one memory being coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit, the instructions causing the electronic device to perform the method according to any one of claims 14 to 19 when executed by the at least one processing unit.

21. A computer-readable storage medium having a computer program stored thereon, wherein the computer program can be executed by a processor to implement the method according to any one of claims 14 to 19.

Images