A vehicle-mounted camera image processing system

Abstract

The method of the present invention provides a method for processing images of in-vehicle cameras, a processing system, and a storage medium. The method includes: first, acquiring image data of an in-vehicle camera; then, performing image processing on the image data to obtain panoramic image data; and transmitting the panoramic image data to a display screen for display to implement image processing of the in-vehicle camera. In the present invention, the image data of the in-vehicle camera is directly processed by a deserialiser, which reduces the workload of the SOC during the image processing of the in-vehicle camera, improves the working efficiency of the SOC, reduces power consumption, and ensures the real-time performance of the in-vehicle panoramic image. Compared with the current technology, it reduces the time for the SOC to configure the serializer and deserializer and wait for feedback, reduces the time for the SOC to process data, and overall shortens the image display time, enabling the display to display images faster.

Description

Technical Field

[0001] The present invention relates to the field of current testing, and more particularly, to an in-vehicle camera image processing system. Background Art

[0002] In-vehicle cameras are devices used to provide a scientific basis for traffic accident handling. Their ability to present video and audio in real time provides a more scientific basis for traffic accident handling and positioning, fully safeguarding our property and personal safety. With the rapid development of China's economy and the continuous progress of technology, automotive manufacturing technology has also witnessed unprecedented development. During vehicle operation, to enable drivers to obtain the surrounding environment of the vehicle in real time and improve driving safety, multiple cameras are required to capture images and synthesize these images to obtain a complete environmental image.

[0003] The prior art has the following technical problems:

[0004] Due to the rapid development of automotive electronics, intelligent driving has gradually evolved from 4-channel surround-view cameras to the current 12-channel cameras, which respectively collect surround view, side view, front view, and rear view. Currently, the technical solutions all transmit the signals output after being parsed by the deserializer to the SOC. After corresponding processing by the SOC, the signals are then transmitted to the serializer and finally transmitted to the screen terminal. As the number of cameras increases, the amount of data processed by the SOC becomes larger. The SOC is unable to maintain a fast image processing speed and is prone to jamming and crashing, requiring the SOC to have stronger performance. Summary of the Invention

[0005] In view of this, the purpose of the present invention is to provide an in-vehicle camera image processing system, which can reduce the task volume of the SOC during the in-vehicle camera image processing, improve the working efficiency of the SOC, reduce power consumption, and ensure the real-time nature of the in-vehicle panoramic image.

[0006] The present invention provides an in-vehicle camera image processing system, which at least includes:

[0007] An acquisition module, which is used to acquire the image data of the in-vehicle camera.

[0008] A processing module, which is used to perform image processing on the image data to obtain panoramic image data and transmit the panoramic image data to the display screen for display.

[0009] And a display module, which is used to display the panoramic image data.

[0010] Furthermore, the acquisition module is communicatively connected to multiple in-vehicle cameras.

[0011] The multi-channel vehicle-mounted camera is used to collect the panoramic view image, side view image, front view image and rear view image of the vehicle.

[0012] Furthermore, the processing module at least includes:

[0013] A judgment module, which is used to judge whether the current image data of each vehicle-mounted camera is in a preset format.

[0014] A deserializer, which is used to perform deserialization processing on the image data to convert it into a data structure or object;

[0015] An image signal processing module, which is used to convert the deserialized image data into image data in a preset format, and splice the image data in the preset format of each vehicle-mounted camera to obtain panoramic image data.

[0016] And a serializer, which is used to perform serialization processing on the panoramic image data to convert it into a format that can be saved or transmitted.

[0017] Furthermore, the image signal processing module is embedded in the deserializer.

[0018] The deserializer and the serializer are connected through an IIC integrated circuit bus.

[0019] The deserializer and the serializer are both provided with MIPI interfaces, and the MIPI interfaces are connected to each other.

[0020] It should be understood that the purpose of the MIPI interface is to standardize the interfaces inside the electronic device, such as the camera, display interface, radio frequency / baseband interface, etc., so as to reduce the complexity of mobile phone design and increase design flexibility.

[0021] In the above technical solution, the image processing method of the vehicle-mounted image processing system includes:

[0022] S100: Obtain the image data of the vehicle-mounted camera.

[0023] S200: Perform image processing on the image data to obtain panoramic image data.

[0024] S300: Transmit the panoramic image data to the display screen for display to implement vehicle-mounted camera image processing.

[0025] In the above technical solution, the vehicle-mounted camera image processing method shown in the embodiment of the present application is applied to simultaneously process multiple vehicle-mounted cameras. Among them, the core processing step is S200. Processing a part of the data directly in the deserializer or the serializer can reduce the task volume of the SOC, improve the working efficiency of the SOC, and reduce power consumption.

[0026] Further, step S100 specifically includes:

[0027] S101: Open the deserialization and serialization transmission channels.

[0028] S102: Collect the image data of each vehicle-mounted camera; the image data at least includes: surround view image data, side view image data, front view image data, and rear view image data.

[0029] Preferably, deserialization is a deserializer module.

[0030] Preferably, serialization is a serializer module.

[0031] Preferably, an image signal processing module is built into the deserializer module.

[0032] Further, opening the deserialization and serialization transmission channels specifically includes:

[0033] The deserializer has an image signal processing module built in. First, the image signal processing module configures the deserializer and the serializer to open the transmission channels.

[0034] Further, collect the image data of each vehicle-mounted camera; the image data at least includes: surround view image data, side view image data, front view image data, and rear view image data, specifically including:

[0035] After the image data of each vehicle-mounted camera is transmitted to the electronic control unit, the deserializer first parses the image information.

[0036] Further, step S200 specifically includes:

[0037] S201: Determine whether the current image data of each vehicle-mounted camera is in a preset format. If so, enter S203; otherwise, enter S202.

[0038] S202: Perform deserialization processing on the current image data and convert the deserialized current image data into image data in a preset format.

[0039] S203: Stitch the image data in the preset format of each vehicle-mounted camera to obtain panoramic image data.

[0040] S204: Perform serialization processing on the panoramic image data to convert it into a format that can be saved or transmitted.

[0041] It should be understood that serialization is the process of converting the state information of an object into a format that can be maintained or transmitted (a bunch of characters), such as the process of converting it into binary, xml, json, etc.;

[0042] Deserialization is the process of converting the binary strings, xml, json, etc. generated during the serialization process into data structures or objects.

[0043] Combining these two processes can easily store and transmit data.

[0044] Exemplarily, if the image data transmitted by the in-vehicle camera is RAW format data, the image signal processing module first converts the RAW format data into YUV422 format image data, then stitches the images of each camera, and then outputs the panoramic image to the serializer, and finally outputs it to the display screen for display.

[0045] If the image data transmitted by the in-vehicle camera is YUV422, the image signal processing module directly stitches the images of each camera, then outputs them to the serializer, and finally outputs them to the display screen for display.

[0046] Further, the preset format is any one of RAW format data, YUV422 format data, HDF5 format, and JPEG format.

[0047] Further, the panoramic image data is transmitted through IIC signals and MIPI signals.

[0048] In summary, compared with the current technology, it reduces the time for the SOC to the serializer and the serializer configuration and waiting for feedback, reduces the time for the SOC to process data, and overall shortens the image display time, enabling the display to display images faster. Since the SOC has to process a large amount of data, it causes overheating, performance degradation, and inability to maintain a fast image processing speed, and is prone to jamming and crashing. Therefore, the above technical solution can improve the image processing speed and enhance the real-time performance of the image processing of the image output system.

[0049] Compared with the prior art, the beneficial effects of this solution are as follows:

[0050] The present invention provides an in-vehicle camera image processing system. First, the image data of the in-vehicle camera is acquired; then, the image data is processed to obtain panoramic image data; the panoramic image data is transmitted to a display screen for display, thereby realizing in-vehicle camera image processing. Among them, the in-vehicle camera image data is directly processed by a deserialiser, which reduces the workload of the SOC during the in-vehicle camera image processing, improves the working efficiency of the SOC, reduces power consumption, and ensures the real-time performance of the in-vehicle panoramic image. Compared with the current technology, since the SOC has to process a large amount of data, it may overheat, resulting in a decline in performance, an inability to maintain a fast image processing speed, and prone to jamming and crashing phenomena, which affects the speed of image generation and reduces the real-time performance of image generation. The present invention reduces the time for the SOC to configure the serializer and deserializer and wait for feedback, reduces the time for the SOC to process data, and overall shortens the image display time, enabling the display to show images faster. Therefore, the above technical solution can improve the image processing speed and enhance the real-time performance of the image output system's image processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 It is a flowchart of a method for processing images of an in-vehicle camera according to the present invention.

[0052] Figure 2 It is a schematic diagram of an in-vehicle camera image processing system according to the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0053] To facilitate the understanding of the present invention, the present invention will be described more comprehensively below with reference to the relevant drawings. Preferred embodiments of the present invention are shown in the drawings. However, the present invention can be implemented in many different forms and is not limited to the embodiments described herein.

[0054] Please refer to Figure 1 , in a preferred embodiment, the image processing method of the in-vehicle camera image processing system proposed by the present invention includes:

[0055] S100: Acquire the image data of the in-vehicle camera.

[0056] S200: Process the image data to obtain panoramic image data.

[0057] S300: Transmit the panoramic image data to a display screen for display, thereby realizing in-vehicle camera image processing.

[0058] In the specific implementation process, the in-vehicle camera image processing method shown in the embodiments of the present application is applied to simultaneously process multiple in-vehicle cameras. Among them, the core processing step is S200. Processing a part of the data directly by the deserialiser or serializer can reduce the workload of the SOC, improve the working efficiency of the SOC, and reduce power consumption.

[0059] In this embodiment, step S100 specifically includes:

[0060] S101: Open the deserialization and serialization transmission channels.

[0061] S102: Collect the image data of each vehicle-mounted camera; the image data at least includes: surround view image data, side view image data, front view image data, and rear view image data.

[0062] Preferably, deserialization is performed by a deserializer module.

[0063] Preferably, serialization is performed by a serializer module.

[0064] Preferably, an image signal processing module is built into the deserializer module.

[0065] Further, opening the deserialization and serialization transmission channels specifically includes:

[0066] The deserializer has an built-in image signal processing module. First, the image signal processing module configures the deserializer and the serializer to open the transmission channels.

[0067] Further, collecting the image data of each vehicle-mounted camera; the image data at least includes: surround view image data, side view image data, front view image data, and rear view image data, specifically includes:

[0068] After the image data of each vehicle-mounted camera is transmitted to the electronic control unit, the deserializer first parses the image information.

[0069] In this embodiment, step S200 specifically includes:

[0070] S201: Determine whether the current image data of each vehicle-mounted camera is in a preset format. If so, enter S203; otherwise, enter S202.

[0071] S202: Perform deserialization processing on the current image data and convert the deserialized current image data into image data in a preset format.

[0072] S203: Stitch the image data in a preset format of each vehicle-mounted camera to obtain panoramic image data.

[0073] S204: Perform serialization processing on the panoramic image data to convert it into a format that can be saved or transmitted.

[0074] It should be understood that serialization is the process of converting the state information of an object into a format that can be maintained or transmitted (a bunch of characters), such as the process of converting it into binary, xml, json, etc.;

[0075] Deserialization is the process of converting the binary strings, xml, json, etc. generated during the serialization process into data structures or objects.

[0076] Combining these two processes can easily store and transmit data.

[0077] Exemplarily, if the image data transmitted by the in-vehicle camera is RAW format data, the image signal processing module first converts the RAW format data into YUV422 format image data, then stitches the images of each camera, and then outputs the panoramic image to the serializer, and finally outputs it to the display screen for display.

[0078] If the image data transmitted by the in-vehicle camera is YUV422, the image signal processing module directly stitches the images of each camera, then outputs them to the serializer, and finally outputs them to the display screen for display.

[0079] In this embodiment, the preset format is any one of RAW format data, YUV422 format data, HDF5 format, and JPEG format.

[0080] In this embodiment, the panoramic image data is transmitted through IIC signals and MIPI signals.

[0081] In summary, compared with the current technology, it reduces the time for the SOC to the serializer and the serializer configuration and waiting for feedback, reduces the time for the SOC to process data, and overall shortens the image display time, enabling the display to display images faster. Since the SOC has to process a large amount of data, it causes overheating, performance degradation, and is unable to maintain a fast image processing speed, and is prone to jamming and crashing. Therefore, the above technical solution can improve the image processing speed and enhance the real-time image processing of the image output system.

[0082] Please refer to Figure 2 , the present invention provides an in-vehicle camera image processing system, and the processing system at least includes:

[0083] An acquisition module, the acquisition is used to acquire the image data of the in-vehicle camera.

[0084] A processing module, configured to perform image processing on the image data, obtain panoramic image data, and transmit the panoramic image data to the display screen for display.

[0085] And a display module, configured to display the panoramic image data.

[0086] In this embodiment, the acquisition module is communicatively connected to multiple in-vehicle cameras.

[0087] The multi-channel vehicle-mounted camera is used to collect the panoramic view image, side view image, front view image and rear view image of the vehicle.

[0088] In this embodiment, the processing module at least includes:

[0089] A judgment module, which is used to judge whether the current image data of each vehicle-mounted camera is in a preset format.

[0090] A deserializer, which is used to perform deserialization processing on the image data to convert it into a data structure or object;

[0091] An image signal processing module, which is used to convert the deserialized image data into image data in a preset format, and splice the image data in the preset format of each vehicle-mounted camera to obtain panoramic image data.

[0092] And a serializer, which is used to perform serialization processing on the panoramic image data to convert it into a format that can be saved or transmitted.

[0093] In this embodiment, the image signal processing module is embedded in the deserializer.

[0094] The deserializer and the serializer are connected through an IIC integrated circuit bus.

[0095] Both the deserializer and the serializer are provided with MIPI interfaces, and the MIPI interfaces are connected to each other.

[0096] It should be understood that the purpose of the MIPI interface is to standardize the interfaces inside the electronic device, such as the camera, display interface, radio frequency / baseband interface, etc., so as to reduce the complexity of mobile phone design and increase design flexibility.

[0097] Compared with the prior art, the beneficial effect of this solution is that:

[0098] The present invention provides an on-vehicle camera image processing system, which first acquires image data of the on-vehicle camera; then performs image processing on the image data to obtain panoramic image data; and transmits the panoramic image data to a display screen for display, thereby realizing on-vehicle camera image processing; wherein the on-vehicle camera image data is directly processed in a deserializer, thereby reducing the task load of a SOC in the process of on-vehicle camera image processing, improving the working efficiency of the SOC, reducing power consumption, and ensuring the real-time performance of the on-vehicle panoramic image; compared with the current technology, since the SOC has to process a large amount of data, it will overheat, the performance will be reduced, and it will be unable to maintain a faster image processing speed, and it will be prone to jamming and freezing, which will affect the speed of image generation and reduce the real-time performance of image generation. The present invention reduces the time for the SOC to configure a serializer and a serializer and wait for feedback, reduces the time for the SOC to process data, and shortens the image display time as a whole, so that the display can display images faster; therefore, the above technical scheme can improve the image processing speed and improve the image processing real-time performance of the image output system.

[0099] Although example embodiments have been described herein with reference to the accompanying drawings, it should be understood that the above example embodiments are merely exemplary and are not intended to limit the scope of the present invention thereto. Various changes and modifications may be made therein by one of ordinary skill in the art without departing from the scope and spirit of the present invention. All such changes and modifications are intended to be included within the scope of the present invention as required by the appended claims.

[0100] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0101] The various system and method embodiments of the present invention can be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. It should be understood by those skilled in the art that a microprocessor or digital signal processor (DSP) can be used in practice to implement some or all of the functions of some modules according to embodiments of the present invention. The present invention can also be implemented as a system program (e.g., a computer program and a computer program product) for executing part or all of the methods described herein. Such a program implementing the present invention can be stored on a computer-readable medium, or can have the form of one or more signals. Such a signal can be downloaded from an Internet website, or provided on a carrier signal, or provided in any other form.

[0102] In several embodiments provided by the present application, it should be understood that the disclosed systems and methods can be implemented in other ways. For example, the system embodiments described above are merely illustrative. For example, the division of functions is only a logical function division. In actual implementation, there may be other division methods. For example, multiple tools or components can be combined or integrated into another system, or some features can be ignored or not executed.

[0103] In addition, the terms "first" and "second" are only used for descriptive purposes and cannot be construed as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, features defined with "first" and "second" may explicitly or implicitly include one or more of such features. In the description of the present invention, "a plurality of" means two or more unless otherwise specifically defined.

[0104] Although the description of the present invention is made in conjunction with the above specific embodiments, it is obvious that those skilled in the art can make many substitutions, modifications and variations based on the above content. Therefore, all such substitutions, improvements and variations are included within the spirit and scope of the appended claims.

Claims

1. A vehicle-mounted camera image processing system, characterized in that, The processing system at least includes: An acquisition module, configured to acquire image data of an in-vehicle camera; A processing module, configured to perform image processing on the image data to obtain panoramic image data, and transmit the panoramic image data to a display screen for display; And a display module, configured to display the panoramic image data; Wherein, the processing module at least includes: A judgment module, configured to judge whether the current image data of each in-vehicle camera is in a preset format; A deserializer, configured to perform deserialization processing on the image data to convert it into a data structure or object; An image signal processing module, configured to convert the deserialized image data into image data in a preset format, and splice the image data in the preset format of each in-vehicle camera to obtain panoramic image data; And a serializer, configured to perform serialization processing on the panoramic image data to convert it into a format that can be saved or transmitted; The image signal processing module is embedded in the deserializer; Wherein, the image processing method of the in-vehicle camera image processing system includes: S100: Acquire image data of an in-vehicle camera; S200: Perform image processing on the image data to obtain panoramic image data; S300: Transmit the panoramic image data to a display screen for display to implement in-vehicle camera image processing; Wherein, the step S100 specifically includes: S101: Open the transmission channels for deserialization and serialization; S102: Collect the image data of each in-vehicle camera; the image data at least includes: surround-view image data, side-view image data, front-view image data, and rear-view image data; The step S200 specifically includes: S201: Judge whether the current image data of each in-vehicle camera is in a preset format. If so, enter S203; otherwise, enter S202; S202: Perform deserialization processing on the current image data, and convert the deserialized current image data into image data in a preset format; S203: Splice the image data in the preset format of each in-vehicle camera to obtain panoramic image data; S204: Perform serialization processing on the panoramic image data to convert it into a format that can be saved or transmitted.

2. The processing system according to claim 1, wherein The acquisition module is communicatively connected to multiple in-vehicle cameras; The multiple in-vehicle cameras are used to collect surround-view images, side-view images, front-view images, and rear-view images of the vehicle.

3. The processing system according to claim 1, characterized in that, The deserializer and the serializer are connected through an IIC integrated circuit bus; both the deserializer and the serializer are provided with MIPI interfaces, and the MIPI interfaces are connected to each other.

4. The vehicle-mounted camera image processing system according to claim 1, characterized in that, The preset format is any one of RAW format data, YUV422 format data, HDF5 format, and JPEG format.

5. The in-vehicle camera image processing system according to claim 1, characterized in that The panoramic image data is transmitted through IIC signals and MIPI signals.

Images