Image processing method, vehicle, electronic equipment and storage medium

Abstract

The invention provides an image processing method, a vehicle, electronic equipment and a storage medium. The image processing method comprises the steps that a high-dynamic-range image collected by a camera is acquired, and the high-dynamic-range image is formed by synthesizing a plurality of low-dynamic-range images through the camera; parameters associated with a target low dynamic range image are obtained, and the target low dynamic range image is one of the multiple low dynamic range images; based on the parameter, the target low dynamic range image is restored from the high dynamic range image.

Description

Technical Field

[0001] This application relates to the field of image processing technology. More specifically, it relates to an image processing method, a vehicle, an electronic device, and a storage medium. Background Art

[0002] In the field of images, the dynamic range characterizes the ratio between the maximum brightness and the minimum brightness of an image. The larger the dynamic range of an image, the richer the brightness levels it can represent, and the more realistic the visual effect of the image. According to the dynamic range, images can be classified into high dynamic range (HDR) images, low dynamic range (LDR) images, etc.

[0003] In some lighting conditions, images captured with low dynamic range may include the loss of details in the bright or dark regions of the picture. High dynamic range (HDR) images can represent dark and bright regions more accurately. Therefore, in autonomous driving, HDR images are usually used for processing (e.g., object detection and tracking) to obtain information about the road or objects around the vehicle for route planning and navigation of the vehicle. However, since LDR images contain less information, in some cases, using LDR images for object detection and tracking can improve the perception efficiency. Summary of the Invention

[0004] Embodiments of this application provide an image processing method, a vehicle, an electronic device, and a storage medium. Without significantly increasing the data transmission bandwidth between the camera and the computing device, in addition to HDR images, the computing device can additionally obtain LDR images.

[0005] In a first aspect, an image processing method is provided, including: obtaining a high dynamic range image collected by a camera, where the high dynamic range image is formed by the camera synthesizing a plurality of low dynamic range images; obtaining parameters associated with a target low dynamic range image, where the target low dynamic range image is one of the plurality of low dynamic range images; and recovering the target low dynamic range image from the high dynamic range image based on the parameters.

[0006] In a second aspect, a vehicle is provided, including: a camera; and a computing device communicatively connected to the camera, where the computing device includes: one or more processors, and a memory storing a program, and the program includes instructions that, when executed by the processor, cause the processor to execute the method according to the first aspect.

[0007] In a third aspect, an electronic device is provided, including: one or more processors, and a memory storing a program, and the program includes instructions that, when executed by the processor, cause the processor to execute the method according to the first aspect.

[0008] In a fourth aspect, there is provided a computer-readable storage medium storing a program, the program including instructions which, when executed by one or more processors of a computing device, cause the computing device to execute the method according to the first aspect.

[0009] In the image processing method provided in the embodiments of the present application, the computing device uses the high-dynamic range image received from the camera and the parameters associated with the target low-dynamic range image to restore the target low-dynamic range image. In this way, without significantly increasing the data transmission bandwidth between the camera and the computing device and without the computing device incurring excessive computational load, the computing device can not only receive the high-dynamic range image from the camera but also obtain the low-dynamic range image. In some scenarios, for example, when the vehicle is driving at night under low-light conditions, while the vehicle's autonomous driving unit can normally acquire the HDR image, an additional low-dynamic range image can be obtained (the low-dynamic range image captured at night usually only has high-brightness objects such as vehicle headlights). The low-dynamic range image reduces redundant information for the detection and tracking of targets in the environment (such as the vehicle), improving the perception efficiency.

[0010] The high-dynamic range image is used to display a large number and complex types of targets, and the target low-dynamic range image is used to display a small number and single type of targets. Therefore, according to the solution of the present disclosure, even when there are a large number and complex types of targets in the high-dynamic range image, it is possible to accurately complete target detection and tracking from the small number and single type of targets on the target low-dynamic range image. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the background art, the following will describe the drawings required to be used in the embodiments of the present application or the background art. It should be noted that the same reference numerals in the drawings represent the same content.

[0012] Figure 1 is a schematic diagram of a vehicle traveling on a road provided according to an exemplary embodiment of the present application.

[0013] Figure 2 is a schematic diagram of a camera provided according to an exemplary embodiment of the present application.

[0014] Figure 3A is a schematic diagram of a computing device provided according to an exemplary embodiment of the present application.

[0015] Figure 3B is a schematic diagram of another computing device provided according to an exemplary embodiment of the present application.

[0016] Figure 4It is a flowchart of an image processing method provided according to an exemplary embodiment of the present application.

[0017] Figure 5A It shows a comparison between an HDR image and a restored target LDR image provided according to an exemplary embodiment of the present application.

[0018] Figure 5B It shows a comparison between another HDR image and the corresponding restored target LDR image provided according to an exemplary embodiment of the present application.

[0019] Figure 6 It is a flowchart of an image processing method provided according to another exemplary embodiment of the present application.

[0020] Figure 7 It is a schematic diagram of synthesizing an LDR image into an HDR image provided according to an exemplary embodiment of the present application.

[0021] Figure 8 It is a schematic structural diagram of an image processing apparatus provided according to an exemplary embodiment of the present application. Detailed implementation manners

[0022] The following makes an explanation of exemplary embodiments of the present disclosure with reference to the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered merely exemplary. Therefore, those of ordinary skill in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope of the present disclosure. Similarly, for the sake of clarity and conciseness, the description of well-known functions and structures is omitted below.

[0023] Figure 1 It is a schematic diagram of a vehicle 100 traveling on a road 130. The vehicle 100 can be a sedan, a truck, a motorcycle, a bus, a recreational vehicle, an amusement park vehicle, a tram, a golf cart, a train, a trolleybus, or other vehicles. The vehicle 100 can operate completely or partially in an autonomous driving mode. In the autonomous driving mode, the vehicle 100 can control itself. For example, the vehicle 100 can determine the current state of the vehicle and the current state of the environment in which the vehicle is located, determine the predicted behavior of at least one other vehicle in the environment, determine the confidence level corresponding to the possibility that the at least one other vehicle performs the predicted behavior, and control the vehicle 100 itself based on the determined information. When in the autonomous driving mode, the vehicle 100 can operate without human interaction.

[0024] Vehicle 100 may include various sensors for sensing information about the environment and conditions of the vehicle, such as camera 103. In addition to the camera, the sensors on vehicle 100 may further include an inertial measurement unit (IMU), a global navigation satellite system (GNSS) transceiver (e.g., a global positioning system (GPS) transceiver), a radio detection and ranging (RADAR) sensor, a light detection and ranging (LIDAR) sensor, an acoustic sensor, or an ultrasonic sensor, etc. Those skilled in the art can also understand that although Figure 1 a single camera 103 is shown in

[0025] Vehicle 100 may also include a computing system, which may include one or more computing devices, such as computing devices 110 and 120. The computing system can control some or all of the functions of vehicle 100. The computing system (e.g., computing devices 110 and / or 120 of the computing system) may include, for example, an autonomous driving control unit (also referred to as an autonomous driving unit), and the autonomous driving unit may be composed of one or more algorithm modules for identifying, evaluating, and avoiding or crossing potential obstacles in the environment where vehicle 100 is located. In some embodiments, the autonomous driving unit is used to combine data from sensors to determine the driving path or trajectory of vehicle 100.

[0026] Each of computing devices 110 and 120 may include at least one processor (which may include at least one microprocessor) and a memory (the memory is an example of a computer-readable storage medium), and the processor executes the processing instructions stored in the memory. In some embodiments, the memory may contain processing instructions (e.g., program logic) executed by the processor to implement various functions of vehicle 100. In addition to storing the processing instructions, the memory may store various information or data, such as images received from the camera. Some or all of the functions of the autonomous driving unit may be implemented using program code instructions resident in the memory of computing device 110 and / or 120 and executed by the processor. Vehicle 100 may also include a communication system not shown in the figure, and the communication system may provide a way for vehicle 100 to communicate with one or more devices or other surrounding vehicles. In an exemplary embodiment, the communication system may communicate directly or through a communication network with one or more devices. The communication system may be, for example, a wired or wireless communication system. For example, the communication system may use 3G cellular communication (e.g., CDMA, EVDO, GSM / GPRS) or 4G cellular communication (e.g., WiMAX or LTE), and may also use 5G cellular communication. Optionally, the communication system may communicate with a wireless local area network (WLAN) (e.g., using )。 Information / data can be propagated between the communication system of the vehicle 100 and a computing device (e.g., computing device 106) remotely located relative to the vehicle 100 via the network 114. The network 114 can be a single network or a combination of at least two different networks. The network 114 can include, but is not limited to, a combination of one or more of a local area network, a wide area network, a public network, a private network, etc.

[0028] During the driving process of the vehicle 100, sensors of the vehicle 100, such as the camera 103, continuously collect data related to the surrounding environment of the vehicle. The sensors of the vehicle 100 can also add timestamps to the collected data, and the timestamps record the time when the data is collected. For example, the camera 103 can add a timestamp to each frame of the image it collects to record the time when the frame of the image is collected. The sensors of the vehicle 100 can also send the collected data to a computing system (e.g., computing devices 110 and / or 120 of the computing system). For example, the camera 103 can output images to the computing system at a predetermined frame rate. In order to obtain more detailed information about the environment, the images output by the camera 103 usually have a high dynamic range (HDR). Each pixel value of the HDR image has a bit width of 24 bits (also referred to as bit depth), for example.

[0029] In one example, the computing device 110 receives sensor data from the sensors of the vehicle 100 (e.g., the camera 103), performs preliminary processing (e.g., format conversion) on the received sensor data, and then sends the processed sensor data to the computing device 120. The autonomous driving unit in the computing device 120 further processes the preliminarily processed sensor data received from the computing device 110 (e.g., object detection and tracking) to obtain information about the road or objects around the vehicle for route planning and navigation of the vehicle.

[0030] It should be noted that although Figure 1 in the example, the computing system of the vehicle 100 includes two computing devices, namely, computing devices 110 and 120, the computing system of the vehicle 100 includes more or fewer computing devices. For example, the computing system of the vehicle 100 can include only the computing device 120.

[0031] Figure 2 is a block diagram showing a camera 200 according to an exemplary embodiment of the present disclosure. The camera 200 can be Figure 1 an example of the camera 103 of the vehicle 100 shown. As Figure 2 shown, the camera 200 includes an optical unit 210, an image sensor 220, and an image signal processor 230. The optical unit 210 can include, for example, one or more lenses. The camera 200 can also include Figure 2 devices such as a memory (e.g., a register), a serializer, or a power supply not shown in

[0032] The optical unit 210 collects light from an object to be photographed (e.g., an object around the vehicle 100 shown), and guides the light to the image sensor 220. The image sensor 220 performs an imaging operation, i.e., performs photoelectric conversion to generate image data. The image sensor 220 provides the generated image data to the image signal processor 230. The image signal processor 230 performs predetermined processing on the image data received from the image sensor 220, and may transmit the processed image data to, for example, Figure 1 the computing system of the vehicle 100 shown. Figure 1 In one example, all or part of the functions of the image signal processor 230 may be implemented by the computing system of the vehicle 100 (e.g., the computing devices 110 and / or 120). In this case, the camera 103 may not include the image signal processor 230.

[0033] To obtain an HDR image, the camera 103 may generate multiple LDR images of the same content of the environment almost simultaneously (e.g., within a short time), and then synthesize the multiple LDR images to obtain an HDR image. The camera 103 may store parameters associated with synthesizing the LDR images in a memory (e.g., a register).

[0034]

[0035] Figure 3A Figure 1 is a schematic diagram of a computing device 300 provided according to an exemplary embodiment of the present application. The computing device 300 may be an example of the computing device 110 shown. The computing device 300 may include a data processor 302 (e.g., a system-on-chip (SoC), a general-purpose processing core, a graphics core, or optionally other processing logic) and a memory 304 that communicate with each other via a bus 306 or other data transfer systems. The computing device 300 may further include various input / output (I / O) devices or interfaces 303.

[0036] Memory 304 is an example of a computer-readable storage medium. The term computer-readable storage medium may be understood to include a single non-transitory medium or a plurality of non-transitory media (e.g., a centralized or distributed database and / or associated cache and computing systems) storing one or more instruction sets. The term computer-readable storage medium may also be understood to include any non-transitory medium capable of storing, encoding, or carrying an instruction set for a machine to execute and cause the machine to execute any one or more of the methods of the various embodiments, or capable of storing, encoding, or carrying a data structure utilized by or associated with such an instruction set. The term computer-readable storage medium may thus be understood to include, but is not limited to, solid-state memory, optical media, or magnetic media. The input / output (I / O) device or interface 303 of computing device 300 may include a deserializer to connect to a serializer of camera 103 (when camera 103 has a serializer).

[0037] Figure 3B is a schematic diagram of another computing device 310 provided according to an exemplary embodiment of the present application. Computing device 310 may be Figure 1 an example of the computing device 120 shown. Computing device 310 may be a server, a personal computer (PC), or an electronic control unit ECU (e.g., a domain controller), etc. Computing device 310 may include a data processor 312 (e.g., a system-on-chip (SoC), a general-purpose processing core, a graphics core, or optionally other processing logic) and a memory 314 that communicate with each other via a bus 316 or other data transfer system. Computing device 310 may also include various input / output (I / O) devices or interfaces 313 and a network interface 315. The network interface 315 may use 3G cellular communication (e.g., CDMA, EVDO, GSM / GPRS) or 4G cellular communication (e.g., WiMAX or LTE), and may also use 5G cellular communication. Optionally, the network interface 315 may communicate with a wireless local area network (WLAN) (e.g., using ). In an exemplary embodiment, the network interface 315 may actually include or support any wired and / or wireless communication and data processing mechanism by which information / data may propagate between computing device 310 and another device or system via a network.

[0038] Memory 314 is an example of a computer-readable storage medium. The term computer-readable storage medium can be understood to include a single non-transitory medium or multiple non-transitory media (e.g., a centralized or distributed database and / or associated cache and computing systems) that store one or more sets of instructions. The term computer-readable storage medium can also be understood to include any non-transitory medium that is capable of storing, encoding, or carrying a set of instructions for execution by a machine and that causes the machine to perform any one or more of the methods of various embodiments, or that is capable of storing, encoding, or carrying a data structure utilized by or associated with such a set of instructions. The term computer-readable storage medium can thus be understood to include, but is not limited to, solid-state memory, optical media, or magnetic media.

[0039] In some cases, for example, when the vehicle is driving at night or under other low-light conditions, the LDR image can reduce redundant information for the target detection and tracking of the autonomous driving unit, improving the perception efficiency. To obtain an LDR image for providing to the autonomous driving unit for target detection and tracking, one way is that the image sensor of the camera does not generate an HDR image, but instead sets different camera parameters according to the type of the target to be detected and tracked. For example, when the type of the target to be detected and tracked is a highlight, the exposure parameters (e.g., exposure time, ISO sensitivity, or sensor photosensitivity) are adjusted to an extremely low level so that only the highlight object is imaged in the image. However, in this case, all other non-highlight objects will be lost. The second way is that the image sensor of the camera does not synthesize multiple LDR images into an HDR image, but directly outputs the multiple LDR images to the computing device. The computing device will synthesize the multiple LDR images into an HDR image, and the autonomous driving unit directly uses the multiple LDR images for target detection and tracking. However, in this case, performing HDR synthesis consumes a great deal of resources of the computing device, and the time required for HDR synthesis is too long. The third way is that the camera directly outputs to the computing device the HDR image synthesized from multiple LDR images and one of the multiple LDR images. However, in this case, transmitting two images simultaneously between the camera and the computing device requires too much transmission bandwidth.

[0040] An embodiment of the present application provides an image processing method. A camera (e.g., camera 103) captures the surrounding environment of the camera 103 based on different exposure parameters (e.g., exposure time, ISO sensitivity, or sensor photosensitivity) to obtain a plurality of LDR images. After obtaining the plurality of LDR images, the camera 103 synthesizes the plurality of LDR images to obtain an HDR image and sends the HDR image to the computing device 110. The computing device (e.g., computing device 110) receives the HDR image and obtains the parameters associated with one of the plurality of LDR images (i.e., the target LDR image) from the memory of the camera 120. The computing device 110 restores the target LDR image from the HDR image based on the parameters associated with the target LDR image in the plurality of LDR images.

[0041] According to the image processing method provided by the embodiment of the present application, the computing device restores the target LDR image by using the HDR image received from the camera and the parameters associated with the target LDR image. In this way, without significantly increasing the data transmission bandwidth between the camera and the computing device and without the computing device having an excessive increase in computational load, the computing device can not only receive the HDR image from the camera but also obtain the LDR image. In some cases, for example, when the vehicle is driving at night under low-light conditions, while the vehicle's autonomous driving unit can normally obtain the HDR image, an additional LDR image can be obtained (the LDR image taken at night usually only has high-brightness objects such as vehicle headlights). The LDR image can reduce redundant information for the detection and tracking of targets (e.g., other vehicles) in the environment and improve the perception efficiency.

[0042] In addition, in the image processing method provided by the embodiment of the present application, the autonomous driving unit can obtain both the HDR image and the LDR image, which is equivalent to additionally performing target detection and tracking on the LDR image while maintaining the normal target detection and tracking of the HDR image. Using the LDR image for target detection and tracking can reduce redundant invalid information, and the computational load of the computing device does not increase excessively, and the perception efficiency is improved without increasing the transmission pressure between the computing device and the camera. For example, for the scenario of a vehicle driving at night, if you want to detect and track other vehicles in the environment, you can only detect and track the vehicle headlights of other vehicles in the environment, and the LDR image contains almost no other information except for the high-brightness objects (vehicle headlights). Compared with directly using the HDR image for vehicle detection and tracking, using the LDR image for vehicle detection and tracking can reduce redundant invalid information (e.g., non-vehicle information such as roads or signs), thereby improving the perception efficiency. In addition, since the HDR image includes non-vehicle information such as roads or signs, the autonomous driving unit can use the HDR image for target detection and tracking of non-vehicle information.

[0043] The following will combine Figure 4 to detail the process of image processing in the exemplary embodiments of this application. Figure 4 is a flowchart of the image processing method provided for the exemplary embodiments of this application. This image processing method can be executed by Figure 1 the computing system of the vehicle 100 shown (for example, the computing device 110 of the computing system).

[0044] As Figure 4 shown, in step 410, the computing device 110 acquires a first dynamic range image collected by a camera (for example, camera 103). For example, the computing device 110 can receive the first dynamic range image from the camera 103. Among them, the first dynamic range image is formed by the camera synthesizing multiple second dynamic range images. The first dynamic range is greater than the second dynamic range. The first dynamic range is, for example, HDR, and the second dynamic range is, for example, LDR. These LDR images can be images obtained by the camera respectively based on different exposure parameters (that is, different values of the exposure parameters) for photographing the surrounding environment of the camera.

[0045] The camera 103 can collect multiple LDR images almost simultaneously or within a very short time based on different exposure parameters, and synthesize the multiple low dynamic images into an HDR image. For example, the multiple LDR images can be synthesized into an HDR image by the image sensor (or image signal processor) of the camera.

[0046] The multiple LDR images come from different image channels of the camera, and these image channels are respectively associated with different exposure parameters. Therefore, the process of synthesizing multiple LDR images into HDR is also called channel synthesis. For example, when the camera 103 photographs the surrounding environment, different exposure parameters are set for different image channels to obtain different LDR images. That is to say, the multiple LDR images are images obtained by the camera respectively based on different exposure parameters for photographing the surrounding environment of the camera.

[0047] The exposure parameters of the camera may include one or more of the following: the sensitivity ISO (i.e., gain), the exposure time, or the sensor sensitivity. The camera may capture the surrounding environment of the camera based on, for example, different sensitivities ISO, different exposure times, and / or different sensor sensitivities, to obtain a plurality of LDR images. The sensor sensitivity is related to the physical structure of the image sensor of the camera (e.g., the structure of the pixels of the image sensor). For example, the pixels of the image sensor may be divided into at least two groups, each group corresponding to an image channel, and different groups respectively capture different LDR images. Different groups of pixels have different pixel structures, for example, having different on-chip microlenses, different photodiodes, different capacitors, and / or different sizes. For the same light, the lower the sensor sensitivity, the lower the pixel value of the pixel.

[0048] In step 420, the computing device 110 obtains the parameters associated with the target LDR image. For example, the computing device 110 may receive the parameters from the camera 103. As described above, the HDR image is synthesized by the camera from a plurality of LDR images, where the target LDR image is one of the plurality of LDR images.

[0049] Which LDR image is selected as the target LDR image from the plurality of LDR images may be determined according to the specific application scenario. For example, in a relatively dark surrounding environment, when it is desired to detect and track a high-brightness type of target, the target LDR image comes from the image channel with the lowest value of the exposure parameters of the camera (e.g., the smallest sensitivity ISO, and / or the shortest exposure time, and / or the smallest sensor sensitivity). Another example is that in a relatively bright surrounding environment, when it is desired to detect and track a low-brightness type of target, the target LDR image comes from the image channel with the highest value of the exposure parameters of the camera (e.g., the largest sensitivity ISO and / or the longest exposure time, and / or the largest sensor sensitivity).

[0050] The parameters associated with the target LDR image are used to recover the target LDR image from the HDR image. The parameters associated with the target LDR image may be the parameters set for the target LDR image when synthesizing the HDR image, for example, the parameters for indicating the bits associated with the target LDR image in the pixel data (or pixel values) of the HDR image, and / or the synthesis gain of the target LDR image, etc.

[0051] In one embodiment, to reduce the computational load of the computing device 110, the computing device does not obtain the parameters associated with the target LDR image at all times. Instead, it can obtain the parameters associated with the target LDR image only under specific ambient conditions. For example, the brightness of the camera's ambient environment can be determined. In response to the brightness of the camera's ambient environment being lower than a threshold (which can be set based on experience), the computing device 110 obtains the parameters associated with the target LDR image from the camera. Alternatively, the brightness of the camera's ambient environment can be determined. In response to the brightness of the camera's ambient environment being higher than a threshold (which can be set based on experience), the computing device 110 obtains the parameters associated with the target LDR image from the camera.

[0052] In step 430, the computing device 110 restores (i.e., splits) the target LDR image from the HDR image based on the obtained parameters associated with the target LDR image. The process of splitting the target LDR image from the HDR image is also referred to as channel splitting.

[0053] It should be noted that based on the obtained parameters associated with the target LDR image, the computing device 110 can fully restore the target LDR image from the HDR image, or it can only restore the target LDR image to a certain extent (i.e., partially). Although it is desirable to fully restore the target LDR image, that is, the pixel values of the corresponding pixels of the target LDR image restored from the HDR image are exactly the same as those of the target LDR image generated by the camera. However, to reduce the computational load, the pixel values of the corresponding pixels of the target LDR image restored from the HDR image and the target LDR image generated by the camera may not be exactly the same, and there can be a difference between them (i.e., the target LDR image is only restored to a certain extent or partially). However, the reason for restoring the target LDR image from the HDR image is to obtain information about objects with a specific brightness in the environment while reducing the interference of other information. A slight (or small) difference between the pixel values of the corresponding pixels of the target LDR image restored (or split) from the HDR image and the target LDR image generated by the camera will not affect the effect of the autonomous driving unit in detecting and tracking objects with a specific brightness.

[0054] Parameters associated with the target LDR image may include: a parameter (also referred to as the first parameter) indicating bits in the pixel data of the HDR image that are associated with the target LDR image. There will be bits in the pixel data of the HDR image that are associated with the target LDR image. By determining the bits associated with the target LDR image, the pixel data of the target LDR image can be obtained. Obtaining the pixel data of the target LDR image means obtaining the target LDR image. Therefore, step S430 specifically includes: determining, according to the first parameter, the bits in the pixel data of the HDR image that are associated with the target LDR image; and recovering the target LDR image from the HDR image based on the bits associated with the target LDR image.

[0055] When alpha - blending is used in the process of the camera synthesizing the HDR image, the first parameter may be the boundary information of the alpha - blending corresponding to the target LDR image. The computing device 110 can determine the bits associated with the target LDR image according to the boundary information of the alpha - blending.

[0056] For example, when the target LDR image comes from the image channel with the lowest exposure parameter value of the camera (e.g., the lowest ISO, and / or the shortest exposure time, and / or the lowest sensor photosensitivity), the computing device 110 can determine the bits above the upper boundary of the alpha - blending corresponding to the target LDR image as the bits associated with the target LDR image. Generally, the bits below the upper boundary contain low - light information, and the bits above the upper boundary contain high - light information.

[0057] It should be understood that the number of bits determined to be associated with the target LDR image may be different from the number of bits of the target LDR image. For example, the number of bits determined to be associated with the target LDR image may be less than the number of bits of the target LDR image. In this case, some bit information of the target LDR image will be lost in the recovered image. Therefore, after determining the bits associated with the target LDR image, it may only be possible to partially recover the target LDR image from the HDR image. The information lost in the target LDR image is usually the dark - part information of the target LDR image (especially when using the boundary information of alpha - blending to determine the bits associated with the target LDR image). The dark - part information in the target LDR image is usually information about low - light objects such as lane lines. In the recovered target LDR image, the autonomous driving unit is mainly concerned with high - light information such as vehicle lights. Therefore, the loss of low - light (low pixel value) information will not affect the target detection and tracking of the autonomous driving unit.

[0058] In one embodiment, recovering the target LDR image from the HDR image based on the bits associated with the target LDR image specifically includes: determining the values represented by the bits associated with the target LDR image as the pixel data of the target LDR image.

[0059] In one embodiment, the parameters associated with the target LDR image include, in addition to the above-mentioned first parameter, a second parameter. The second parameter is, for example, a synthesis gain. Recovering the target LDR image from the HDR image based on the bits associated with the target LDR image specifically includes: setting the bits other than the bits associated with the target LDR image in the pixel data of the HDR image to zero, and then dividing by the second parameter to obtain the pixel data of the target LDR image. For example, setting the bits other than the bits associated with the target LDR image in each pixel value of the HDR image to zero, and dividing the obtained pixel value by the second parameter (if the obtained value is a decimal, only the integer part can be retained) to obtain the pixel data of the target LDR image.

[0060] For ease of understanding the differences between the HDR image and the recovered target LDR image, please refer to Figure 5A and Figure 5B , Figure 5A In the upper figure of, the HDR image includes not only vehicle information but also non-vehicle information such as roads or signs. Figure 5A In the lower figure of, the target LDR image recovered from the HDR image in the upper figure includes the headlight information of the vehicle, but it hardly includes non-vehicle information such as roads or signs. Figure 5B In the upper figure of, the HDR image includes not only vehicle information but also non-vehicle information such as roads. Figure 5B In the lower figure of, the target LDR image recovered from the HDR image in the upper figure includes the headlight information of the vehicle, but it hardly includes non-vehicle information such as roads.

[0061] Figure 6 is a flowchart of an image processing method according to another exemplary embodiment of the present application. Figure 6 shows the changes experienced by the image from being captured by the camera until it is provided to the autonomous driving unit in the computing device. As Figure 6As shown, a camera (e.g., camera 103) can generate multiple (e.g., three) LDR images and synthesize at least two of them into one HDR image. These three LDR images are respectively called the SP1H image, the SP1L image, and the SP2 image. The SP1H image, the SP1L image, and the SP2 image are respectively from the SP1H image channel, the SP1L image channel, and the SP2 image channel of the camera. The exposure parameters (e.g., sensitivity ISO, exposure time, and / or sensor photosensitivity) of the SP1H image channel, the SP1L image channel, and the SP2 image channel are different. Specifically, the value of the exposure parameter of the SP1H image channel is greater than the value of the exposure parameter of the SP1L image channel, and the value of the exposure parameter of the SP1L image channel is greater than the value of the exposure parameter of the SP2 image channel. For example, for sensitivity ISO, exposure time, and sensor photosensitivity, the SP1H image channel is greater than the SP1L image channel in all cases, and the SP1L image channel is greater than the SP2 image channel in all cases. The bit widths of the SP1H image, the SP1L image, and the SP2 image are all 12 bits (bit) for example. Synthesizing at least two of the SP1H image, the SP1L image, and the SP2 image can obtain an HDR image with a bit width of 16 bits, 18 bits, or 24 bits for example.

[0062] It should be noted that the process of synthesizing the SP1H image, the SP1L image, and the SP2 image is actually the process of synthesizing their corresponding channels. The synthesis of the images can be completed in the image sensor (or image signal processor) of the camera 103. Therefore, the computing device 110 communicatively connected to the camera can obtain the HDR image from the camera 103. The computing device 110 can recover the target LDR image from the HDR image using the method described above with reference to Figure 4 the description.

[0063] For ease of understanding, taking the above-mentioned SP1L image and SP2 image as an example, the process of the camera synthesizing the HDR image using the SP1L image and the SP2 image will be described below. The processes of synthesizing the HDR image from the SP1L image and the SP1H image and the process of synthesizing the HDR image from the SP2 image and the SP1H image are the same as the process of synthesizing the HDR image from the SP1L image and the SP2 image, and will not be elaborated here.

[0064] The SP2 image and the SP1L image are taken under the same brightness environment but different exposure parameters. Therefore, their pixel values are different. Suppose the difference in sensitivity ISO between the two is 4 times (i.e., the sensitivity of the SP1L image channel is 4 times that of the SP2 image channel, and their other exposure parameters are the same). Then, the difference in sensitivity will bring a 4-fold difference in pixel values. If the pixel value of a pixel in the SP1L image is 200, then the pixel value of the corresponding pixel in the SP2 image is 50.

[0065] Similarly, if the exposure time of the SP1L image channel is 4 ms and the exposure time of the SP2 image channel is 1 ms, and other exposure parameters of the two are the same, then the difference in exposure time will also bring a 4-fold difference in pixel values. If the photosensitivity of the sensor in the SP1L image channel is 4 times that of the sensor in the SP2 image channel, and other exposure parameters of the two are the same, then the difference in sensor photosensitivity will also bring a 4-fold difference in pixel values.

[0066] In summary, if the ISO sensitivity, exposure time, and sensor photosensitivity of the SP1L image channel are all 4 times those of the SP2 image channel, then the difference in exposure parameters between the SP2 image and the SP1L image is 4×4×4 times, and the synthesis gain of the SP2 image is 64. That is to say, the synthesis gain of the image is determined by the exposure parameters of the camera (or the image channel of the camera). That is, when synthesizing the HDR image, all pixel values of the SP2 image will be multiplied by the synthesis gain (64 in this example). For example, the bit widths of both the SP1L image and the SP2 image are 12 bits, and the corresponding pixel value ranges are both 0 to 2 12 -1. When synthesizing the HDR image from the SP1L image and the SP2 image, the pixel values of the SP1L image remain unchanged, and its range is still 0 to 2 12 -1. The pixel values of the SP2 image will be multiplied by the synthesis gain (64 in this example), that is, the pixel value range of the SP2 image becomes 0 to 2 18 -1.

[0067] In one example, in the process of synthesizing the HDR image using the SP1L image and the SP2 image, the camera can use, for example, bits 1-10 of the pixel values of the SP1L image as bits 1-10 of the HDR image, and use bits 11-18 of the pixel values of the SP2 image after multiplying by the synthesis gain as bits 11-18 of the HDR image, thereby obtaining the HDR image. The camera can store parameters in the register to indicate the bits associated with the SP1L image and the bits associated with the SP2 image in the pixel values of the HDR image, as well as the synthesis gain of the SP2 image (that is, the synthesis gain multiplied by the pixel values of the SP2 image). After receiving the HDR image, the computing device 110 can use the method described above with reference to Figure 4The described method restores the target LDR image. For example, when the target LDR image is an SP2 image, the computing device 110 can read the registers of the camera and, based on the read parameters, determine the bits in the HDR image associated with the SP2 image and the synthesis gain of the SP2 image. For example, based on the read parameters, the computing device 110 determines that the bits in the HDR image associated with the SP2 image are bits 11 - 18, and the synthesis gain of the SP2 image is 64. The computing device 110 sets each of the bits 1 - 10 in the pixel value of the HDR image to 0, keeps the bits 11 - 18 in the pixel value of the HDR image unchanged, and divides the obtained pixel value by the synthesis gain 64 (if the obtained value is a decimal, only the integer part can be retained), thereby partially restoring the SP2 image. The bit width of the restored SP2 image is, for example, 12 bits.

[0068] In one example, during the process of synthesizing an HDR image using an SP1L image and an SP2 image, considering that there will be a partially overlapping pixel value range between the SP1L image and the SP2 image, the camera can determine at least a part of the overlapping pixel value range as the blending area for alpha - blending. For example, it is preset that the blending area for alpha - blending is bits 10 to 11 of the pixel value of the HDR image. At this time, the lower boundary of the blending area is bit 10 of the pixel value, and the upper boundary of the blending area is bit 11 of the pixel value. When synthesizing the HDR image, the pixel data with a pixel value of the HDR image less than 2 10 (i.e., bits 1 to 9 of the pixel value of the HDR image) uses the pixel value of the SP1L image, and the pixel data with a pixel value greater than 2 11 (i.e., bits 12 to 18 of the pixel value of the HDR image) uses the pixel value of the SP2 image multiplied by the synthesis gain. In the blending area (i.e., bits 10 to 11 of the pixel value of the HDR image), the alpha - blending method is used for blending, or the bits 10 to 11 of the pixel value of the SP1L image and the bits 10 to 11 of the pixel value of the SP2 image multiplied by the synthesis gain are added according to a predetermined weight respectively to obtain the bits 10 to 11 of the pixel value of the HDR image. The camera can store parameters in the register to indicate the bits in the pixel value of the HDR image associated with the SP1L image and the bits associated with the SP2 image (e.g., the boundaries of the alpha - blending area) and the synthesis gain of the SP2 image (i.e., the synthesis gain multiplied by the pixel value of the SP2 image). After receiving the HDR image, the computing device 110 can adopt the method described above with reference to Figure 4The described method restores the target LDR image. For example, when the target LDR image is the SP2 image, the computing device 110 can read the registers of the camera and determine the bits in the HDR image associated with the SP2 image and the synthesis gain of the SP2 image according to the read parameters. For example, the computing device 110 determines that the bits in the HDR image associated with the SP2 image are bits 12 - 18 according to the read parameters, and the synthesis gain of the SP2 image is 64. The computing device 110 sets each of the 1st - 11th bits of the pixel values of the HDR image to 0, keeps the 12th - 18th bits of the pixel values of the HDR image unchanged, and divides the obtained pixel values by the synthesis gain 64 (if the obtained value is a decimal, only the integer part can be retained), thereby partially restoring the SP2 image. The bit width of the restored SP2 image is, for example, 12 bits.

[0069] The SP2 image has low exposure parameters and includes more information about bright objects in the environment (for example, information about high - brightness objects such as car headlights), and less information about dark objects. For a night - time environment, when using the restored SP2 image to detect and track surrounding vehicles in the environment, vehicles in the environment can be better detected and tracked through the vehicle lights. The above text describes the process of synthesizing two of the SP1H image, SP1L image, and SP2 image into an HDR image and restoring the target LDR image according to the HDR image. The following will describe the process of synthesizing the SP1H image, SP1L image, and SP2 image into an HDR image and restoring the target LDR image according to the HDR image.

[0070] Figure 7 Figure 7 shows a schematic diagram of synthesizing the SP1H image, SP1L image, and SP2 image to obtain an HDR image according to an example. As Figure 7 shown, the bottom line segment is used to represent the relationship between the pixel values of the SP2 image and the ambient light, the middle line segment is used to represent the relationship between the pixel values of the SP1L image and the ambient light, and the upper line segment is used to represent the relationship between the pixel values of the synthesized HDR image and the ambient light. It should be noted that Figure 7 only shows the relationship between partial pixel values of the SP1H image and the ambient light, that is, Figure 7 the line segment OA in Figure 7. It should be noted that the line segment OA represents both the relationship between partial pixel values of the HDR image and the ambient light and the relationship between partial pixel values of the SP1H image and the ambient light.

[0071] As Figure 7As shown, the part of the pixel value of the HDR image between point O and point A is obtained according to the pixel value of the SP1H image. The part of the pixel value of the HDR image between point A and point B is obtained after the first alpha-blending of the pixel values of the SP1H image and the SP1L image. The part of the pixel value of the HDR image between point B and point C is obtained according to the pixel value of the SP1L image (that is, the value represented by one or more bits of the pixel value of the SP1L image is multiplied by the synthesis gain of the SP1L image). The part of the pixel value of the HDR image between point C and point D is obtained after the second alpha-blending of the pixel values of the SP2 image and the SP1L image. The part after point D is obtained according to the pixel value of the SP2 image (that is, the value represented by one or more bits of the pixel value of the SP2 image is multiplied by the synthesis gain of the SP2 image). The camera can store parameters in the register to indicate the bits associated with the SP1H image, the bits associated with the SP1L image, and the bits associated with the SP2 image in the pixel value of the HDR image (for example, the boundaries of the first and second alpha-blending regions), and the synthesis gain (that is, the synthesis gain of the SP1L and SP2 images). After receiving the HDR image, the computing device 110 can use the method described above with reference to Figure 4 to restore the target LDR image. For example, when the target LDR image is the SP2 image, the computing device 110 can read the register of the camera and determine the bits associated with the SP2 image in the HDR image and the synthesis gain of the SP2 image according to the read parameters. The computing device 110 sets the bits other than the bits associated with the SP2 image in each pixel value of the HDR image to zero, and divides the obtained pixel value by the synthesis gain of the SP2 image (if the obtained value is a decimal, only the integer part can be retained), so as to partially restore the SP2 image. The bit width of the restored SP2 image is, for example, 12 bits.

[0072] The above is only an example with the target LDR image being the SP2 image for illustration, and is not used to limit the present application. With different surrounding environments, the target LDR image will also be different. For example, for a night environment, the target LDR image is the SP2 image, which includes highlight information, that is, the target LDR image is the one with the smallest exposure parameter among multiple LDR images; for another example, for an environment with very strong sunlight, the target LDR image is the SP1H image, which includes shadow information, that is, the target LDR image is the one with the largest exposure parameter among multiple LDR images.

[0073] Continue to refer to Figure 6, the computing device 110 converts the format of the restored target LDR image and sends the image with the converted format to the computing device 120. The autonomous driving unit of the computing device 120 performs target detection and tracking based on the target LDR image with the converted format. The computing device 110 also converts the format of the HDR image received from the camera 103 and sends the image with the converted format to the computing device 120. The autonomous driving unit of the computing device 120 performs target detection based on the HDR image with the converted format.

[0074] Since both the target LDR image restored by the computing device 110 and the HDR image received by the computing device 110 from the camera 103 are images in the raw format (i.e., the format of the images collected or output by the camera), in order to reduce the transmission bandwidth and the computing resources required for the autonomous driving unit to process the images, the computing device 110 will first convert the format of these images to convert the raw format images into images in a visual format, such as JPEG or PNG, etc., so that the autonomous driving unit can perform target detection and tracking based on the images in the visual format.

[0075] The above format conversion may include: the computing device 110 converts the restored target LDR image or HDR image into an RGB image through demosaicing (Demosaic), that is, through demosaicing, the mosaic-like black and white image can be converted into a visual normal color image.

[0076] In order to reduce the transmission bandwidth, the above format conversion may further include: the computing device 110 encodes the RGB image into a JPEG image through encoding (such as Jpeg encoding), or encodes the RGB image into a PNG image through encoding. For example, for an 8-bit RGB image with a resolution of 1920×1080 and a size of 6.2M, if the 8-bit RGB image is not encoded but directly transmitted, the transmission bandwidth is too large. If the 8-bit RGB image is encoded, the size of the resulting JPEG image is only a few hundred k, the transmission bandwidth is very small, and the computing resources required for processing it are also small.

[0077] Of course, with the different requirements of the computing device for the image format, the format conversion process will also be different. For example, if the computing device only supports images with a bit width of 8 bits, and neither the restored target LDR image (e.g., with a bit width of 12 bits) nor the HDR image (e.g., with a bit width of 24 bits) is an 8-bit image, then the format conversion process may further include: through tone mapping (ToneMapping), mapping the restored target LDR image or HDR image to an 8-bit space to obtain an image with a bit width of 8 bits, and then the computing device 110 performs demosaicing and encoding on the tone-mapped image.

[0078] It should be noted that the order of the above-mentioned format conversion process can be changed according to actual needs, and the embodiments of the present application do not make specific limitations on this, as long as an image in a visual format can be obtained. For example, in one example, the order of the format conversion process performed by the computing device 110 on the restored LDR image and HDR image is: first tone mapping, then demosaic, and then encoding (such as Jpeg encoding).

[0079] In summary, in Figure 6 the example, the camera 103 (for example, the image sensor or image signal processor of the camera) synthesizes at least two of the SP1H image, the SP1L image, and the SP2 image to obtain an HDR image, and the camera 103 sends the HDR image to the computing device 110. The processing of the image by the computing device 110 is divided into two pipelines. One performs format conversion on the HDR image to obtain a format-converted HDR image (for example, a visual JPEG image), and the other restores (i.e., splits) one of the SP1H image, the SP1L image, and the SP2 image (for example, the SP2 image with a lower exposure parameter) from the HDR image, and then performs format conversion on the restored SP2 image to obtain a format-converted SP2 image (for example, a visual JPEG image). Finally, the computing device 110 sends the format-converted HDR image and SP2 image to the computing device 120, and the autonomous driving unit in the computing device 120 uses the format-converted SP2 image to detect and track a specific target (for example, a vehicle), and uses the format-converted HDR image to detect other objects other than the specific target (for example, lane lines, road signs, etc.).

[0080] It should be noted that Figure 6 the example shown shows that the computing device 110 and the computing device 120 are separate different devices. Those skilled in the art should understand that the vehicle may have only one of the computing device 110 and the computing device 120, and the other is omitted. For example, the vehicle only includes the computing device 120, and all the functions of the computing device 110 are implemented by the computing device 120.

[0081] Figure 8 is a schematic structural diagram of an image processing device 700 provided by an embodiment of the present application. As Figure 8 shown, the image processing device 700 may include: a first acquisition module 710, a second acquisition module 720, and a restoration module 730. These modules will be introduced in detail below.

[0082] The first acquisition module 710 is configured to acquire an HDR image collected by a camera, where the HDR image is formed by the camera synthesizing a plurality of LDR images.

[0083] The second acquisition module 720 is configured to acquire parameters associated with a target LDR image, where the target LDR image is one of a plurality of LDR images.

[0084] The restoration module 730 is configured to restore the target LDR image from the HDR image based on the parameters.

[0085] In some embodiments, the parameters include a first parameter that indicates bits in the pixel data of the HDR image associated with the target LDR image.

[0086] In some embodiments, the restoration module 730 is further configured to: determine, according to the first parameter, bits in the pixel data of the HDR image associated with the target LDR image.

[0087] In some embodiments, the parameters further include a second parameter, and the restoration module 730 is further configured to: set bits other than the bits in the pixel data of the HDR image associated with the target LDR image to zero, and then divide by the second parameter to obtain the pixel data of the target LDR image.

[0088] In some embodiments, the first parameter includes boundary information of alpha blending, and the second parameter includes a synthesis gain.

[0089] In some embodiments, the image processing apparatus 700 further includes: a first application module configured to: perform format conversion on the restored target LDR image; perform target detection and tracking based on the target LDR image after format conversion.

[0090] In some embodiments, the image processing apparatus 700 further includes: a second application module configured to: perform format conversion on the HDR image; perform target detection based on the HDR image after format conversion.

[0091] In some embodiments, the plurality of LDR images are images obtained by the camera capturing the surrounding environment of the camera based on different exposure parameters, and the target LDR image is the one with the smallest value of the exposure parameter among the plurality of LDR images.

[0092] In some embodiments, the first acquisition module 710 is further configured to: acquire the HDR image from the camera by a computing device communicatively connected to the camera.

[0093] In some embodiments, the second acquisition module 720 is further configured to: acquire the parameters associated with the target LDR image from the camera by the computing device.

[0094] In some embodiments, when the second acquisition module 720 acquires parameters associated with the target LDR image from a camera by a computing device, it is further configured to: determine the brightness of the surrounding environment of the camera; and in response to the brightness of the surrounding environment of the camera being lower than a threshold, acquire, by the computing device, parameters associated with the target LDR image from the camera.

[0095] In some embodiments, multiple LDR images are from different image channels of a camera, and the image channels are respectively associated with different values of exposure parameters.

[0096] An embodiment of the present application also provides a computer-readable storage medium, on which a program is stored, the program includes instructions, and when the instructions are executed by one or more processors of a computing device, the computing device is caused to execute the image processing method in any of the above embodiments.

[0097] An embodiment of the present application also provides a computer program product including instructions, and when the instructions are executed by a computer, the computer is caused to execute the image processing method in any of the above embodiments.

[0098] It can be understood that the specific examples herein are only for helping those skilled in the art better understand the embodiments of the present application, rather than limiting the scope of the present application.

[0099] It can be understood that in various embodiments of the present application, the magnitudes of the sequence numbers of the various processes do not mean the order of execution is prior or posterior, and the execution order of the various processes should be determined according to their functions and internal logics, and should not constitute any limitation to the implementation process of the embodiments of the present application.

[0100] It can be understood that the various embodiments described in the present application can be implemented alone or in combination, and the embodiments of the present application do not limit this.

[0101] Unless otherwise specified, all technical and scientific terms used in the embodiments of the present application have the same meanings as those commonly understood by those skilled in the technical field of this specification. The terms used in this specification are only for the purpose of describing specific embodiments, and are not intended to limit the scope of this specification. The term "and / or" used in this specification includes any and all combinations of one or more of the related listed items. The singular forms "a", "above", and "the" used in the embodiments of the present application and the appended claims are also intended to include the plural forms, unless the context clearly indicates otherwise.

[0102] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in connection with the embodiments disclosed herein can be implemented in electronic hardware, or in a combination of computer software and electronic hardware. Whether these functions are executed in hardware or software depends on the specific application and design constraints of the technical solution. Skilled artisans may use different methods for each specific application to implement the described functions, but such implementation should not be considered to exceed the scope of this specification.

[0103] As described above, the foregoing is only a specific embodiment of this specification, but the protection scope of this application is not limited thereto. Any person skilled in the art within the technical scope disclosed in this specification can easily conceive of changes or substitutions, which should all be covered within the protection scope of this specification. Therefore, the protection scope of this application should be subject to the protection scope of the claims.

Claims

1. An image processing method, comprising: Obtaining a high dynamic range image collected by a camera, wherein the high dynamic range image is formed by the camera synthesizing a plurality of low dynamic range images; Obtaining parameters associated with a target low dynamic range image, wherein the target low dynamic range image is one of the plurality of low dynamic range images; Recovering the target low dynamic range image from the high dynamic range image based on the parameters.

2. The image processing method according to claim 1, wherein, The parameters include a first parameter, and the first parameter indicates bits in the pixel data of the high dynamic range image associated with the target low dynamic range image.

3. The image processing method according to claim 2, wherein, The recovering the target low dynamic range image from the high dynamic range image based on the parameters includes: Determining, according to the first parameter, bits in the pixel data of the high dynamic range image associated with the target low dynamic range image.

4. The image processing method according to claim 3, wherein, The parameters further include a second parameter, and wherein the recovering the target low dynamic range image from the high dynamic range image based on the parameters further includes: Setting bits other than the bits in the pixel data of the high dynamic range image associated with the target low dynamic range image to zero, and then dividing by the second parameter to obtain the pixel data of the target low dynamic range image.

5. The image processing method according to claim 4, wherein, The first parameter includes boundary information of alpha blending, and the second parameter includes a synthesis gain.

6. The image processing method according to any one of claims 1 to 5, further comprising: Performing format conversion on the recovered target low dynamic range image; Performing target detection and tracking based on the target low dynamic range image after format conversion.

7. The image processing method according to any one of claims 1 to 5, further comprising: Performing format conversion on the high dynamic range image; Performing target detection based on the high dynamic range image after format conversion.

8. The image processing method according to any one of claims 1 to 5, wherein, The plurality of low dynamic range images are images obtained by the camera respectively shooting the surrounding environment of the camera based on different exposure parameters, and the target low dynamic range image is the one with the smallest value of the exposure parameter among the plurality of low dynamic range images.

9. The image processing method according to any one of claims 1 to 5, wherein, The obtaining the high dynamic range image collected by the camera includes: A computing device communicatively connected to the camera obtains the high dynamic range image from the camera, wherein the obtaining the parameters associated with the target low dynamic range image includes: The computing device obtains the parameters associated with the target low dynamic range image from the camera.

10. The image processing method according to claim 9, wherein, The obtaining, by the computing device, the parameters associated with the target low dynamic range image from the camera includes: Determining the brightness of the surrounding environment of the camera; In response to the brightness of the surrounding environment of the camera being lower than a threshold, the computing device obtains the parameters associated with the target low dynamic range image from the camera.

11. The image processing method according to any one of claims 1 to 5, wherein, The plurality of low dynamic range images come from different image channels of the camera, and the image channels are respectively associated with different values of exposure parameters.

12. A vehicle, comprising: A camera; And A computing device communicatively connected to the camera, wherein the computing device includes: One or more processors, and A memory storing a program, the program including instructions which, when executed by the processor, cause the processor to execute the image processing method according to any one of claims 1 to 11.

13. An electronic device, comprising: one or more processors, and a memory storing a program, the program including instructions which, when executed by the processor, cause the processor to execute the image processing method according to any one of claims 1 to 11.

14. A computer-readable storage medium storing a program, the program including instructions which, when executed by one or more processors of a computing device, cause the computing device to execute the image processing method according to any one of claims 1 to 11.

Images