Image processing method and apparatus

Abstract

The application provides an image processing method and device, which can be applied to the field of intelligent driving, automatic driving or unmanned driving of an intelligent automobile. The image processing method comprises the following steps: acquiring at least one initial image; obtaining a first preprocessed image after preprocessing according to the at least one initial image and a first mapping relationship, wherein the first mapping relationship exists between the pixel positions of the at least one initial image and the first preprocessed image, and the first mapping relationship corresponds to a plurality of preprocessing modes. Through preprocessing of the initial image by using the first mapping relationship, the first preprocessed image obtained after preprocessing has the first mapping relationship with the pixel positions of the initial image, the generation and storage of intermediate data are avoided, and the occupation of storage resources is reduced.

Description

Technical Field

[0001] This application relates to the field of image processing, specifically to intelligent driving, autonomous driving, or driverless driving, and more specifically, to an image processing method and apparatus. Background Technology

[0002] In image analysis, image quality directly affects the design and accuracy of recognition algorithms. Therefore, preprocessing is necessary before image analysis (feature extraction, segmentation, matching, and recognition). The main purpose of image preprocessing is to eliminate irrelevant information in the image, restore useful real information, enhance the detectability of relevant information, and simplify the data to the maximum extent, thereby improving the reliability of feature extraction, image segmentation, matching, and recognition.

[0003] When preprocessing the initial image captured by the camera, various preprocessing operations are performed sequentially, including translation, transpose, mirroring, rotation, affine transformation, inverse perspective transformation (IPM), lens distortion correction (LDC), and resolution adjustment. During the preprocessing of the initial image, a large number of intermediate calculation results are generated, consuming memory or other storage resources. Summary of the Invention

[0004] This application provides an image processing method and apparatus that can reduce the resource consumption of image preprocessing.

[0005] In a first aspect, an image processing method is provided, comprising: acquiring at least one initial image; and obtaining a preprocessed first preprocessed image based on the at least one initial image and a first mapping relationship, wherein a first mapping relationship exists between the pixel positions of the at least one initial image and the first preprocessed image, and the first mapping relationship corresponds to multiple preprocessing methods.

[0006] By using the first mapping relationship to preprocess the initial image, a first mapping relationship exists between the pixel positions of the preprocessed first image and the initial image, avoiding the generation and storage of intermediate data and reducing the occupation of storage resources.

[0007] In conjunction with the first aspect, in some possible implementations, obtaining the preprocessed first preprocessed image based on the at least one initial image and the first mapping relationship includes: preprocessing a first region image located in a first ROI within at least one region of interest (ROI) in the at least one initial image according to the first mapping relationship to obtain the first preprocessed image.

[0008] By preprocessing the image located in the first ROI, the amount of data that needs to be preprocessed is reduced, and the time required for preprocessing is also reduced.

[0009] In conjunction with the first aspect, in some possible implementations, the number of the at least one ROI is multiple, the first ROI corresponds to the first mapping relationship, and the method further includes: preprocessing a second region image located in the second ROI in the at least one initial image using a mapping relationship corresponding to the second ROI, wherein the second ROI is an ROI other than the first ROI among the at least one ROI.

[0010] If subsequent image processing imposes multiple requirements on image preprocessing, meaning it requires preprocessed results of the initial image obtained through different preprocessing methods, then the number of Regions of Interest (ROIs) can be multiple, with each ROI corresponding to a specific preprocessing requirement. Employing a mapping relationship between each ROI and the preprocessing method provides a more flexible image processing approach.

[0011] It should be understood that each ROI can correspond to a mapping relationship. Each ROI can be preprocessed using the mapping relationship between pixel positions in the initial image and the preprocessed image. When each ROI requires preprocessing using multiple methods, preprocessing the region image located in each ROI using a mapping method avoids storing intermediate data generated by each preprocessing method, thus reducing the consumption of storage resources.

[0012] Each ROI can consist of a single contiguous region or multiple contiguous regions that are separate from each other.

[0013] In conjunction with the first aspect, in some possible implementations, the at least one initial image comes from an onboard camera device, and the location of each ROI is obtained based on the vehicle's driving state.

[0014] Adjusting the position of the ROI based on the vehicle's driving status avoids situations where the preprocessing results of the initial image located in the ROI region do not meet the requirements of subsequent image processing due to changes in the vehicle's driving status.

[0015] In conjunction with the first aspect, in some possible implementations, the first mapping relationship includes multiple sub-mapping relationships, each of which corresponds to a sub-region in the first ROI. The step of preprocessing the first region image located in the first ROI of the at least one initial image according to the first mapping relationship includes: preprocessing the image located in each of the sub-regions in the at least one initial image in parallel according to the multiple sub-mapping relationships.

[0016] The first mapping relationship includes multiple sub-mapping relationships. The initial images in each region corresponding to the sub-mapping relationship are preprocessed in parallel using these sub-mapping relationships. Thus, while reducing image preprocessing time through parallel processing, the division of the initial image is avoided, making parallel preprocessing of the initial image simpler.

[0017] In conjunction with the first aspect, in some possible implementations, the first mapping relationship includes multiple sub-mapping relationships, each of which corresponds to a sub-region. Obtaining the preprocessed first preprocessed image based on the at least one initial image and the first mapping relationship includes: performing parallel preprocessing on the images located in each of the sub-regions in the at least one initial image according to the multiple sub-mapping relationships.

[0018] The first mapping relationship includes multiple sub-mapping relationships. The initial images in each region corresponding to the sub-mapping relationship are preprocessed in parallel using these sub-mapping relationships. Thus, while reducing image preprocessing time through parallel processing, the division of the initial image is avoided, making parallel preprocessing of the initial image simpler.

[0019] In conjunction with the first aspect, among some possible implementations, the multiple preprocessing methods include at least one of affine transformation, inverse perspective transformation (IPM), lens distortion correction (LDC), and geometric transformation. It should be understood that the multiple preprocessing methods may also include other methods.

[0020] In conjunction with the first aspect, in some possible implementations, the at least one initial image comes from an in-vehicle camera device, and the method further includes: acquiring demand information, the demand information being used to indicate the multiple preprocessing methods; and generating the first mapping relationship based on the demand information.

[0021] Based on the demand information, a first mapping relationship is generated. On the one hand, this makes the determination of the first mapping relationship more flexible; on the other hand, it eliminates the need to store a large number of mapping relationships, allowing the generation of only the required first mapping relationship when needed, thus reducing the consumption of storage resources.

[0022] In a second aspect, an image processing apparatus is provided, comprising an acquisition module and a processing module. The acquisition module is configured to acquire at least one initial image; the processing module is configured to obtain a preprocessed first preprocessed image based on the at least one initial image and a first mapping relationship, wherein a first mapping relationship exists between the pixel positions of the at least one initial image and the first preprocessed image, and the first mapping relationship corresponds to multiple preprocessing methods.

[0023] In conjunction with the second aspect, in some possible implementations, the processing module is specifically used to preprocess the first region image located in the first ROI of at least one region of interest (ROI) in the at least one initial image according to the first mapping relationship, so as to obtain the first preprocessed image.

[0024] In conjunction with the second aspect, in some possible implementations, the number of the at least one ROI is multiple, the first ROI corresponds to the first mapping relationship, and the processing module is further configured to preprocess a second region image located in the second ROI in the at least one initial image using a mapping relationship corresponding to the second ROI, wherein the second ROI is an ROI other than the first ROI among the at least one ROI.

[0025] In conjunction with the second aspect, in some possible implementations, the at least one initial image comes from an onboard camera device, and the location of the ROI is obtained based on the vehicle's driving status.

[0026] In conjunction with the second aspect, in some possible implementations, the first mapping relationship includes multiple sub-mapping relationships, each of which corresponds to a sub-region in the first ROI. The processing module is specifically used to preprocess the images located in each sub-region in the at least one initial image in parallel according to the multiple sub-mapping relationships.

[0027] In conjunction with the second aspect, in some possible implementations, the first mapping relationship includes multiple sub-mapping relationships, each of which corresponds to a sub-region. Specifically, the processing module is used to preprocess the images located in each sub-region in the at least one initial image in parallel according to the multiple sub-mapping relationships.

[0028] In conjunction with the second aspect, among some possible implementations, the various preprocessing methods include at least one of affine transformation, inverse perspective transformation (IPM), lens distortion correction (LDC), and geometric transformation.

[0029] In conjunction with the second aspect, in some possible implementations, the acquisition module is further configured to acquire requirement information, which is used to indicate the various preprocessing methods; the processing module is further configured to generate the first mapping relationship based on the requirement information.

[0030] Thirdly, an image processing apparatus is provided, comprising at least one memory and at least one processor, wherein the at least one memory is used to store a program, and the at least one processor is used to run the program to implement the method of the first aspect.

[0031] It should be understood that a program can also be called program code, computer instructions, computer program, program instructions, etc.

[0032] Fourthly, a chip is provided, comprising at least one processor and an interface circuit, wherein the interface circuit is used to provide program instructions or data to the at least one processor, and the at least one processor is used to execute the program instructions to implement the method described in the first aspect.

[0033] Optionally, as one implementation, the chip system may further include a memory storing a program, and the processor is used to execute the program stored in the memory. When the program is executed, the processor is used to perform the method in the first aspect.

[0034] Fifthly, a computer-readable storage medium is provided, the computer-readable medium storing program code for execution by a device, which, when executed by the device, implements the method described in the first aspect.

[0035] In a sixth aspect, a computer program product is provided, the computer program product comprising a computer program, wherein when the computer program product is executed by a computer, the computer performs the method described in the first aspect above.

[0036] It should be understood that, in this application, the method of the first aspect can specifically refer to the first aspect and the method in any of the various implementations of the first aspect.

[0037] In a seventh aspect, a terminal is provided, comprising the image processing apparatus described in the second or third aspect.

[0038] Furthermore, the terminal can be intelligent transportation equipment (vehicles or drones), smart home devices, intelligent manufacturing equipment, or robots. The intelligent transportation equipment can be, for example, an automated guided vehicle (AGV) or an unmanned transport vehicle. Attached Figure Description

[0039] Figure 1 This is a schematic structural diagram of an image processing system.

[0040] Figure 2 This is a schematic flowchart of an image processing method.

[0041] Figure 3 This is a schematic flowchart of an image processing method provided in an embodiment of this application.

[0042] Figure 4 This is a schematic structural diagram of an image processing system provided in an embodiment of this application.

[0043] Figure 5 This is a schematic diagram of an image processing method provided in an embodiment of this application.

[0044] Figure 6 This is a schematic diagram of an image storage format.

[0045] Figure 7 This is a mapping table provided in the embodiments of this application.

[0046] Figure 8 This is a schematic structural diagram of an image processing apparatus provided in an embodiment of this application.

[0047] Figure 9 This is a schematic structural diagram of another image processing apparatus provided in the embodiments of this application. Detailed Implementation

[0048] The technical solutions in this application will now be described with reference to the accompanying drawings.

[0049] Intelligent driving technology relies on the collaborative efforts of computer vision, radar, monitoring devices, and global positioning systems to enable motor vehicles to achieve autonomous or assisted driving without active human intervention or with only partial human intervention. Intelligent driving vehicles use various computing systems to help transport passengers or goods from one location to another. Some intelligent driving vehicles may require initial or continuous input from the operator (such as a navigator, driver, or passenger). Intelligent driving vehicles allow the operator to switch from manual mode to autonomous driving mode or a mode in between. Because autonomous driving technology eliminates the need for human drivers, it theoretically effectively avoids human driving errors, reduces traffic accidents, and improves highway transportation efficiency. Therefore, autonomous driving and other intelligent driving technologies are receiving increasing attention.

[0050] The mode between manual operation and autonomous driving can be called assisted driving mode. In assisted driving mode, the driving of a motor vehicle does not rely entirely on human active operation. The realization of autonomous driving and assisted driving depends on the development of intelligent driving technology.

[0051] Intelligent driving technology includes stages such as perception, decision-making, and control. In the perception stage, it receives information about the surrounding environment and understands its surroundings based on this information. In the decision-making stage, it uses the information output from the perception stage to predict the behavior of other road users, thereby making behavioral decisions for the vehicle. In the control stage, it calculates the vehicle's lateral and longitudinal acceleration based on the output from the decision-making stage, controlling the vehicle's movement.

[0052] Information about the surrounding environment can include images captured by the vehicle-mounted camera. During the perception phase, these images can be processed.

[0053] In image analysis, including feature extraction, segmentation, matching, and recognition, the quality of the initial image directly affects the design of the analysis algorithm and the accuracy of the analysis results. Therefore, image preprocessing is necessary before image analysis. The main purpose of image preprocessing is to eliminate irrelevant information such as noise in the image, enhance the detectability of effective information, and simplify the data to the maximum extent, thereby improving the reliability of feature extraction, image segmentation, matching, and recognition.

[0054] Figure 1 This is a schematic structural diagram of an image processing system.

[0055] The image processing system 100 includes a preprocessing module 110 and a processing module 120. The functions of the preprocessing module 110 and the processing module 120 can be implemented by at least one processor.

[0056] The preprocessing module 110 is used to preprocess the initial image.

[0057] The initial image can be an image captured by at least one camera. For example... Figure 1 The initial image shown may have been captured by a forward-facing camera mounted on the vehicle.

[0058] The preprocessing module 110 can perform one or more operations on the initial image, such as translation, transpose, mirroring, rotation, affine transformation, inverse perspective transformation (IPM), lens distortion correction (LDC), and resolution adjustment.

[0059] Affine transformations are used to linearly transform the coordinates of each point in an image, and can also translate the transformed coordinates by a certain distance. Through affine transformations, an image can be rotated around any center at any angle.

[0060] In intelligent driving fields such as autonomous driving and assisted driving, lane line detection is crucial. In images captured by a forward-facing camera, due to perspective effects, normally parallel lane lines and other objects may appear to intersect. Inverse perspective transformation (IPM) can eliminate this perspective effect. IPM can obtain the vertically downward projection (i.e., orthographic projection) of objects in the initial image.

[0061] Lens distortion can be introduced due to manufacturing defects and assembly process variations, leading to image distortion. Lens distortion is classified into two types: radial distortion and tangential distortion. Radial distortion is distortion distributed along the lens radius, caused by light rays bending more near the center of the lens than closer to the center. Tangential distortion occurs when the lens itself is not parallel to the camera sensor plane (imaging plane) or image plane, often due to installation misalignment when the lens is mounted on the lens module. Lens-induced image distortion can be reduced or even eliminated using LDC (Laser Distortion Control).

[0062] like Figure 1 The preprocessed image shown is obtained through LDC.

[0063] The processing module 120 can further process the preprocessed image.

[0064] For example, in the field of autonomous driving, the pre-processed image is sent to the backend processing module 120. The processing module 120 identifies the pre-processed image to determine people, vehicles, lane lines, traffic signs, etc. in the pre-processed image.

[0065] In general, different neural network models can be used to identify different types of targets. Figure 1 The processing results shown are illustrated using vehicle identification as an example.

[0066] When the initial image needs to be processed by multiple preprocessing methods to obtain the required preprocessed image, the preprocessing module 110 sequentially uses multiple preprocessing methods to preprocess the initial image, and needs to store the data obtained by each preprocessing method after the processing is completed.

[0067] like Figure 2 The diagram shown is a schematic flowchart of an image processing method.

[0068] Many vehicles are equipped with an in-vehicle surround view system, which uses at least four cameras installed around the vehicle to generate an inverse perspective mapping (IPM) image (i.e., bird's-eye view or top view) through a series of image preprocessing steps, providing assistance to the driver in scenarios such as parking.

[0069] A fisheye camera can be used to acquire the initial image. The azimuth field of view of a fisheye camera reaches 360°, and the pitch field of view can reach 90°, enabling large-area monitoring without blind spots.

[0070] Specifically, four fisheye cameras installed on the vehicle body acquire initial images. The preprocessing module 110 can perform lens distortion correction and inverse perspective transformation on the initial images acquired by each fisheye camera. Then, the results of perspective transformation of the multiple initial images are stitched together to obtain a panorama view, which is the preprocessed image.

[0071] Each of the various preprocessing methods for images, such as lens distortion correction and inverse perspective transformation, can be implemented through matrix operations or by mapping the pixel relationships between images before and after the preprocessing method. The preprocessing module 110 performs multiple preprocessing methods sequentially and generates the corresponding preprocessing result after completing each method.

[0072] Taking a vehicle driving on the right side of the road as an example, when the vehicle needs to park sideways, the parking route can be determined based on the environmental conditions on the right side of the vehicle. Image detection can be performed on the right-side region of the preprocessed surround view.

[0073] Following the preprocessing workflow, the initial image undergoes various preprocessing methods sequentially. Due to processor limitations, the image preprocessing process is relatively slow. Furthermore, aside from the final preprocessing step, each preprocessing step generates intermediate calculation results, consuming memory or other storage resources.

[0074] To address the aforementioned issues, this application provides an image preprocessing method that reduces the resource consumption associated with image preprocessing.

[0075] Figure 3 This is a schematic flowchart of an image processing method provided in an embodiment of this application.

[0076] In S210, at least one initial image is acquired.

[0077] It can receive at least one initial image sent by other devices. It can also read the at least one initial image from memory.

[0078] Each initial image can be captured by a camera device. After capturing the initial image, each camera device transmits and stores the initial image in memory.

[0079] It should be noted that one or more of the at least one initial image can also be obtained by performing preliminary processing on images acquired by the camera device. For example, images acquired by the camera device can be preprocessed using at least one preprocessing method to obtain at least one initial image. This application does not limit the method of obtaining the initial image, but rather focuses on the ability to perform subsequent preprocessing based on the mapping relationship.

[0080] In S220, a first preprocessed image is obtained based on the at least one initial image and the first mapping relationship. There is a first mapping relationship between the pixel positions of the at least one initial image and the first preprocessed image. The first mapping relationship corresponds to multiple preprocessing methods.

[0081] The preprocessing methods corresponding to the first mapping relationship can include one or more of affine transformation, IPM, LDC, and geometric transformation. It should be understood that these preprocessing methods may also include other methods. Geometric transformation includes at least one of translation, transpose, mirroring, rotation, and resolution adjustment.

[0082] In other words, by using the first mapping relationship to process the initial image, the result obtained after processing by these multiple preprocessing methods can be achieved.

[0083] Through steps S210 to S220, the first mapping relationship is used to preprocess at least one initial image, eliminating the need to perform calculations for each preprocessing method separately, avoiding the storage of intermediate data generated by each preprocessing method, and reducing the occupation of storage resources.

[0084] Demand information can be obtained, which indicates the various preprocessing methods. A first mapping relationship can be generated based on the demand information. The demand information can be used to indicate the requirements of subsequent image processing for these various preprocessing methods.

[0085] When the requirements for preprocessing methods change in subsequent image processing, the apparatus for performing subsequent image processing can generate requirement information and transmit it to the image processing apparatus for executing S210 to S220.

[0086] Based on the demand information, a first mapping relationship is generated. On the one hand, this makes the determination of the first mapping relationship more flexible; on the other hand, it eliminates the need to store a large number of mapping relationships, allowing the generation of only the required first mapping relationship when needed, thus reducing the consumption of storage resources.

[0087] This means that preprocessing can be performed on the entire region of each initial image, or only on a portion of the region.

[0088] Furthermore, based on the first mapping relationship, the first region image located in the first region of interest (ROI) of the at least one initial image can be preprocessed to obtain the first preprocessed image.

[0089] By preprocessing the image located in the first ROI, the amount of data requiring preprocessing is reduced, thus reducing the time required for preprocessing. It should be understood that the first ROI is a partial region in the initial image.

[0090] The initial images used by vehicle-mounted cameras exhibit variations in the locations of people, vehicles, lane lines, traffic signs, etc. When a vehicle needs to detect different types of targets, preprocessing can be performed only on the ROIs corresponding to each type of target in the initial image.

[0091] Optionally, multiple ROIs can correspond to the same preprocessing method. Alternatively, multiple ROIs can have a one-to-one correspondence with multiple preprocessing methods. Or, at least two ROIs can correspond to different preprocessing methods. For each ROI, the preprocessing method corresponding to that ROI can be used to preprocess the region image within that ROI.

[0092] If subsequent image processing only requires processing a portion of the preprocessed result of the initial image, the Region of Interest (ROI) corresponding to that portion can be identified in the initial image. Preprocessing only the regions of the initial image that fall within the ROI reduces the amount of data requiring preprocessing, conserves processing resources, and shortens processing time.

[0093] If subsequent image processing raises multiple requirements for image preprocessing, that is, if different preprocessing methods are needed to obtain the preprocessed results of the initial image, then the number of ROIs can be multiple, and each ROI can correspond to a requirement for a preprocessing method.

[0094] Based on the correspondence between multiple Regions of Interest (ROIs) and multiple mapping relationships, at least one region image of an initial image located within a given ROI can be preprocessed using the mapping relationship corresponding to each ROI. If a first ROI corresponds to a first mapping relationship, then the first region image of the first ROI is preprocessed according to the first mapping relationship.

[0095] In other words, a mapping relationship corresponding to any one or more second ROIs, or each second ROI, can be used to preprocess the second region image located in the second ROI in the at least one initial image, where the second ROI is the ROI other than the first ROI among the at least one ROIs.

[0096] It should be understood that for each second ROI, a second preprocessed image can be obtained by preprocessing at least one initial image located in a second region of that second ROI using the second mapping relationship corresponding to that second ROI. The second mapping relationship exists between the pixel positions of at least one initial image and the second preprocessed image.

[0097] It should be understood that each ROI can correspond to a mapping relationship. Each ROI can be preprocessed using the mapping relationship between pixel positions in the initial image and the preprocessed image. When each ROI requires preprocessing using multiple methods, preprocessing the region image located in each ROI using a mapping method avoids storing intermediate data generated by each preprocessing method, thus reducing the consumption of storage resources.

[0098] It should be understood that each ROI can consist of only one continuous region or multiple consecutive, separate regions. When multiple ROIs use the same preprocessing method, they may not overlap, avoiding repetitive processing of the same region and improving image processing efficiency. When multiple ROIs use different preprocessing methods, they may or may not overlap.

[0099] Preprocessing of region images within multiple ROIs can be performed serially or in parallel. Parallel preprocessing of region images within multiple ROIs can reduce the time required for image preprocessing.

[0100] The initial image can come from an in-vehicle camera device. It can be captured by the in-vehicle camera device. That is, steps S210 to S220 can be used to preprocess the initial image captured by the in-vehicle camera device. Further processing of the preprocessed image obtained in S220 can be used to achieve image recognition, prediction, and other functions for autonomous driving / assisted driving.

[0101] The at least one ROI can be determined based on the vehicle's driving status, and an onboard camera device for initial image acquisition is installed on the vehicle. Compared to when the vehicle is traveling horizontally, when the vehicle is going uphill, downhill, or bumpy, the angle at which the onboard camera acquires the initial image changes, and the area requiring detection of various types of targets changes.

[0102] Different vehicle driving states can correspond to different ROI locations.

[0103] Adjusting the position of the ROI based on the vehicle's driving status avoids situations where the preprocessing results of the initial image located in the ROI region do not meet the requirements of subsequent image processing due to changes in the vehicle's driving status.

[0104] Driving status information can be obtained, which indicates the vehicle's driving status. The location of the first ROI can be determined based on this driving status information. For example, different driving status information can correspond to different locations of the first ROI.

[0105] It should be understood that the first mapping relationship can be used to map each pixel in the initial image. When preprocessing a first region image located in a first ROI in at least one initial image, mapping is performed only on individual pixels in the first region image.

[0106] Alternatively, the first mapping relationship can be used only to map individual pixels within the first ROI. When the vehicle's driving state changes, a new first mapping relationship can be generated based on the required information.

[0107] For at least one initial image, preprocessing can be performed serially or in parallel.

[0108] The serial preprocessing method can be understood as processing each pixel in at least one initial image sequentially. For each pixel to be processed in the at least one initial image, the position of the pixel in the first preprocessed image can be determined sequentially using a first mapping relationship, thereby generating a first preset image. The pixel to be processed can be a pixel in at least one Region of Interest (ROI).

[0109] Parallel processing for preprocessing can be understood as simultaneously processing multiple pixels in at least one initial image. A first mapping relationship can be used to simultaneously determine the positions of multiple pixels to be processed in the first preprocessed image.

[0110] Preprocessing at least one initial image in parallel can reduce image preprocessing time and increase processing speed. To achieve parallel preprocessing, at least one initial image can be divided. Each initial image is stored in memory according to a certain format. Dividing at least one initial image into multiple image blocks can be understood as storing each image block in that format. In other words, when dividing at least one initial image, it is necessary to copy that initial image, making the process relatively complex.

[0111] Alternatively, the first mapping relationship can include multiple sub-mapping relationships, each corresponding to a sub-region. That is, each sub-mapping relationship is used for mapping pixel positions within a sub-region.

[0112] Based on these multiple sub-mapping relationships, the initial images in these multiple sub-regions can be preprocessed in parallel. Thus, while reducing image preprocessing time through parallel processing, the division of at least one initial image is avoided, making parallel preprocessing of the initial images simpler.

[0113] It should be understood that preprocessing can also be performed in parallel for at least one initial image located in a region within each ROI. For example, for a first ROI, a first mapping relationship may include multiple sub-mapping relationships, each corresponding to a sub-region within the first ROI. Thus, images located in multiple sub-regions within the first ROI can be preprocessed in parallel based on these multiple sub-mapping relationships.

[0114] The image processing method provided in this application embodiment can be applied in the fields of autonomous driving or assisted driving. Specifically, please refer to... Figures 4 to 6 The initial image can be captured by the vehicle's onboard camera.

[0115] The image processing method provided in this application embodiment can also be applied to scenarios such as security monitoring, VR devices, and projection devices that require image processing through multiple preprocessing methods.

[0116] In security monitoring scenarios, it is necessary to preprocess the initial images captured by multiple cameras used for image acquisition. Cameras installed in supermarkets primarily monitor locations such as the entrance and checkout counters. The initial images captured by all security cameras can be preprocessed uniformly. Multiple Regions of Interest (ROIs) within the initial images from multiple cameras are preprocessed in parallel for LDC and IPM (Local Area Conversion) calculations. These preprocessed ROI images are then stitched together and displayed for viewing by relevant personnel.

[0117] By preprocessing multiple Regions of Interest (ROIs) in parallel from initial images captured by multiple cameras, processing efficiency can be improved. Compared to multiple processors preprocessing initial images captured by a single camera, having a single processor uniformly preprocess multiple ROIs avoids the inconsistency in time points between different regions in the stitched image resulting from the varying processing capabilities of different processors.

[0118] For VR devices and projection devices, the acquired initial image typically requires various preprocessing methods such as resolution adjustment and radiometric transformation. Using S210 to S220 to preprocess the initial image can avoid storing intermediate data generated by each preprocessing method, thus reducing the consumption of storage resources.

[0119] Figure 4 This is a schematic structural diagram of an image processing system provided in an embodiment of this application.

[0120] The image processing system 300 can be applied to fields such as autonomous driving and driver assistance. The image processing system 300 can be located in a vehicle.

[0121] The image processing system 300 includes a first processing module 310 and a second processing module 320.

[0122] The first processing module 310 receives the initial image captured by the camera.

[0123] The camera device used to capture the initial image can be a vehicle-mounted camera. Figure 4 The following explanation uses the initial image captured by a forward-facing camera mounted on a vehicle as an example. The forward-facing camera captures images in real time to obtain an initial image with distortion.

[0124] The locations of multiple ROIs can be determined based on the vehicle's driving status.

[0125] After acquiring the initial image, the first processing module 310 can determine the region image located in each of the multiple Regions of Interest (ROIs) within the initial image. For each region image in the ROI, preprocessing is performed using a mapping relationship with the pixel positions corresponding to that ROI, thereby determining the position of each pixel in that ROI in the preprocessed image, which is the preprocessing result. Furthermore, this preprocessing of the region images in multiple ROIs can be performed in parallel.

[0126] After the initial image is processed by the first processing module 310, a preprocessed image corresponding to each ROI can be obtained.

[0127] The preprocessing of the first processing module 310 can perform one or more functions such as affine transformation, inverse perspective transformation, LDC, and resolution adjustment on the image.

[0128] like Figure 4 As shown, after the first processing module 310 preprocesses the initial image captured by the front-view camera, three preprocessed images can be obtained, namely image 1 to image 3.

[0129] The locations of targets such as people, vehicles, lane lines, and traffic signs vary in the initial image. For example, the initial image captured by a forward-facing camera (e.g., ...) may differ in location. Figure 4 In the initial image shown, traffic signs are generally located in the upper part of the initial image, vehicles are generally located in the middle part of the preprocessed image, and lane lines are generally located in the lower part of the preprocessed image.

[0130] The ROI corresponding to Image 1 is the area in the initial image where targets such as people and vehicles typically appear; the ROI corresponding to Image 2 is the area in the initial image where targets such as lane lines typically appear; and the ROI corresponding to Image 3 is the area in the initial image where targets such as traffic signs typically appear.

[0131] The second processing module 320 requires different image resolutions when processing different types of targets. For example, images of areas containing targets such as people and vehicles require higher resolution to achieve more accurate vehicle driving planning; while images of areas containing targets such as lane lines can achieve lane line detection using lower resolution images.

[0132] Each ROI can correspond to different image requirements of the second processing module 320.

[0133] Image 1 implements LDC and resolution enhancement; that is, the preprocessing method that can be implemented using the mapping relationship corresponding to Image 1 is LDC and resolution enhancement. The preprocessing method that can be implemented using the mapping relationship corresponding to Image 2 is LDC and resolution reduction. The preprocessing method that can be implemented using the mapping relationship corresponding to Image 3 is LDC.

[0134] The second processing module 320 is used to further process the preprocessed image. The second processing module 320 can use neural networks or traditional computer vision methods to perform functions such as target detection, target classification, and ranging on the preprocessed image.

[0135] The first processing module 310 can be located in an image signal processor (ISP). The ISP can be a processor in an image acquisition device such as a camera.

[0136] The second processing module 320 can use one or more processors such as a central processing unit (CPU), a digital signal processor (DSP), or a graphics processing unit (GPU).

[0137] The mapping relationship can be stored in memory as a mapping table. The first processing module 310 can use this mapping table when processing the initial image. Alternatively, the mapping relationship can also be a mapping function or any other form that can represent the correspondence between pixel positions.

[0138] The mapping relationship corresponding to each ROI can be determined based on the mapping relationship of each preprocessing method required for the subsequent image processing of that ROI, or it can be determined based on the parameters of the matrix operation corresponding to each preprocessing method.

[0139] When subsequent image processing for a certain ROI requires multiple preprocessing methods, the mapping relationship between pixel positions in the initial image and the preprocessed image can be used to process the initial image and generate the preprocessed image. This eliminates the need to generate intermediate results, reduces the amount of data generated, and saves storage and processing resources.

[0140] When each operation is performed sequentially using matrix operations or other methods, in order to avoid interference with the initial image during the preprocessing of one combination, thereby affecting the preprocessing of other combinations, the initial image can be copied for each combination. However, copying the initial image consumes a lot of resources.

[0141] When using a mapping method for preprocessing, there is no need to copy the initial image, reducing the consumption of storage resources.

[0142] The pixel position mapping table stores the coordinates of each pixel in the initial image within the preprocessed image. The first processing module 310 uses this mapping table and employs a bilinear interpolation algorithm to determine the preprocessed image. Specifically, see [link to documentation]. Figure 5 Explanation.

[0143] It should be understood that after the camera captures the initial image, it can store the initial image in the memory or other storage corresponding to the first processing module 310. The preprocessed image generated by the first processing module 310 can be stored in the memory corresponding to the first processing module 310, the memory corresponding to the second processing module 320, or other storage.

[0144] Figure 5 This is a schematic diagram of an image processing method provided in an embodiment of this application.

[0145] In the image preprocessing process, each pixel in the preprocessed image can be understood as originating from a pixel in the initial image before preprocessing, meaning there is a correspondence between the pixel positions in the initial image and the preprocessed image.

[0146] Taking camera distortion correction as an example, each pixel in the distorted image is calculated from the pixels in the distorted image captured by the camera using camera intrinsics and the distortion model. The specific relationship can be expressed as follows:

[0147] (x',y')=F(x,y)

[0148] Here, (x, y) represents the coordinates of a pixel in the initial image, and (x', y') can be understood as the pixel coordinates in the preprocessed image. The function F represents the mapping relationship. That is, the pixel (x', y') in the preprocessed image has the same color as the pixel (x, y) in the initial image. When x and y are integers, x' and / or y' may not be integers. After determining the pixel coordinates of each pixel in the initial image in the preprocessed image, the color corresponding to each integer coordinate in the preprocessed image can be determined using a bilinear interpolation algorithm.

[0149] The pixel correspondence between the initial image and the preprocessed image can be represented as a mapping table from the initial image to the preprocessed image. Processor resources such as ISP, GPU, CPU, and DSP can utilize this mapping F to generate the preprocessed image. The mapping F can be a mapping table, meaning the preprocessed image can be obtained by looking up a table.

[0150] An initial image contains a large number of pixels (such as an image with a resolution of 1920×1080, which contains 2,073,600 pixels). Even if the mapping table is queried for each pixel in the initial image in turn to generate a preprocessed image, it still takes a long time to process.

[0151] like Figure 5 As shown, multiple regions of the initial graphic can be processed in parallel. Figure 5 Let's take increasing image resolution (which can also be understood as enlarging the image) as an example.

[0152] The mapping table can include four sub-mapping relationships, each corresponding to a specific mapping relationship. Figure 5 The initial image shown contains one of four regions A1 to A4. Therefore, when generating the preprocessed image, regions A1 to A4 can be mapped in parallel using each sub-mapping relationship to generate images of regions B1 to B4 in the preprocessed image.

[0153] For more complex image preprocessing operations, the same approach can be used: during the lookup of the mapping table, multiple sub-mapping relationships in the table can be used in parallel for mapping. If the pixel coordinates in the preprocessed image indicated by the mapping table are not integers, image interpolation can be used to generate the preprocessed image.

[0154] Images can be stored in memory in various formats. Figure 6This explanation uses the YUV format. Each pixel in an image consists of three types of data: Y, U, and V. Y represents luminance (or luma), which is the grayscale value; U and V represent chrominance (or chroma), which describe the image's color and saturation, indicating the pixel's color.

[0155] An image stored in memory in YUV format has its Y data for each pixel arranged sequentially in pixel order. Following the Y data of all pixels in the image, the U and V data for each pixel are arranged adjacently in memory, also sequentially in pixel order. In other words, the U and V data of the image are interleaved after the Y data of all pixels in memory.

[0156] If we divide the initial YUV format image into image blocks to achieve parallel preprocessing of the initial image, we need to store each image block as YUV format data and then preprocess each image block.

[0157] When using a preprocessing method that uses multiple sub-mapping relationships in parallel, the storage location of the image data in the memory corresponding to each sub-mapping relationship of the initial image can be determined in parallel according to the YUV format, thereby achieving parallel image preprocessing.

[0158] like Figure 7 The following is used to implement Figure 5 The table shown illustrates the preprocessing mapping relationships for improving the overall resolution of the initial image.

[0159] The serial image preprocessing method can be understood as looking up the mapping table in the order of pixels in the initial image to determine the position of each pixel in the preprocessed image. Then, following the pixel order in the preprocessed image, the color of each pixel is calculated using a difference method.

[0160] By employing a forward-biased image preprocessing method, the mapping relationship table can be queried simultaneously for image pixels in the regions corresponding to each sub-mapping relationship of the initial image.

[0161] for Figure 5 The preprocessing method shown here, used to improve the overall resolution of the initial image, utilizes four sub-mapping relationships to determine the corresponding coordinates of pixels in the initial image located in the region corresponding to each sub-mapping relationship within the preprocessed image. These four sub-mapping relationships correspond to regions A1 to A4. Then, using the Y, U, and V data of each region, the color at the integer pixel coordinates is calculated for each region using methods such as interpolation. This allows for the parallel generation of images in regions B1 to B4 of the preprocessed image.

[0162] By mapping pixel positions into multiple sub-maps, a reasonable region division of the initial image can be achieved. Using a parallel processing preprocessing method can save the time required for image preprocessing, reduce the complexity of image preprocessing, save computing resources, and improve the efficiency of image preprocessing.

[0163] The image processing method provided in the embodiments of this application can be used to process images. Figure 2 The image processing procedure is optimized.

[0164] When a vehicle needs to perform parallel parking, the Region of Interest (ROI) in the initial images captured by the four fisheye cameras can be determined. The ROI corresponds to the image in the surround view that represents the environment to the right of the vehicle. A mapping table is then used to look up the images in the ROI to generate a preprocessed image of the right-side environment. This mapping table represents the correspondence between each pixel in the initial images captured by the four fisheye cameras and each pixel in the surround view.

[0165] On the one hand, preprocessing is performed only on the ROI regions of the initial image, reducing the amount of data that needs to be preprocessed and improving processing efficiency. On the other hand, the preprocessed image is obtained through a mapping table, eliminating the need for separate operations such as lens distortion correction and inverse perspective transformation, which improves processing efficiency and avoids the generation and storage of intermediate data, reducing the consumption of processing and storage resources.

[0166] The above text combined Figures 1 to 7 The image processing method of the embodiments of this application is described below, in conjunction with Figures 8 to 9 This document describes an image processing apparatus according to embodiments of the present application. It should be understood that the description of the image processing apparatus corresponds to the description of the image processing method; therefore, any parts not described in detail can be referred to the preceding description of the image processing method.

[0167] Figure 8 This is a schematic structural diagram of an image processing apparatus provided in an embodiment of this application.

[0168] The image processing apparatus 200 includes an acquisition module 2010 and a processing module 2020.

[0169] The acquisition module 2010 is used to acquire at least one initial image.

[0170] The processing module 2020 is used to obtain a preprocessed first preprocessed image based on the at least one initial image and a first mapping relationship, wherein there is a first mapping relationship between the pixel positions of the at least one initial image and the first preprocessed image, and the first mapping relationship corresponds to multiple preprocessing methods.

[0171] Optionally, the processing module 2020 is specifically used to preprocess the first region image located in the first ROI of the at least one initial image according to the first mapping relationship, so as to obtain the first preprocessed image.

[0172] Optionally, the number of the at least one ROI is multiple, and the first ROI corresponds to the first mapping relationship.

[0173] The processing module 2020 is further configured to preprocess the second region image located in the second ROI in the at least one initial image using a mapping relationship corresponding to any one or more, or each, second ROI, wherein the second ROI is an ROI other than the first ROI among the at least one ROIs.

[0174] Optionally, the at least one initial image comes from an onboard camera device, and the location of each ROI is obtained based on the vehicle's driving status.

[0175] Optionally, the first mapping relationship includes multiple sub-mapping relationships, each of which corresponds to a sub-region in the first ROI.

[0176] The processing module 2020 is specifically used to preprocess the images located in each of the sub-regions in the at least one initial image in parallel according to the plurality of sub-mapping relationships.

[0177] Optionally, the first mapping relationship includes multiple sub-mapping relationships, each of which corresponds to a sub-region.

[0178] The processing module 2020 is specifically used to preprocess the images located in each of the sub-regions in the at least one initial image in parallel according to the plurality of sub-mapping relationships.

[0179] Optionally, the various preprocessing methods include at least one of affine transformation, inverse perspective transformation (IPM), lens distortion correction (LDC), and geometric transformation.

[0180] Optionally, the acquisition module 2010 is further configured to acquire demand information, the demand information being used to indicate the various preprocessing methods;

[0181] The processing module 2020 is also used to generate the first mapping relationship based on the requirement information.

[0182] Figure 9 This is a schematic structural diagram of an image processing apparatus provided in an embodiment of this application.

[0183] The image processing apparatus 3000 includes at least one memory 3010 and at least one processor 3020, wherein the at least one memory 3010 is used to store a program and the at least one processor 3020 is used to run the program to implement the method described above.

[0184] Specifically, the processor 3020 is used to acquire at least one initial image.

[0185] The processor 3020 is further configured to obtain a preprocessed first preprocessed image based on the at least one initial image and a first mapping relationship, wherein there is a first mapping relationship between the pixel positions of the at least one initial image and the first preprocessed image, and the first mapping relationship corresponds to multiple preprocessing methods.

[0186] Optionally, the processor 3020 is specifically configured to preprocess a first region image located in a first ROI within at least one region of interest (ROI) in the at least one initial image, according to the first mapping relationship, to obtain the first preprocessed image.

[0187] Optionally, the number of the at least one ROI is multiple, and the first ROI corresponds to the first mapping relationship.

[0188] The processor 3020 is further configured to preprocess a second region image located in a second ROI in the at least one initial image using a mapping relationship corresponding to any one or more, or each, second ROI, wherein the second ROI is an ROI other than the first ROI among the at least one ROIs.

[0189] Optionally, the at least one initial image comes from an onboard camera device, and the location of each ROI is obtained based on the vehicle's driving status, the onboard camera device being mounted on the vehicle.

[0190] Optionally, the first mapping relationship includes multiple sub-mapping relationships, each of which corresponds to a sub-region in the first ROI.

[0191] The processor 3020 is specifically configured to preprocess the images located in each of the sub-regions in the at least one initial image in parallel, according to the plurality of sub-mapping relationships.

[0192] Optionally, the first mapping relationship includes multiple sub-mapping relationships, each of which corresponds to a sub-region.

[0193] The processor 3020 is specifically configured to perform parallel preprocessing on the images located in each of the sub-regions in the at least one initial image, based on the plurality of sub-mapping relationships.

[0194] Optionally, the various preprocessing methods include at least one of affine transformation, inverse perspective transformation (IPM), lens distortion correction (LDC), and geometric transformation.

[0195] Optionally, the processor 3020 is further configured to acquire demand information, the demand information being used to indicate the various preprocessing methods.

[0196] The processor 3020 is also used to generate the first mapping relationship based on the demand information.

[0197] It should be understood that the processor in the embodiments of this application can be a central processing unit (CPU), but it can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.

[0198] It should also be understood that the memory in the embodiments of this application can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate synchronous DRAM (DDR SDRAM), enhanced synchronous DRAM (ESDRAM), synchronous linked DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0199] The descriptions of the processes corresponding to the above-mentioned figures each have their own emphasis. For parts of a process that are not described in detail, please refer to the relevant descriptions of other processes.

[0200] This application also provides a computer-readable storage medium, characterized in that the computer-readable storage medium has program instructions that, when the program instructions are executed directly or indirectly, enable the method described above to be implemented.

[0201] This application also provides a computer program product containing instructions that, when run on a computing device, cause the computing device to perform the methods described above, or cause the computing device to perform the functions of the apparatus described above.

[0202] This application also provides a chip system, characterized in that the chip system includes at least one processor, and when program instructions are executed in the at least one processor, the method described above is implemented.

[0203] This application also provides a terminal, including the image processing device described above.

[0204] Furthermore, the terminal can be intelligent transportation equipment (vehicles or drones), smart home devices, intelligent manufacturing equipment, or robots. The intelligent transportation equipment can be, for example, an automated guided vehicle (AGV) or an unmanned transport vehicle.

[0205] The above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more sets of available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. A semiconductor medium can be a solid-state drive.

[0206] It should be understood that the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. A and B can be singular or plural. Additionally, the character " / " in this article generally indicates an "or" relationship between the preceding and following related objects, but it can also represent an "and / or" relationship. Please refer to the context for a more accurate understanding.

[0207] In this application, "at least one" means one or more, and "more than one" means two or more. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or multiple items. For example, at least one of a, b, or c can mean: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple.

[0208] It should be understood that in the various embodiments of this application, the order of the above-mentioned processes does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0209] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0210] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0211] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0212] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0213] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0214] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0215] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. An image processing method, characterized in that, The method includes: Acquire at least one initial image, said at least one initial image being obtained from an in-vehicle camera device; Based on the at least one initial image and the first mapping relationship, a preprocessed first preprocessed image is obtained. There is a first mapping relationship between the pixel positions of the at least one initial image and the first preprocessed image. The first mapping relationship corresponds to multiple preprocessing methods. The first mapping relationship is updated according to the driving state of the vehicle. The driving state includes one of horizontal driving, uphill, downhill, and bumpy driving. The step of obtaining a preprocessed first preprocessed image based on the at least one initial image and the first mapping relationship includes: preprocessing a first region image located in a first ROI within at least one region of interest (ROI) in the at least one initial image according to the first mapping relationship to obtain the first preprocessed image. The at least one ROI region is either a continuous region corresponding to a target to be detected or multiple continuous regions that are separated from each other. When multiple ROI regions use the same preprocessing method, the multiple ROI regions are separated from each other. The target to be detected includes one of a person, a vehicle, a lane line, or a traffic sign. The position of the ROI is associated with the acquisition angle of the at least one initial image, and the acquisition angle is associated with the vehicle's driving state.

2. The method according to claim 1, characterized in that, The number of at least one ROI is multiple, and the first ROI corresponds to the first mapping relationship. The method further includes: The second region image located in the second ROI in the at least one initial image is preprocessed using a mapping relationship corresponding to the second ROI, where the second ROI is the ROI other than the first ROI among the at least one ROIs.

3. The method according to claim 1 or 2, characterized in that, The first mapping relationship includes multiple sub-mapping relationships, each of which corresponds to a sub-region in the first ROI. The step of preprocessing the first region image located in the first ROI of at least one region of interest (ROI) in the at least one initial image according to the first mapping relationship includes: Based on the multiple sub-mapping relationships, the images located in each of the sub-regions in the at least one initial image are preprocessed in parallel.

4. The method according to claim 1, characterized in that, The first mapping relationship includes multiple sub-mapping relationships, each of which corresponds to a sub-region. The step of obtaining a preprocessed first preprocessed image based on the at least one initial image and the first mapping relationship includes: performing parallel preprocessing on the images located in each of the sub-regions in the at least one initial image according to the plurality of sub-mapping relationships.

5. The method according to any one of claims 1-4, characterized in that, The various preprocessing methods include at least one of affine transformation, inverse perspective transformation (IPM), lens distortion correction (LDC), and geometric transformation.

6. The method according to any one of claims 1-5, characterized in that, The method further includes: Obtain requirement information, which is used to indicate the various preprocessing methods; The first mapping relationship is generated based on the required information.

7. An image processing apparatus, characterized in that, include: Acquisition module and processing module; The acquisition module is used to acquire at least one initial image, the at least one initial image being from an in-vehicle camera device; The processing module is used to obtain a preprocessed first preprocessed image based on the at least one initial image and the first mapping relationship. There is a first mapping relationship between the pixel positions of the at least one initial image and the first preprocessed image. The first mapping relationship corresponds to multiple preprocessing methods. The first mapping relationship is updated according to the driving state of the vehicle. The driving state includes one of horizontal driving, uphill, downhill, and bumpy driving. The processing module is specifically used to preprocess the first region image located in the first ROI of the at least one initial image according to the first mapping relationship, so as to obtain the first preprocessed image. The at least one ROI region is a continuous region or multiple continuous regions that are separated from each other, corresponding to the target to be detected. When multiple ROI regions adopt the same preprocessing method, the multiple ROI regions are separated from each other. The target to be detected includes one of people, vehicles, lane lines, and traffic signs. The position of the ROI is related to the acquisition angle of the at least one initial image, and the acquisition angle is related to the driving state of the vehicle.

8. The apparatus according to claim 7, characterized in that, The number of at least one ROI is multiple, and the first ROI corresponds to the first mapping relationship. The processing module is further configured to preprocess the second region image located in the second ROI in the at least one initial image using a mapping relationship corresponding to the second ROI, wherein the second ROI is an ROI other than the first ROI among the at least one ROI.

9. The apparatus according to claim 7 or 8, characterized in that, The first mapping relationship includes multiple sub-mapping relationships, each of which corresponds to a sub-region in the first ROI. The processing module is specifically used to preprocess the images located in each of the sub-regions in the at least one initial image in parallel according to the multiple sub-mapping relationships.

10. The apparatus according to claim 7, characterized in that, The first mapping relationship includes multiple sub-mapping relationships, each of which corresponds to a sub-region. The processing module is specifically used to preprocess the images located in each of the sub-regions in the at least one initial image in parallel according to the multiple sub-mapping relationships.

11. The apparatus according to any one of claims 7-10, characterized in that, The various preprocessing methods include at least one of affine transformation, inverse perspective transformation (IPM), lens distortion correction (LDC), and geometric transformation.

12. The apparatus according to any one of claims 7-11, characterized in that, The acquisition module is also used to acquire demand information, which is used to indicate the various preprocessing methods; The processing module is further configured to generate the first mapping relationship based on the requirement information.

13. An image processing apparatus, characterized in that, It includes at least one memory and at least one processor, the at least one memory being used to store a program and the at least one processor being used to run the program to implement the method of any one of claims 1-6.

14. A chip, characterized in that, It includes at least one processor and an interface circuit, the interface circuit being used to provide program instructions or data to the at least one processor, the at least one processor being used to execute the program instructions to implement the method of any one of claims 1-6.

15. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores program code for execution by the device, which, when executed by the device, implements the method as described in any one of claims 1-6.

Images