Image processing method, device, computer storage medium and head-up display device

Abstract

The embodiments of the present disclosure disclose an image processing method, device, computer storage medium and head-up display device. The image processing method may include: determining the distortion type of a displayed image; matching at least one distortion adjustment method according to the distortion type; and correcting the displayed image based on at least one distortion adjustment method.

Description

Technical Field

[0001] The embodiments of the present disclosure relate to the field of image processing technology, and in particular to an image processing method, device, computer storage medium, and head-up display device. Background Art

[0002] Head Up Display (HUD) technology is a system that projects information onto the front windshield of a car, allowing the driver to view important information without diverting his or her eyes, thereby improving driving safety.

[0003] With the rapid development of new energy vehicles, the demand for smart cockpit products is growing. As an important component of the smart cockpit, HUD is becoming more and more popular and important. HUD products not only need to display rich content, but also require high-quality images projected onto the windshield to meet the personalized needs of different users for image shapes.

[0004] At present, the distortion correction method commonly used by domestic HUD manufacturers relies on the on-board diagnostics system (OBD) and external high-definition motion camera. The specific process includes: connecting the device host computer through OBD for communication, using the motion camera to focus and take pictures to identify the current distortion point, and calculating the correction data and sending it to the image generation software for use. This method has the disadvantages of high production cost, error in the correction effect of each vehicle, and inability to fully meet the personalized needs of customers. In addition, the above method requires the correction process to be completed before the vehicle is offline, which limits the possibility of adjustment during the use of the vehicle. Summary of the invention

[0005] In view of this, the embodiments of the present disclosure hope to provide an image processing method, device, computer storage medium and head-up display device; which can reduce the cost of image processing, have high adaptability, and can perform image correction during vehicle use.

[0006] The technical solution of the embodiment of the present disclosure is implemented as follows:

[0007] In a first aspect, an embodiment of the present disclosure provides an image processing method, comprising:

[0008] Determining the type of distortion of the displayed image;

[0009] Matching at least one distortion adjustment method according to the distortion type;

[0010] The displayed image is corrected based on at least one of the distortion adjustment methods.

[0011] In some examples, the correcting the displayed image based on at least one of the distortion adjustment methods includes:

[0012] Acquire a unit moving pixel of the distortion adjustment mode, and determine an upper limit number of times of calling the distortion adjustment mode according to the size of the display area and the unit moving pixel;

[0013] Determining the number of times the distortion adjustment method has been called;

[0014] According to the upper limit number and the called number of times, the displayed image is corrected based on the distortion adjustment method.

[0015] In some examples, correcting the displayed image based on the distortion adjustment method according to the upper limit number and the called number of times includes:

[0016] In the case where the number of times called is less than the upper limit, calling the distortion adjustment method to correct the displayed image to obtain a corrected image, and updating the number of times called;

[0017] In the case that the corrected image does not meet the correction requirement, the displayed image is updated using the corrected image.

[0018] In some examples, correcting the displayed image based on the distortion adjustment method according to the upper limit number and the called number of times includes:

[0019] When the number of times N called is less than the upper limit, acquiring an initial image corresponding to the display image, the initial image being a display image with a number of times called being zero;

[0020] Determine, according to the unit moving pixel, a target correction amount corresponding to N+1 times of the distortion adjustment method;

[0021] Correcting the initial image based on the target correction amount to obtain a corrected image, and updating the number of calls until the corrected image meets the correction requirement;

[0022] Outputting the corrected image that meets the correction requirements as the target image;

[0023] Wherein, N is an integer.

[0024] In some examples, determining the number of times the distortion adjustment method has been called includes:

[0025] Determine a reverse distortion adjustment method that moves pixels in the same manner as the distortion algorithm but in an opposite direction;

[0026] Obtaining a first historical call count of the distortion adjustment method and a second historical call count of the reverse distortion adjustment method;

[0027] The difference between the first historical call count and the second historical call count is used as the number of calls completed.

[0028] In some examples, the correcting the displayed image based on at least one of the distortion adjustment methods includes:

[0029] Determining an initial feature matrix corresponding to the displayed image;

[0030] Determine an affine change matrix based on the initial feature matrix and the unit moved pixel;

[0031] The displayed image is corrected using the affine transformation matrix.

[0032] In some examples, after determining an affine transformation matrix based on the initial feature matrix and the unit moved pixel, the method further includes:

[0033] Determining initial feature points of the displayed image;

[0034] Determine a reference feature point among the initial feature points, and determine the center position of the display area;

[0035] The reference feature point is moved to the center position, and the affine transformation matrix is ​​updated.

[0036] In some examples, the correcting the displayed image based on at least one of the distortion adjustment methods includes:

[0037] Determining an initial feature matrix corresponding to the displayed image;

[0038] Determine the affine change matrix based on the initial feature matrix, the number of times called, and the unit movement pixel;

[0039] Acquire an initial image corresponding to the displayed image;

[0040] The initial image is corrected using the affine transformation matrix.

[0041] In some examples, correcting the initial image using the affine transformation matrix includes:

[0042] Acquire the coordinates of the initial feature points of the initial image, and determine the coordinates of the target feature points of the corrected display image based on the affine transformation matrix;

[0043] Determine a correction offset according to the coordinates of the target feature points of the corrected display image and the coordinates of the initial feature points of the initial image;

[0044] The initial image is corrected according to the offset.

[0045] In some examples, the correcting the displayed image based on at least one of the distortion adjustment methods includes:

[0046] Grid points are generated according to the unit moving pixels, and a smile curve is changed for each grid point to correct the displayed image.

[0047] In some examples, the correcting the displayed image based on at least one of the distortion adjustment methods includes:

[0048] Generating grid points according to the unit moving pixels;

[0049] Performing smile curve changes on each grid point according to the number of times the call has been made, so as to correct an initial image corresponding to the displayed image;

[0050] The initial image is a display image whose calling times is zero.

[0051] In some examples, determining the type of distortion of the displayed image includes:

[0052] displaying the display image;

[0053] In response to a type selection instruction based on the display image, the distortion type is determined from a plurality of candidate distortion types.

[0054] In some examples, matching at least one distortion adjustment method according to the distortion type includes:

[0055] Determine a matching mapping table between the distortion type and the distortion adjustment method;

[0056] At least one distortion adjustment method is matched in the matching mapping table based on the distortion type.

[0057] In a second aspect, an embodiment of the present disclosure provides an image processing device, including:

[0058] A determination module, determining a distortion type of a displayed image;

[0059] A matching module, matching at least one distortion adjustment method according to the distortion type;

[0060] A correction module is used to correct the displayed image based on at least one of the distortion adjustment methods.

[0061] In a third aspect, an embodiment of the present disclosure provides an image processing device, comprising: a processor and a memory; the processor is used to execute instructions stored in the memory to implement the image processing method described in the first aspect.

[0062] In a fourth aspect, an embodiment of the present disclosure provides a computer storage medium, wherein the storage medium stores at least one instruction, and the at least one instruction is used to be executed by a processor to implement the image processing method as described in the first aspect.

[0063] In a fifth aspect, an embodiment of the present disclosure provides a head-up display device, the head-up display device comprising a display control unit and a display unit;

[0064] Wherein, the display control unit is configured to determine the distortion type of the displayed image;

[0065] and, matching at least one distortion adjustment method according to the distortion type;

[0066] and, correcting the displayed image based on at least one of the distortion adjustment methods;

[0067] The display unit is configured to project the corrected display image onto a windshield of a vehicle for display.

[0068] The embodiments of the present disclosure provide an image processing method, device, computer storage medium and head-up display device; on the one hand, by determining the distortion type of the displayed image, corresponding correction measures can be taken for different types of distortion, thereby improving the accuracy and effect of the correction. On the other hand, by matching at least one distortion adjustment method according to the distortion type, the flexibility and adaptability of image processing can be improved, and a suitable correction algorithm can be selected according to the specific distortion of the image. On the other hand, by correcting the displayed image based on the matched distortion adjustment method, the visual effect of the image can be effectively improved, the adverse effects of the distortion can be reduced, and the clarity and readability of the image can be improved. On the other hand, the corresponding algorithm can be matched according to different types of distortion, which is suitable for a variety of different application scenarios and image processing requirements, and can meet the personalized needs of users. On the fifth hand, there is no need to introduce external devices such as OBD and high-definition sports cameras for image processing, which reduces the cost of image processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] Figure 1 A schematic diagram of the composition of a vehicle-mounted system provided in an embodiment of the present disclosure.

[0070] Figure 2 An exemplary top view of a vehicle provided for the present disclosure.

[0071] Figure 3 An exemplary perspective view from the driver's seat of a vehicle is provided for the present disclosure.

[0072] Figure 4 A schematic diagram of the architecture of a head-up display device provided in the present disclosure.

[0073] Figure 5 A flowchart of an image processing method provided by an embodiment of the present disclosure.

[0074] Figure 6 A schematic diagram of an image user interface for determining a distortion type provided in an embodiment of the present disclosure.

[0075] Figure 7 A matching diagram of a distortion type and a distortion adjustment method provided in an embodiment of the present disclosure.

[0076] Figure 8 A schematic diagram of movement beyond the display area provided in an embodiment of the present disclosure.

[0077] Fig. 9 A schematic diagram of coordinate transformation for a left-expansion and right-contraction distortion adjustment method provided in an embodiment of the present disclosure.

[0078] Fig.10 A schematic diagram of coordinate transformation for a left-cut distortion adjustment method provided in an embodiment of the present disclosure.

[0079] Fig.11 A schematic diagram of coordinate transformation for a distortion adjustment method of counter-clockwise rotation provided in an embodiment of the present disclosure.

[0080] Fig.12 A schematic diagram of generating grid points provided in an embodiment of the present disclosure.

[0081] Fig.13 A schematic diagram of coordinate transformation for a distortion adjustment method with both ends turned up provided in an embodiment of the present disclosure.

[0082] Fig.14 A flowchart of another image processing method provided by an embodiment of the present disclosure.

[0083] Fig.15 A schematic diagram of the structure of an image processing device provided in an embodiment of the present disclosure.

[0084] Fig.16 A schematic diagram of the structure of an image processing device provided in an embodiment of the present disclosure.

[0085] The above drawings show clear embodiments of the present disclosure, which will be described in more detail below. These drawings and text descriptions are not intended to limit the scope of the present disclosure in any way, but to illustrate the concepts of the present disclosure to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0086] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0087] like Figure 1 As shown, the vehicle-mounted system 100 includes: a navigation subsystem 110, an environment detection device group 120 for obtaining the vehicle's environment during the vehicle's driving, a vehicle driving state detection device group 130, a data processing unit 140, a display control unit 150, and a display unit 160. The above components or device groups are coupled together through a communication bus 12. In some examples, the communication bus 12 is used for connection and communication between the above components or device groups. It should be noted that Figure 1 Only a portion of the in-vehicle system 100 is shown, not all of the components of the in-vehicle system 100 .

[0088] exist Figure 1 In the navigation subsystem 110, a positioning device 111 and a map information storage device 112 are included. The positioning device 111 can locate the position of the vehicle based on the global positioning system (GPS), China's Beidou system, Russia's GLONASS system, Europe's Galileo system, Japan's Quasi-Zenith Satellite System (QZSS), India's Indian Regional Navigation Satellite System (IRNSS) and other positioning systems to obtain the position information of the vehicle. The map information storage device 112 stores map information, and can obtain a navigation path leading to the destination according to the position information obtained from the positioning device 111, and display the position information and the navigation path in the map application.

[0089] exist Figure 1 In the embodiment, the environment detection device group 120 may include an on-vehicle communication device 121, a radar 122, a laser rangefinder 123, and a camera 124. These devices can obtain environmental information representing the surrounding environment of the vehicle.

[0090] The vehicle-mounted communication device 121 can communicate wirelessly with one or more devices directly or via a communication network. These devices that can communicate with the vehicle-mounted communication device 121 can be other vehicles, roadside machines or roadside stations, or mobile terminal devices used by the occupants of the vehicle. In some examples, the vehicle-mounted communication device 121 can use 3G cellular communication, such as code division multiple access (CDMA), EVDO, global system for mobile communications (GSM) / general packet radio service (GPRS), or 4G cellular communication, such as long term evolution (LTE), or 5G cellular communication. In some examples, the vehicle-mounted communication device 121 can also communicate with a wireless local area network (WLAN) using WiFi. In some embodiments, the vehicle-mounted communication device 121 can also communicate directly with the device using an infrared link, Bluetooth or ZigBee. In some examples, the vehicle-mounted communication device 121 can also communicate with the device using other wireless protocols.

[0091] Radar 122 is used to sense objects in the surrounding environment of the vehicle, and can also be used to sense the speed and / or direction of these objects. In some examples, radar 122 can use electromagnetic waves or lasers as a medium to detect objects based on time of flight (TOF) or phase-shift methods, and detect the position of the detected object, the distance to the detected object, and the relative speed. In some examples, in order to be able to detect objects located in front of, behind, or to the side of the vehicle, radar 122 can be configured at an appropriate position outside the vehicle.

[0092] The laser rangefinder 123 may utilize laser light to sense objects in the environment in which the host vehicle is located. In some embodiments, the laser rangefinder 123 may include one or more laser sources, a laser scanner, and one or more detectors, among other system components.

[0093] The camera 124 can be used to capture multiple images of the surrounding environment of the vehicle. The camera 124 can be a static camera or a video camera. In some examples, in order to obtain the external image of the vehicle, the camera 124 can be located at an appropriate position outside the vehicle. For example, in order to obtain the image in front of the vehicle, the camera 124 can be configured in the interior of the vehicle close to the front windshield. Alternatively, the camera 124 can be configured around the front bumper or radiator grille. In some examples, in order to obtain the image behind the vehicle, the camera 124 can be configured in the interior of the vehicle close to the rear window glass. Alternatively, the camera 124 can be configured around the rear bumper, trunk or tailgate. In some examples, in order to obtain the image of the side of the vehicle, the camera 124 can be configured in the interior of the vehicle close to at least one of the side windows. Alternatively, the camera 124 can be configured around the side mirrors, fenders or doors.

[0094] exist Figure 1 In the embodiment, the vehicle driving state detection device group 130 may include: a steering angle sensor 131 for detecting the steering angle of the vehicle, a vehicle speed sensor 132 for detecting the driving speed of the vehicle, and an acceleration sensor 133 for detecting the acceleration applied to the vehicle. In some examples, as shown in the dotted box, an inertial sensor 134 for detecting the position and orientation change of the vehicle based on inertial acceleration may also be included. The inertial sensor 134 may be a combination of the acceleration sensor 133 and a gyroscope in a specific implementation process.

[0095] exist Figure 1 In the embodiment, the data processing unit 140 can be implemented as a computing system having a memory, a processor, an input / output interface, and a bus connected to these. In some examples, the data processing unit 140 causes the processor to execute multiple commands through program instructions stored in the memory to process the data obtained by the navigation subsystem 110, the environment detection device group 120, and the vehicle driving state detection device group 130. In some examples, the data processing unit 140 can also partially or completely control the driving of the vehicle based on the processed data.

[0096] exist Figure 1 In the figure, as shown in the dotted box, the data processing unit 140, the display control unit 150 and the display unit 160 can serve as the main body of the head-up display (HUD) device 170. The display control unit 150 can process the received data processed by the data processing unit 140, or after receiving the data obtained by the navigation subsystem 110, the environment detection device group 120 and the vehicle driving state detection device group 130, to obtain display information to be displayed, and project the display information onto the windshield of the vehicle through the display unit 160 for display.

[0097] The Head Up Display (HUD) device projects the light of the display image output by the image source onto an imaging window (e.g., an imaging board, a windshield, etc.) through, for example, a reflective optical design, so as to display vehicle status information such as vehicle speed and fuel level, as well as indication information such as navigation and hazard warnings, at an appropriate position in front of the driver. This allows the driver to obtain relevant information such as vehicle speed and fuel level without shifting his line of sight away from the road ahead, thereby improving the driving safety factor and driving experience.

[0098] Specifically, refer to Figure 2 and Figure 3 The head-up display device can be arranged on a vehicle, and the vehicle includes a windshield 204 located at the front of the vehicle. The driver and passengers in the vehicle cabin 208 can see the front of the vehicle through the windshield 204.

[0099] exist Figure 3 In the embodiment of the present invention, the windshield 204 is visually located above the vehicle dashboard 206. The driver can turn the steering wheel 210 in the passenger cabin 208 to steer the vehicle, such as changing lanes, merging and parking the vehicle. In some embodiments, the steering wheel 210 can be retracted or omitted.

[0100] Reference Figure 3 , head-up display device 170 (see Figure 4 ) projects display information 212 (eg, a virtual image) onto a portion of windshield 204 through one or more apertures (eg, aperture 216) in instrument panel 206. Figure 4 An example size of display information 212 is shown, but display information 212 may be presented over a larger or smaller area. Examples of display information 212 include various vehicle information, such as current vehicle speed, current gear of the vehicle's transmission, engine speed, vehicle direction, current infotainment system settings, and / or other vehicle information. Heads-up display device 170 provides information to the vehicle driver without the driver having to look away from objects in front of the vehicle.

[0101] See also Figure 4 In the exemplary implementation architecture of the head-up display device 170 shown in FIG. 1 , the display control unit 150 generates a signal 42 based on the data processed by the data processing unit 40 or the data 420 transmitted by the navigation subsystem 110, the environment detection device group 20, and the vehicle driving state detection device group 30. The communication control circuit is provided in the data processing unit 40 and transmits the display information to the display control unit 150.

[0102] The display unit 160 may include a light source 161 and an optical path component 162. The light source 161 outputs light (e.g., a virtual image) based on a signal 42 from the display control unit 150 to display on the windshield 204. For example, the light source 161 may include one or more lasers and output red light, green light, and blue light.

[0103] Optionally, the display control unit 150 is configured to determine the distortion type of the displayed image; specifically, a user selection instruction based on the type of the displayed image may be received, and the distortion type may be determined from multiple candidate distortion types, or the display control unit may obtain an image that has been displayed in the display unit 160, and analyze the image to determine the distortion type of the displayed image. Furthermore, at least one distortion adjustment method is matched according to the distortion type; and the displayed image is corrected based on the at least one distortion adjustment method, and the display unit 160 is configured to project the corrected displayed image onto the windshield of the vehicle for display.

[0104] The optical path component 162 can reflect the output of the light source 161 onto the windshield 204 through the hole 216. The viewer (e.g., the driver) can view the display information 212 in the display area where the display information 212 is projected onto the windshield 204. In some examples, the optical path component 162 may include one or more reflectors (plane mirrors) and concave mirrors (magnifying glasses). The output of the light source 161 is reflected back through the reflector and magnified by the concave mirror and then reflected to the windshield 204 to form a virtual image 40 that can be visually observed by the driver. The visual effect presented by the virtual image 40 is that the virtual image 40 is projected onto the projection surface 41 at a set distance in front of the vehicle, but the real environment remains visible through the projection surface 41. In some examples, the optical path component 162 can also be omitted, and the light source 161 can directly project the display information 212 onto the windshield 204 to form a virtual image 40 on the projection surface 41.

[0105] In the related art, when the image displayed on the display unit is distorted, the distortion correction method used depends on the on-board diagnostics system (OBD) of the actual vehicle and an external high-definition motion camera. The specific process includes: connecting the device host computer to communicate through OBD, using the motion camera to focus and take pictures to identify the current distortion point, and calculating the correction data and sending it to the image generation software for use.

[0106] Existing distortion correction methods rely on the correction process through the on-board peripheral OBD (On-Board Diagnostics) port and an external high-definition motion camera before the car product comes off the assembly line. This means that the correction process requires additional hardware support, which increases cost and complexity. The need to use external equipment and each car needs to be corrected individually leads to increased production costs. Using an external camera to focus and take pictures to identify the current distortion point may cause errors in the point where the camera focuses and identifies each car, thereby affecting the accuracy of the correction; existing correction methods may not be able to fully meet customers' personalized needs for image shape because the correction process is standardized and lacks flexibility.

[0107] Based on this, the present disclosure first provides an image processing method. Figure 5 A flow chart of an image processing method is shown, and the image processing method can be applied to the above-mentioned display control unit, wherein the above-mentioned image processing method can include steps S510 to S530.

[0108] In step S510, the distortion type of the displayed image is determined.

[0109] In some example implementations, the distortion type of the displayed image may be determined first, and the distortion type may be determined by receiving a user's selection operation on the distortion type, referring to Figure 6 Optionally, when the displayed image is distorted, a graphical user interface is provided to the user, which includes a plurality of candidate distortion types, and in response to the user's selection operation on the candidate distortion type, the distortion type of the displayed image is determined. For example, the graphical user interface may display a plurality of candidate distortion types, such as candidate distortion type 1 to candidate distortion type n, and may display the candidate distortion type to be selected selected by the user in a differentiated manner. When the user triggers the "confirm" mark, the candidate distortion type to be selected is determined as the distortion type of the above-mentioned displayed image. The user may trigger the "cancel" mark to cancel the selection of the candidate distortion type to be selected and cancel the differentiated display of the candidate distortion type to be selected. The "confirm" mark may be triggered after determining a plurality of candidate distortion types to be selected to determine the distortion type of the above-mentioned displayed image.

[0110] In some examples, the display control unit can obtain the display image being displayed in the above display control unit by interacting with the display unit, and then the display control unit can perform image analysis on the above display image to determine the distortion type. Specifically, a standard image can be first obtained, and then the distortion type of the above display image can be determined based on the comparison between the display image and the standard image. The standard image is the display image under the ideal state set by the user. The above distortion type can also be determined by a distortion type determination model. The distortion type determination model is a machine learning model trained with training data for determining the distortion type of the display image. The specific training process is not described in detail here.

[0111] In some examples, the distortion type may include one or more of left narrow and right wide, left wide and right narrow, top wide and bottom narrow, bottom wide and top narrow, clockwise rotation offset, etc. This is not specifically limited in this example. For example, the distortion type may be a combination of clockwise rotation offset and top wide and bottom narrow distortion.

[0112] In step S520, at least one distortion adjustment method is matched according to the distortion type.

[0113] Reference Figure 7 The distortion adjustment methods corresponding to the above distortions may include left expansion and right contraction, left contraction and right expansion, upper expansion and lower contraction, upper contraction and lower expansion, clockwise rotation, counterclockwise rotation, right shearing, left shearing, downward shearing, upward shearing, both ends upward rotation, and both ends downward rotation, etc., and may also be customized according to user needs, which will not be described in detail in this example implementation. Specifically, a matching mapping table of distortion types and distortion adjustment methods may be determined, and then the distortion adjustment method corresponding to the distortion type may be obtained in the matching mapping table according to the distortion type.

[0114] In some example implementations of the present disclosure, after determining the above-mentioned distortion type, at least one distortion adjustment method may be matched according to the above-mentioned distortion type. Specifically, different distortion types may correspond to different distortion adjustment methods. For example, the distortion adjustment method corresponding to the distortion of narrow on the left and wide on the right is left expansion and right contraction, and the distortion corresponding to wide on the top and narrow on the bottom is contraction on the top and expansion on the bottom.

[0115] Specifically, if the above-mentioned distortion type includes one distortion type, a distortion adjustment method can be configured; if the above-mentioned distortion type is a combined distortion type composed of multiple distortion types, different distortion adjustment methods can be configured for different distortions, that is, the distortion type corresponding to the above-mentioned displayed image will correspond to multiple distortion adjustment methods.

[0116] In step S530, the displayed image is corrected based on at least one of the distortion adjustment methods.

[0117] In some example implementations of the present disclosure, after the distortion adjustment method is determined, the displayed image may be corrected using the distortion adjustment method.

[0118] Specifically, when there is only one distortion adjustment method, the distortion adjustment method can be directly called to correct the displayed image, and when there are multiple distortion adjustment methods, the algorithm is called according to the execution priority of the algorithm to correct the displayed image until the corrected displayed image meets the correction requirement.

[0119] Among them, the correction requirement may be that the difference from the standard image is less than a preset threshold. The preset threshold can be customized based on user needs and is not described in detail in this example. The standard image is the display image under the ideal state set by the user.

[0120] It should be noted that each distortion adjustment method may correspond to a correction requirement, that is, the correction requirement in the correction direction corresponding to the distortion adjustment method. For example, if the distortion adjustment method is clockwise rotation, the correction requirement corresponding to the distortion adjustment method is that the deflection angle of the displayed image after correction is zero. If the distortion adjustment method is left expansion and right contraction, the correction requirement of the distortion adjustment method is that the width of the displayed image after correction is consistent. Each distortion adjustment method may correspond to a correction requirement, which will not be described in detail in this example implementation.

[0121] The priority order of the above-mentioned distortion adjustment methods can be customized according to user needs. The above-mentioned display image can be corrected first based on one of the correction algorithms. After the image corrected by the correction algorithm meets the correction requirements in the correction direction corresponding to the correction algorithm, the display image corrected by the first correction algorithm is corrected based on the second correction algorithm, and so on, to complete the correction of the above-mentioned display image.

[0122] The image processing method provided by the embodiment of the present disclosure, on the first hand, by determining the distortion type of the displayed image, can take corresponding corrective measures for different types of distortion, thereby improving the accuracy and effect of the correction. On the second hand, by matching at least one distortion adjustment method according to the distortion type, the flexibility and adaptability of image processing can be improved, and a suitable correction algorithm can be selected according to the specific distortion of the image. On the third hand, correcting the displayed image based on the matched distortion adjustment method can effectively improve the visual effect of the image, reduce the adverse effects of the distortion, and improve the clarity and readability of the image. On the fourth hand, the corresponding algorithm can be matched according to different types of distortion, which is suitable for a variety of different application scenarios and image processing requirements, and can meet the personalized needs of users. On the fifth hand, there is no need to introduce external devices such as OBD and high-definition sports cameras for image processing, which reduces the cost of image processing.

[0123] In some example embodiments of the present disclosure, when the displayed image is corrected based on at least one of the distortion adjustment methods, the following operations may be performed for each matched distortion adjustment method.

[0124] Optionally, the number of times the distortion adjustment method has been called may be first obtained. Specifically, historical call data may be obtained, and the number of times the distortion adjustment method has been called may be determined in the historical call data.

[0125] Specifically, we can first determine the first historical call count of the distortion adjustment method, then determine a reverse distortion adjustment method with the same unit movement pixel as the distortion adjustment method and opposite movement direction, and obtain the second historical call count of the reverse distortion adjustment method, and then use the difference between the first historical call count and the second historical call count as the above-mentioned call count.

[0126] For example, assuming that the distortion adjustment method is shearing to the right, and the unit pixel movement is 2, the reverse distortion adjustment method is shearing to the left, and the unit pixel movement is also 2. If the first historical call count corresponding to the distortion adjustment method is 10 times, and the call count corresponding to the reverse distortion adjustment method is 6 times, then the number of calls corresponding to the above distortion adjustment method is 4.

[0127] In some examples, reference Figure 8 , the upper limit of the number of times the above distortion adjustment method is called can be determined according to the size of the display area. For example, if the distortion adjustment method is right shearing, the unit moving pixel is 2, then the size of the right edge of the above display image from the display area can be obtained, assuming it is 8. At this time, the upper limit of the number of times the distortion adjustment method is called is 4 times, to prevent the above display image from being offset outside the display area, resulting in unclear display of the display image. If the distortion adjustment method is called 6 times, the moving pixel is 12, which will cause incomplete image display.

[0128] In an example implementation of the present disclosure, when the above-mentioned number of calls is less than the upper limit, the above-mentioned distortion adjustment method can be called to correct the above-mentioned display image to obtain a corrected image, and then the above-mentioned number of calls is updated. Specifically, the above-mentioned number of calls is increased by one.

[0129] Then, determine whether the corrected image meets the correction requirements corresponding to the distortion adjustment method. If the corrected image does not meet the correction requirements corresponding to the distortion adjustment method, use the corrected image to update the displayed image. Specifically, continue to correct the corrected image as the displayed image until the corrected image meets the correction requirements corresponding to the distortion adjustment method, and output the corrected image as the target image.

[0130] In another example implementation of the present disclosure, when the above-mentioned number of calls N is less than the upper limit, an initial image corresponding to the above-mentioned display image can be obtained, and the above-mentioned initial image is the display image when the number of calls is zero, and then the total correction amount N+1 times can be determined as the target correction amount corresponding to the distortion adjustment method.

[0131] Then, the initial image can be corrected based on the target correction amount to obtain a corrected image, and the number of calls N can be updated. Specifically, the number of calls is increased by one, and the steps are repeated until the corrected image meets the correction requirement corresponding to the distortion adjustment method, and the corrected image that meets the correction requirement corresponding to the distortion adjustment method is output as the target image. The initial image is corrected based on the number of calls, and the system memory does not need to store multiple display images in the correction process, which reduces the storage pressure of the memory.

[0132] In some examples, when correcting a displayed image based on the above-mentioned distortion adjustment method, the initial feature matrix of the displayed image can be first determined, and then the affine change matrix can be determined based on the initial feature matrix and the unit moved pixel, and then the displayed image can be corrected based on the above-mentioned affine change matrix.

[0133] Optionally, the method may further include determining a reference feature point based on the initial feature point, determining a center position of the display area, and then moving the reference feature point to the center position, and updating the affine change matrix.

[0134] For example, in some examples, assuming that the above-mentioned distortion adjustment method is left expansion and right contraction, the initial feature matrix of the 3x3 matrix can be first constructed to obtain the unit movement pixel of this algorithm, and then the above-mentioned initial feature matrix and unit movement pixel value are used as input parameters of the interface for calculating the affine change matrix to determine the above-mentioned affine change matrix.

[0135] Optional, see Fig. 9 , the initial feature points of the above-mentioned display image can be obtained, such as points A, B, C, and D. It should be noted that the initial feature points are default feature points, that is, the display image formed by the connection of the initial feature points meets the correction requirements, that is, the display image formed after the initial feature points are connected is an ideal image. The starting point A of the display image can be used as the above-mentioned reference feature point (the position of the upper left vertex of the image is the origin) and translated to the center of the display area to obtain a new affine change matrix, in which the expansion only generates a translation matrix, and directly increases the pixels moved for the four vertices, expanding left and right (the y-axis of the point plus the movement amount), and expanding up and down (the x-axis of the point plus the movement amount).

[0136] For the distortion adjustment method of left expansion and right contraction, the y value of point A in the above initial feature points is added with the unit moving pixel, the y value of point B is subtracted with the unit moving pixel, the y value of point C is added with the unit moving pixel, and the y value of point D is subtracted with the unit moving pixel; refer to Fig. 9 Then the coordinate values ​​of the four points after addition and subtraction are calculated according to the affine transformation matrix to obtain the affine transformed Four target feature points. Then, the perspective affine change matrix can be calculated based on the target feature points and the initial feature points before the movement, and the displayed image can be corrected based on the perspective affine change matrix.

[0137] In some example implementations, the above-mentioned number of calls can be obtained, and then the affine change matrix corresponding to the initial image can be determined according to the above-mentioned initial feature matrix, the number of calls and the unit movement pixel, and the initial image can be corrected based on the above-mentioned affine change matrix.

[0138] It should be noted that the method for obtaining the affine transformation matrix has been described in detail above, so it will not be repeated here.

[0139] In some examples, the coordinates of the initial feature points of the initial image can be first obtained, and then the matrix of the target feature points of the corrected display image can be determined based on the affine transformation matrix, and the correction offset can be determined according to the coordinates of the target feature points of the corrected display image and the coordinates of the initial feature points of the initial image; the initial image can be corrected according to the offset.

[0140] Specifically, the offset may be obtained by subtracting the coordinates of the initial feature point from the coordinates of the target feature point, and then the initial image may be updated based on the offset.

[0141] The above image processing method is described below through specific embodiments.

[0142] For example, refer to Fig.10 If the above distortion type is left offset, the distortion adjustment method can be right shearing, and the number of right shearing calls is obtained. The number of right shearing calls this time is 1. The number of calls is multiplied by the unit movement pixel 2, and the total pixel value of the algorithm call movement is 2*1=2.

[0143] Then construct a 3*3 initial feature matrix, calculate the affine change matrix for right shearing, substitute the feature matrix and the total pixel value moved by the algorithm into the affine transformation interface to calculate the affine change matrix for right shearing, as follows:

[0144] The required amount of shearing to the right can be determined first, and then after translating the above-mentioned reference feature points to the center of the display area, the matrix to be sheared M is calculated according to the required amount of shearing to the right, the translation matrix during translation, and the initial feature matrix. After obtaining the matrix to be sheared M, the affine change matrix M0 for shearing to the right can be calculated based on the above-mentioned matrix to be sheared M and the translation amount of the above-mentioned translation matrix. Specifically, the detranslation matrix can be determined based on the translation matrix, and then the detranslation matrix and the matrix to be sheared M are multiplied to obtain the above-mentioned affine change matrix M0. Among them, the detranslation matrix is ​​obtained by replacing the elements representing the translation amount in the translation matrix with their opposite numbers.

[0145] Then, the coordinate values ​​of the four initial feature points ABCD are the set displayed image shape, assuming A(X0, Y0), B(X1, Y1), C(X2, Y2), D(X3, Y3), and the coordinate values ​​of the four vertices ABCD based on the initial feature points can be multiplied by the above affine transformation matrix M0 to obtain the target feature points of the corrected image. The calculated target feature points are The coordinate values ​​are set as

[0146] Then, the perspective change matrix can be calculated based on the target feature points and the initial feature points to correct the displayed image. Specifically, according to the right-cut correction point data of the entire coordinate point obtained by the perspective transformation matrix, the correction point data is subtracted from the original point data to obtain the correction offset to correct the displayed image.

[0147] For another example, assuming that the above distortion adjustment method is clockwise rotation, take the first clockwise rotation as an example, assuming that the unit movement pixel of the clockwise rotation algorithm is 2 (the pixel value moved by each algorithm can be customized), assuming that the upper limit of the clockwise rotation algorithm call is 10 times, the details are as follows:

[0148] The first clockwise rotation is performed to obtain the number of calls for the clockwise rotation. The number of calls for this clockwise rotation is 1. The number of calls is multiplied by the unit movement pixel 2, and the total pixel value of the algorithm call movement is 2*1=2. Construct a 3*3 initial feature matrix and calculate the clockwise rotation affine change matrix. Specifically, substitute the initial feature matrix and the total pixel value of the algorithm call movement into the affine transformation interface to calculate the clockwise rotation affine change matrix.

[0149] In some examples, reference Fig.11, the image starting point A (the upper left vertex position of the image is the origin) can be translated to the center of the display area; the ABCD points with the four vertices of the dot map area as the default vertex coordinates and the affine transformation matrix are used as input, and the ABCD points after affine transformation are calculated according to the element values ​​of the affine transformation matrix. Finally, all of its points are used according to the perspective transformation matrix to obtain the target feature points rotated clockwise in the entire coordinate point, and the target feature points are subtracted from the initial feature points to obtain the correction offset to correct the displayed image.

[0150] In some examples of the present disclosure, when correcting the above-mentioned displayed image, grid points can be first generated according to the unit shifted pixels, and the smile curve can be changed based on the grid points to correct the displayed image. Specifically, after determining the unit shifted pixels, it is necessary to generate a grid point array according to the resolution of the displayed image and the requirements of the distortion adjustment method. These grid points are the image coordinate points to be adjusted during the correction process.

[0151] For example, assuming that the number of calls to smile is 1, and the pixel value moved each time the smile algorithm is called is 2, multiply the number of calls by the pixel value moved each time 2, and the total pixel value moved by the algorithm call is 2*1=2, as follows:

[0152] Reference Fig.12 , when the distortion type is downward rotation at both ends, determine the distortion adjustment method for upward rotation at both ends. The moving pixel of this algorithm is 2, and then the unit moving pixel is used as the input parameter of the smile curve calculation interface to generate grid points, and the smile curve transformation is performed on each grid point. The number of points in the grid points is customized according to the display area of ​​the project, and the smile transformation is performed on these points to obtain new grid points, and then the display image is rendered to correct the above display image.

[0153] Specifically, refer to Fig.12 As shown, the above display image can first be gridded according to the settings to determine multiple grid points. The format of the grid points can be set based on user needs, such as three rows and seven columns, 21 points, etc., which will not be repeated in this example implementation.

[0154] When the distortion type is upward or downward rotation at both ends, the Y coordinate can be transformed by smile. Specifically, the square of the X-axis coordinate value of the initial feature point can be used. Multiply by the total pixels moved and sum with the Y-axis coordinate value Y0 of the initial feature point to get the Y-axis coordinate value of the target feature point

[0155] When the distortion type is left-handed or right-handed, the X coordinate can be smile transformed. Specifically, the square of the Y-axis coordinate value of the initial feature point can be used. Multiply by the total pixels moved and sum with the X-axis coordinate value X0 of the initial feature point to get the X-axis coordinate value of the target feature point

[0156] The XY values ​​of the original grid coordinate points are calculated according to the smile curve law to generate the correction point data. Fig.13 , subtract the original point data from the correction point data to obtain the correction offset to correct the displayed image.

[0157] Refer to the following Fig.14 The above image processing method is described, which may specifically include steps S1410 to S1440.

[0158] In step S1410, the distortion type of the displayed image is obtained;

[0159] In step S1420, at least one distortion adjustment method is matched according to the distortion type;

[0160] In step S1430, the displayed image is corrected based on the distortion adjustment method.

[0161] In step S1440, it is determined whether the corrected image meets a preset condition.

[0162] If not, execute step S1430, and if so, execute step S1450.

[0163] In step S1450, offset data is generated based on the corrected image, and a target image is generated based on the offset data and output.

[0164] It should be noted that each step from step S1410 to step S1450 has been described in detail above, and therefore will not be repeated here.

[0165] The image processing method provided by the embodiment of the present disclosure, in the first aspect, by determining the distortion type of the displayed image, corresponding correction measures can be taken for different types of distortion, thereby improving the accuracy and effect of the correction. In the second aspect, matching at least one distortion adjustment method according to the distortion type can improve the flexibility and adaptability of image processing, and can select a suitable correction algorithm according to the specific distortion of the image. In the third aspect, correcting the displayed image based on the matched distortion adjustment method can effectively improve the visual effect of the image, reduce the adverse effects of distortion, and improve the clarity and readability of the image. In the fourth aspect, the corresponding algorithm can be matched according to different types of distortion, which is suitable for a variety of different application scenarios and image processing requirements, and can meet the personalized needs of users. In the fifth aspect, there is no need to introduce external devices such as OBD and high-definition sports cameras for image processing, which reduces the cost of image processing. In the sixth aspect, by setting the upper limit number of times, it is avoided that the displayed image is adjusted outside the real area due to too many calls to the distortion adjustment method. In the seventh aspect, the above-mentioned initial image can be corrected based on the number of calls, and the system memory does not need to store multiple displayed images in the correction process, which reduces the storage pressure of the memory.

[0166] Furthermore, the present disclosure also provides an image processing device, referring to Fig.15 As shown, the image processing device 1500 may include a determination module 1510, a matching module 1520, and a correction module 1530. Among them:

[0167] The determination module 1510 may be used to determine the distortion type of the displayed image;

[0168] The matching module 1520 may be used to match at least one distortion adjustment method according to the distortion type;

[0169] The correction module 1530 may be configured to correct the displayed image based on at least one distortion adjustment method.

[0170] In some examples, the determination module 1510 may also be used to display a display image; in response to a user's type selection instruction based on the display image, determine a distortion type from a plurality of candidate distortion types.

[0171] In some examples, the matching module 1520 may also be used to determine a matching mapping table of distortion types and distortion adjustment methods;

[0172] At least one distortion adjustment method is matched in the matching mapping table based on the distortion type.

[0173] In some examples, the correction module 1530 can also be used to perform the following steps for each distortion adjustment method: obtain the unit moving pixel of the distortion adjustment method, and determine the upper limit number of times the distortion adjustment method is called based on the size of the display area and the unit moving pixel; determine the number of times the distortion adjustment method has been called; and correct the displayed image based on the distortion adjustment method according to the upper limit number and the number of times the distortion adjustment method has been called.

[0174] In some examples, the correction module 1530 can also be used to call the distortion adjustment method to correct the displayed image to obtain a corrected image when the number of calls is less than the upper limit, and update the number of calls; when the corrected image does not meet the correction requirements, the displayed image is updated using the corrected image.

[0175] The image processing device 1500 may also be used to output the corrected image as a target image when the corrected image meets the correction requirements.

[0176] In some examples, the correction module 1530 can also be used to obtain an initial image corresponding to the displayed image when the number of calls N is less than the upper limit, the initial image being the displayed image with zero calls; determine the target correction amount corresponding to the N+1 distortion adjustment method based on the unit moving pixel; correct the initial image based on the target correction amount to obtain a corrected image, and update the number of calls until the corrected image meets the correction requirements; output the corrected image that meets the correction requirements as the target image; wherein N is an integer.

[0177] In some examples, the correction module 1530 can also be used to determine a reverse distortion adjustment method having the same unit pixel movement as the distortion adjustment method but in the opposite direction of movement; obtain a first historical call number of the distortion adjustment method and a second historical call number of the reverse distortion adjustment method; and take the difference between the first historical call number and the second historical call number as the number of calls already made.

[0178] In some examples, the correction module 1530 can also be used to determine an initial feature matrix corresponding to a displayed image; determine an affine change matrix based on the initial feature matrix and unit shifted pixels; and correct the displayed image using the affine change matrix.

[0179] In some examples, the correction module 1530 can also be used to determine the initial feature points of the displayed image; determine the reference feature points among the initial feature points and determine the center position of the display area; move the reference feature points to the center position and update the affine transformation matrix.

[0180] In some examples, the correction module 1530 can also be used to determine an initial feature matrix corresponding to the displayed image, determine an affine change matrix based on the initial feature matrix, the number of calls, and the unit movement pixel, obtain an initial image corresponding to the displayed image, and correct the initial image using the affine change matrix.

[0181] In some examples, the correction module 1530 can also be used to obtain the coordinates of the initial feature points of the initial image, and determine the coordinates of the target feature points of the corrected display image based on the affine transformation matrix; determine the correction offset according to the coordinates of the target feature points of the corrected display image and the coordinates of the initial feature points of the initial image; and correct the initial image according to the offset.

[0182] In some examples, the correction module 1530 can also be used to determine the unit moving pixel corresponding to the distortion adjustment method; generate grid points according to the unit moving pixel, and perform smile curve changes on each grid point to correct the displayed image.

[0183] In some examples, the correction module 1530 can also be used to generate grid points based on unit moving pixels; change the smile curve of each grid point according to the number of calls to correct the initial image corresponding to the displayed image; wherein the initial image is a displayed image with a number of calls of zero.

[0184] It should be understood that the above-mentioned device embodiments are only illustrative, and the device of the present disclosure can also be implemented in other ways. For example, the division of units / modules in the above-mentioned embodiments is only a logical function division, and there may be other division methods in actual implementation. For example, multiple units, modules or components can be combined, or can be integrated into another system, or some features can be ignored or not executed.

[0185] In addition, unless otherwise specified, each functional unit / module in each embodiment of the present disclosure may be integrated into one unit / module, or each unit / module may exist physically separately, or two or more units / modules may be integrated together. The above-mentioned integrated unit / module may be implemented in the form of hardware or in the form of a software program module.

[0186] If the integrated unit / module is implemented in the form of hardware, the hardware may be a digital circuit, an analog circuit, etc. The physical implementation of the hardware structure includes but is not limited to transistors, memristors, etc. If not specifically stated, the processor may be any appropriate hardware processor, such as a CPU, a GPU, an FPGA, a DSP, an ASIC, etc. If not specifically stated, the storage unit may be any appropriate magnetic storage medium or magneto-optical storage medium, such as a resistive random access memory RRAM (Resistive Random Access Memory), a dynamic random access memory DRAM (Dynamic Random Access Memory), a static random access memory SRAM (Static Random-Access Memory), an enhanced dynamic random access memory EDRAM (Enhanced Dynamic Random Access Memory), a high-bandwidth memory HBM (High-Bandwidth Memory), a hybrid memory cube HMC (Hybrid Memory Cube), etc.

[0187] If the integrated unit / module is implemented in the form of a software program module and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present disclosure, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a memory and includes several instructions for a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the various embodiments of the present disclosure. The aforementioned memory includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, disk or optical disk, etc. Various media that can store program codes.

[0188] Please refer to Fig.16 , which shows a structural block diagram of an image processing device provided by an exemplary embodiment of the present disclosure. In some examples, the image processing device can be at least one of a smart phone, a smart watch, a desktop computer, a laptop, a virtual reality terminal, an augmented reality terminal, a wireless terminal, and a laptop. The image processing device has a communication function and can access a wired network or a wireless network. The image processing device can generally refer to one of a plurality of terminals, and those skilled in the art can know that the number of the above terminals can be more or less. It can be understood that the image processing device undertakes the calculation and processing work of the technical solution of the present disclosure, and the embodiments of the present disclosure are not limited to this.

[0189] like Fig.16 As shown, the image processing apparatus 1600 may include: at least one processor 1610 , a memory 1620 , and a communication interface 1630 .

[0190] The memory 1620 is used to store programs. Specifically, the programs may include program codes, and the program codes include computer operation instructions.

[0191] The memory 1620 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0192] The processor 1610 is used to execute the computer-executable instructions stored in the memory 1620 to implement the image processing method described in the aforementioned method embodiment. The processor 1610 may be a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present disclosure.

[0193] The image processing device 1600 may also include a communication interface 1630, so that it can communicate and interact with external devices through the communication interface 1630. In a specific implementation, if the communication interface 1630, the memory 1620 and the processor 1610 are implemented independently, the communication interface 1630, the memory 1620 and the processor 1610 can be connected to each other through a bus and complete the communication between each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc., but it does not mean that there is only one bus or one type of bus.

[0194] Optionally, in a specific implementation, if the communication interface 1630, the memory 1620 and the processor 1610 are integrated on a chip, the communication interface 1630, the memory 1620 and the processor 1610 can complete communication through an internal interface.

[0195] The present disclosure also provides a computer-readable storage medium, which may include: a U disk, a mobile hard disk, a read-only memory, a random access memory, a disk or an optical disk, and other media that can store program codes. Specifically, the computer-readable storage medium stores program instructions, and the program instructions are used for the image processing method in the above-mentioned embodiment.

[0196] The embodiments of the present disclosure also provide a computer program product, which includes computer instructions stored in a computer-readable storage medium; a processor of an image processing device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the image processing device executes to implement the image processing methods of the above-mentioned embodiments.

[0197] Those skilled in the art should be aware that in one or more of the above examples, the functions described in the embodiments of the present disclosure can be implemented using hardware, software, firmware, or any combination thereof. When implemented using software, these functions can be stored in a computer-readable medium or transmitted as one or more instructions or codes on a computer-readable medium. Computer-readable media include computer storage media and communication media, wherein communication media include any media that facilitates the transmission of computer programs from one place to another. Storage media can be any available media that can be accessed by general or special-purpose computers.

[0198] In the above embodiments, the description of each embodiment has its own emphasis. For the part not described in detail in a certain embodiment, please refer to the relevant description of other embodiments. The technical features of the above embodiments can be combined arbitrarily. In order to make the description concise, all possible combinations of the technical features in the above embodiments are not described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0199] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the inventions claimed herein. The present disclosure is intended to cover any variations, uses or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary technical means in the art that are not claimed in the present disclosure.

[0200] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. An image processing method, characterized in that: include: Determining the type of distortion of the displayed image; Matching at least one distortion adjustment method according to the distortion type; Correcting the displayed image based on at least one of the distortion adjustment methods; Wherein, when the displayed image is corrected based on at least one of the distortion adjustment modes, determining the number of times the distortion adjustment mode has been called for each matched distortion adjustment mode includes: Determine a reverse distortion adjustment method having the same unit moving pixel as that of the distortion adjustment method and an opposite moving direction; Obtaining a first historical call count of the distortion adjustment method and a second historical call count of the reverse distortion adjustment method; The difference between the first historical call count and the second historical call count is used as the number of calls completed.

2. The method according to claim 1, characterized in that The correcting the displayed image based on at least one of the distortion adjustment methods includes: Acquire a unit moving pixel of the distortion adjustment mode, and determine an upper limit number of times of calling the distortion adjustment mode according to the size of the display area and the unit moving pixel; According to the upper limit number and the called number of times, the displayed image is corrected based on the distortion adjustment method.

3. The method according to claim 2, characterized in that According to the upper limit number and the called number, the displayed image is corrected based on the distortion adjustment method, including: In the case where the number of times called is less than the upper limit, calling the distortion adjustment method to correct the displayed image to obtain a corrected image, and updating the number of times called; In the case that the corrected image does not meet the correction requirement, the displayed image is updated using the corrected image.

4. The method according to claim 2, characterized in that: According to the upper limit number and the called number, the displayed image is corrected based on the distortion adjustment method, including: When the number of times N called is less than the upper limit, acquiring an initial image corresponding to the display image, the initial image being a display image with a number of times called being zero; Determine, according to the unit moving pixel, a target correction amount corresponding to N+1 times of the distortion adjustment method; Correcting the initial image based on the target correction amount to obtain a corrected image, and updating the number of calls until the corrected image meets the correction requirement; Outputting the corrected image that meets the correction requirements as the target image; Wherein, N is an integer.

5. The method according to any one of claims 2 to 4, characterized in that: The correcting the displayed image based on at least one of the distortion adjustment methods includes: Determining an initial feature matrix corresponding to the displayed image; Determine an affine change matrix based on the initial feature matrix and the unit moved pixel; The displayed image is corrected using the affine transformation matrix.

6. The method according to claim 5, characterized in that After determining an affine change matrix based on the initial feature matrix and the unit moved pixel, the method further includes: Determining initial feature points of the displayed image; Determine a reference feature point among the initial feature points, and determine the center position of the display area; The reference feature point is moved to the center position, and the affine transformation matrix is ​​updated.

7. The method according to claim 2 or 3, characterized in that: The correcting the displayed image based on at least one of the distortion adjustment methods includes: Determining an initial feature matrix corresponding to the displayed image; Determine an affine change matrix based on the initial feature matrix, the number of times called, and the unit movement pixel; Acquire an initial image corresponding to the display image, wherein the initial image is a display image that has been called a zero number of times; The initial image is corrected using the affine transformation matrix.

8. The method according to claim 4, characterized in that The correcting the displayed image based on at least one of the distortion adjustment methods includes: Determining an initial feature matrix corresponding to the displayed image; Determine an affine change matrix based on the initial feature matrix, the number of times called, and the unit movement pixel; Acquire an initial image corresponding to the displayed image; The initial image is corrected using the affine transformation matrix.

9. The method according to claim 7, characterized in that: The correcting the initial image by using the affine transformation matrix includes: Acquire the coordinates of the initial feature points of the initial image, and determine the coordinates of the target feature points of the corrected display image based on the affine transformation matrix; Determine a correction offset according to the coordinates of the target feature points of the corrected display image and the coordinates of the initial feature points of the initial image; The initial image is corrected according to the offset.

10. The method according to any one of claims 2 to 4, characterized in that: The correcting the displayed image based on at least one of the distortion adjustment methods includes: Grid points are generated according to the unit moving pixels, and a smile curve is changed for each grid point to correct the displayed image.

11. The method according to claim 2 or 3, characterized in that: The correcting the displayed image based on at least one of the distortion adjustment methods includes: Generating grid points according to the unit moving pixels; Performing smile curve changes on each grid point according to the number of times the call has been made, so as to correct an initial image corresponding to the displayed image; The initial image is a display image whose calling times is zero.

12. The method according to claim 4, characterized in that The correcting the displayed image based on at least one of the distortion adjustment methods includes: Generating grid points according to the unit moving pixels; A smile curve is changed for each grid point according to the number of times the smile curve has been called, so as to correct the initial image corresponding to the displayed image.

13. The method according to claim 1, characterized in that Determining the distortion type of the displayed image includes: displaying the display image; In response to a type selection instruction based on the display image, the distortion type is determined from a plurality of candidate distortion types.

14. The method according to claim 1, characterized in that The matching of at least one distortion adjustment method according to the distortion type includes: Determine a matching mapping table between the distortion type and the distortion adjustment method; At least one distortion adjustment method is matched in the matching mapping table based on the distortion type.

15. An image processing device, characterized in that: include: A determination module, used to determine the distortion type of the displayed image; A matching module, used to match at least one distortion adjustment method according to the distortion type; A correction module, configured to correct the displayed image based on at least one of the distortion adjustment methods; Among them, the correction module is also used for: Determine a reverse distortion adjustment method having the same unit moving pixel as that of the distortion adjustment method and an opposite moving direction; Obtaining a first historical call count of the distortion adjustment method and a second historical call count of the reverse distortion adjustment method; The difference between the first historical call count and the second historical call count is used as the number of calls completed.

16. An image processing device, characterized in that: The image processing device comprises: a processor and a memory; the processor is used to execute instructions stored in the memory to implement the image processing method according to any one of claims 1 to 14.

17. A computer storage medium, characterized in that: The storage medium stores at least one instruction, and the at least one instruction is used to be executed by a processor to implement the image processing method according to any one of claims 1 to 14.

18. A head-up display device, characterized in that: The head-up display device includes a display control unit and a display unit; Wherein, the display control unit is configured to determine the distortion type of the displayed image; and, matching at least one distortion adjustment method according to the distortion type; and, correcting the displayed image based on at least one of the distortion adjustment methods; And, when correcting the displayed image based on at least one of the distortion adjustment methods, determining the number of times the distortion adjustment method has been called for each matched distortion adjustment method, including: Determine a reverse distortion adjustment method having the same unit moving pixel as that of the distortion adjustment method and an opposite moving direction; Obtaining a first historical call count of the distortion adjustment method and a second historical call count of the reverse distortion adjustment method; The difference between the first historical call number and the second historical call number is used as the number of calls completed; The display unit is configured to project the corrected display image onto a windshield of the vehicle for display.

Images