An image processing method and apparatus

Abstract

The application discloses an image processing method and device, which comprises the following steps: collecting a video in front of a vehicle through a camera; processing the video collected by the camera, so that the processed video is clearer; and projecting the processed video on a windshield glass of the vehicle, so that a driver can see the road condition information more clearly through the projected picture on the windshield glass in a blurred vision scene, the accident rate is reduced, and the driving safety is improved.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to an image processing method and device. Background Technology

[0002] In recent years, heads-up displays (HUDs) have been increasingly used in automobiles. HUDs project important information, such as vehicle speed and navigation, onto the windshield, allowing drivers to see this information without looking down, thus reducing driver distraction and improving driving safety. Augmented reality heads-up displays (ARHUDs) can project images at a focal length close to the user's eye, combining the image projected onto the windshield with the real interface for a more realistic and natural driving assistance experience.

[0003] However, in situations with poor visibility, such as rain or darkness, even if ARHUD can project driving or navigation information in front of the driver, the driver may still experience reduced driving safety because they cannot clearly see the road conditions, vehicles, pedestrians, etc. Summary of the Invention

[0004] This application provides an image processing method and apparatus to improve driving safety.

[0005] In a first aspect, this application provides an image processing method, the method comprising: acquiring a first image, wherein the first image is an image corresponding to a video stream of road conditions in front of a vehicle captured by a camera; then processing the first image to obtain a second image; and finally projecting the second image onto the windshield of the vehicle for display.

[0006] The above technical solution can process the video stream captured by the camera in front of the vehicle, making the processed image more clearly reflect road conditions, reducing the accident rate caused by the driver's inability to see the road conditions in poor visibility scenarios, and improving driving safety.

[0007] In one possible design, processing the first image to obtain the second image includes:

[0008] The video perspective of the first image is adjusted to obtain an image from the driver's perspective; the display range of the image from the driver's perspective is cropped to the size that the head-up display (HUD) can project, and the cropped image is used as the third image; the third image is optimized to obtain a fourth image; key elements are extracted from the fourth image and processed, and the processed image is used as the second image, wherein the key elements include pedestrians, vehicles, or trees.

[0009] The above technical solutions can be used to process images captured by cameras, including perspective conversion, cropping, and sharpening, making the processed images clearer from the driver's perspective, thereby reducing the accident rate in scenarios with poor visibility.

[0010] In one possible design, processing the key element includes:

[0011] The key element is virtualized, and the projection position of the key element is determined; the key element and the virtualized key element are displayed at the projection position.

[0012] The above technical solutions enable drivers to have a clearer understanding of road conditions from the processed images, thereby improving driving safety.

[0013] In one possible design, the key elements are virtualized, including:

[0014] Obtain the icon or model corresponding to the key element from the pre-stored material library; overlay the icon or model corresponding to the key element onto the layer of the key element.

[0015] The above technical solution allows for the selection of icons or models corresponding to key elements in an image from a resource library, which are then overlaid on the image. This enables drivers to see the icons or models representing key elements on the image, further improving driving safety.

[0016] In one possible design, the key element includes the Nth frame image.

[0017] Determining the projection position of the Nth frame image, which includes the key element, includes:

[0018] Obtain the first position of the Nth frame image of the key element, the duration of the Nth frame image, and the image processing delay; wherein, the first position is the position before the Nth frame image is processed; based on the first position of the Nth frame image, the duration of the Nth frame image, and the image processing delay, determine the projection position of the Nth frame image of the key element.

[0019] The above technical solution can achieve a good overlap between the real object seen from the driver's perspective and the projected image, achieving a combination of virtual and real effects, realizing what you see is what you get, improving user experience, and enhancing driving safety.

[0020] In a second aspect, this application provides an image processing apparatus, the image processing apparatus including a processor; a memory and one or more computer programs; wherein the one or more computer programs are stored in the one or more memories, and the one or more computer programs include instructions that, when invoked and executed by the one or more processors, cause the image processing apparatus to perform the methods described in the first aspect and any possible design of the first aspect.

[0021] Thirdly, this application also provides an image processing apparatus, which includes modules / units for performing the first aspect or any possible design of the first aspect; these modules / units can be implemented in hardware or by hardware executing corresponding software.

[0022] Fourthly, this application also provides a computer-readable storage medium storing instructions that, when executed on an image processing device, cause the image processing device to perform the method of the first aspect and any possible design of the first aspect.

[0023] Fifthly, this application also provides a computer program product that, when run on an image processing device, causes the image processing device to execute the method of the first aspect of the embodiments of this application and any possible design of the first aspect.

[0024] For the various aspects of the second to fifth aspects mentioned above, and the technical effects that each aspect may achieve, please refer to the above description of the technical effects that the various possible solutions for the first aspect may achieve, which will not be repeated here. Attached Figure Description

[0025] Figure 1 A schematic diagram of the display interface of a HUD provided in an embodiment of this application;

[0026] Figure 2 This is a schematic diagram of an application scenario provided by an embodiment of this application;

[0027] Figure 3 A schematic diagram of a software module provided in an embodiment of this application;

[0028] Figure 4 A flowchart of an image processing method provided in an embodiment of this application;

[0029] Figure 5A schematic diagram provided for an embodiment of this application;

[0030] Figure 6 A schematic block diagram of an image processing apparatus provided in an embodiment of this application;

[0031] Figure 7 This is a schematic diagram of an image processing device provided in an embodiment of this application. Detailed Implementation

[0032] The technical solutions in the embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0033] For example, such as Figure 1 The image shown is a schematic diagram of the display interface of a HUD provided in an embodiment of this application. Figure 1 The image shown can be displayed on the vehicle's windshield, including navigation information, speed, and road conditions. In most vehicles, navigation and speed information are displayed on the infotainment system, requiring the driver to look down to access this information. Projecting (or displaying) this information on the windshield eliminates the need for the driver to look down at the infotainment system, improving driving safety. However, in situations with poor visibility, even with HUD technology projecting the image onto the windshield, the driver still cannot clearly see the road conditions, significantly reducing driving safety.

[0034] In view of this, embodiments of this application provide an image display method that performs image processing on the image projected onto the windshield to make key information in the image, such as pedestrians and vehicles, clearer, thereby enabling the driver to see road conditions more clearly and improving driving safety.

[0035] For example, such as Figure 2 The diagram shown is an application scenario illustration provided by an embodiment of this application. (See attached image.) Figure 2 As shown, this application scenario may include a camera 10, a vehicle infotainment system 20, and a display device 30. The camera 10 is used to capture video of the area in front of the vehicle, and can send the captured video to the vehicle infotainment system 20. For example, the camera 10 can be a dashcam.

[0036] In some embodiments, the vehicle-mounted unit 20 receives video (also referred to as images or video streams) of the front of the vehicle sent by the camera 10, and processes the video captured by the camera 10 to ensure that the driver can clearly see the video image, avoiding situations where blurred vision prevents the driver from seeing the road conditions clearly. The vehicle-mounted unit 20 can send the processed video to the display device 30, which can project the processed video onto the windshield in front of the vehicle. For example, the display device 30 can be the optical engine of a HUD device.

[0037] like Figure 3 The diagram shown is a schematic diagram of a software module provided in an embodiment of this application. Figure 3 In the schematic diagram shown, the camera 10 may include a video acquisition module 101; the vehicle-mounted system 20 may include a vehicle-mounted system processing module 201, which may specifically include a perspective conversion module 201A, an image cropping module 201B, an image enhancement module 201C, and a virtual-real combination processing module 201D; the display device 30 may include a display module 301.

[0038] The system includes a video acquisition module 101 for acquiring video from the front of the vehicle; a perspective conversion module 201A for converting the perspective of the video from the front of the vehicle acquired by the video acquisition module 101 to obtain a video from the driver's perspective; an image cropping module 201B for cropping the video from the driver's perspective to make it fit within the projection range of the HUD; an image enhancement module 201C for optimizing the cropped video, such as image sharpening and noise reduction, to make the video clearer; and a virtual-real combination processing module 201D for virtualizing key elements in the video, such as pedestrians, vehicles, and trees (e.g., labeling a person's image at the pedestrian's location), calculating the processing delay, and then projecting the key elements onto the HUD at the time corresponding to the calculated processing delay, thus overlapping with the real objects and achieving the purpose of virtual-real combination. A display module 301 receives the processed video from the vehicle's infotainment system processing module 201 and projects the processed video onto the vehicle's windshield.

[0039] It should be understood that the image processing method in the embodiments of this application can be applied to vehicle HUD devices, as well as other devices, such as projection devices. Any device capable of image projection can apply the method of this application, and no specific limitation is made.

[0040] The image processing method involved in the embodiments of this application is described below with reference to flowchart illustrations. For example, as shown... Figure 4 The diagram shown is a flowchart of an image processing method provided in an embodiment of this application. (See attached diagram.) Figure 4 As shown, the method may include the following steps:

[0041] Step 401: Camera 10 captures video from the front of the vehicle.

[0042] In some embodiments, the camera 10, such as a dashcam, can capture video in front of the vehicle in real time, which refers to real-time road video located in front of the vehicle.

[0043] Step 402: The camera 10 sends the video captured in front of the vehicle to the vehicle system 20.

[0044] Step 403: The vehicle-mounted unit 20 processes the video collected from the front of the vehicle to obtain the processed video.

[0045] In some embodiments, after receiving the video of the front of the vehicle sent by the camera 10, the vehicle system 20 can process the video of the front of the vehicle to make the processed video clearer, which helps the driver to see the road conditions clearly while driving, reduces the accident rate in the case of blurred vision, and improves driving safety.

[0046] Specifically, step 403 may include the following sub-steps:

[0047] Sub-step 4031: The vehicle-mounted unit 20 performs a perspective change on the video in front of the vehicle to obtain a video from the driver's perspective.

[0048] In some embodiments, the vehicle-mounted system 20 can convert the video captured by the camera 10 into a video from the driver's perspective through coordinate system transformation. For example, two coordinate systems can be constructed with the camera 10 and the driver's eye as origins respectively. For instance, coordinate system A can be constructed with the camera 10 as the origin, and coordinate system B can be constructed with the driver's eye (driver's perspective) as the origin. Then, the video captured by the camera 10 is divided into a three-dimensional mesh, for example, into N meshes, and these N meshes are mapped to N coordinates in coordinate system A. Next, the coordinates in coordinate system A can be converted to coordinates in coordinate system B, for example, coordinate A1 in coordinate system A can be converted to coordinate B1 in coordinate system B. By sequentially completing the coordinate transformation of the N coordinates in different coordinate systems in this way, the video captured by the camera 10 can be converted to the video in coordinate system B, thus completing the perspective transformation.

[0049] Sub-step 4032: The vehicle-mounted system 20 performs video cropping on the driver's perspective video so that the cropped video conforms to the size of the HUD's projection range.

[0050] For example, if the HUD can project a screen with a field of view of 4 degrees * 13 degrees, while the driver's view video screen has a field of view of 30 degrees * 50 degrees, then the driver's view video screen can be cropped to a screen size with a field of view of 4 degrees * 13 degrees.

[0051] Sub-step 4033: The vehicle-mounted system 20 performs image optimization processing on the cropped video to obtain the optimized video.

[0052] In some embodiments, the vehicle infotainment system 20 can perform image optimization processing on the cropped video, such as noise reduction, sharpening, contrast enhancement, brightness enhancement and other video enhancement technologies to optimize the video image and make the video image clearer.

[0053] Sub-step 4034: The vehicle system 20 extracts key elements from the optimized video and performs virtualization processing on the key elements.

[0054] Key elements include pedestrians, vehicles, and trees.

[0055] In some embodiments, the vehicle-mounted system 20 can extract key elements from the video and virtualize these key elements into icons (outlines) or models. For example, pedestrians can be virtualized into outlines representing people. Specifically, the vehicle-mounted system 20 can pre-store a media library, which may include various media, such as different styles of person icons (or outlines). The vehicle-mounted system 20 can use image recognition technology to identify key elements in the video, such as pedestrians, vehicles, trees, etc., and then select the icon corresponding to the key element from the media library and overlay the icon on top of the key element. For example, as shown... Figure 5 As shown, assuming the vehicle's infotainment system 20 identifies the target using image recognition technology. Figure 5 After viewing the figures on the sidewalk in the diagram, you can select the corresponding icon from the figure assets in the asset library, and then overlay the figure icon on top of the figures in the image to complete the layer overlay. After the layer overlay is complete, you can re-render the current frame.

[0056] It should be understood that the icons selected from the material library that correspond to the key elements only need to indicate the characteristics of the key elements. For example, if the key element is a vehicle, the icon only needs to indicate that the vehicle is located in the image. This application does not impose any specific limitations on this.

[0057] Sub-step 4035: Vehicle system 20 determines the projection position of key elements.

[0058] In some embodiments, the vehicle system 20 can estimate the speed of key elements based on the pixel movement of the video stream, and then perform latency compensation based on the vehicle's driving speed and the video processing duration T (i.e., overall latency). That is, it calculates the time difference T required for the key element to be acquired from the camera 10 and finally sent to the display device 30, thereby predicting the projection position of the key element based on the time difference T.

[0059] For example, the velocity of a key element can be predicted based on the pixel movement of the previous Z frames of the video stream. For instance, the velocity of a pedestrian can be predicted based on the pixel movement of the pedestrian in the video stream in the previous Z frames. Assuming the distance the pedestrian moves in the video stream in the previous Z frames is *s*, and the time the pedestrian moves in the video stream in the previous Z frames is *t1*, then the pedestrian's velocity can be estimated using (v = s / t1). This estimates the pedestrian's position during projection.

[0060] The following describes the calculation process of the coordinate position after time delay compensation, taking the X-axis as an example. For instance, the calculation of the X-axis in the coordinate position after time delay compensation is as follows:

[0061]

[0062] Among them, X m This indicates the X-axis coordinate position after time delay compensation; X n This represents the X-axis coordinate position of the current processing frame n before latency compensation, where T represents the overall latency (i.e., the time from when the image is captured by the camera, through viewpoint transformation and image processing, to when it is finally displayed); X n-1 This represents the X-axis coordinate position before delay compensation for processing frame (n-1); X n-z Z represents the X-axis coordinate position before delay compensation for the processing frame (nz); t represents the duration of each frame; and Z represents the number of frames for predicting the object's velocity.

[0063] The calculation methods for the Y-axis and Z-axis are similar to those for the X-axis. For example, the Y-axis calculation in the coordinate position after time delay compensation is as follows:

[0064]

[0065] Among them, Y m This indicates the Y-axis coordinate position after delay compensation; Y n This represents the Y-axis coordinate position of the current processing frame n before latency compensation, where T represents the overall latency (i.e., the time from when the image is captured by the camera, through viewpoint transformation and image processing, to when it is finally displayed); Y n-1 This represents the Y-axis coordinate position before delay compensation for processing frame n-1; Y n-z Z represents the Y-axis coordinate position before delay compensation for the processed frame nz; t represents the duration of each frame; and Z represents the number of frames for predicting the object's velocity.

[0066] For example, the Z-axis in the coordinate position after time delay compensation is calculated as follows:

[0067]

[0068] Among them, Z m This indicates the Z-axis coordinate position after delay compensation; Zn Z represents the Z-axis coordinate position of the current processing frame n before delay compensation, and T represents the overall delay (i.e., the time from when the image is acquired by the camera, through viewpoint transformation and image processing, to when it is finally displayed); Z n-1 This indicates the Z-axis coordinate position before delay compensation for processing frame n-1; Z n-z The value nz represents the Z-axis coordinate position before delay compensation for the processed frame; t represents the duration of each frame; and Z represents the number of frames for predicting the object's velocity.

[0069] Through the above calculations, the coordinate position (X) of the key element after time delay compensation can be obtained. m Y m Z m ).

[0070] Step 404: The vehicle-mounted system 20 sends the processed video to the display device 30.

[0071] Step 405: Project the processed video onto display device 30.

[0072] After the vehicle's infotainment system 20 finishes processing the video, it can send the processed video to the display device 30, which will then project the video onto the vehicle's windshield.

[0073] Through the above embodiments, the vehicle-mounted system 20 can process the video captured by the camera in front of the vehicle, making the processed video clearer. This allows the driver to see road conditions more clearly in situations with blurred vision, reducing the accident rate and improving driving safety.

[0074] It should be noted that all or part of the embodiments provided in this application can be freely and arbitrarily combined with each other. The combined technical solutions are also within the protection scope of this application.

[0075] The methods provided in the embodiments of this application above are described from the perspective of an electronic device as the executing entity. To implement the functions of the methods provided in the embodiments of this application above, the electronic device may include hardware structures and / or software modules, implementing the above functions in the form of hardware structures, software modules, or a combination of hardware structures and software modules. Whether a particular function is executed in the form of hardware structures, software modules, or a combination of hardware structures and software modules depends on the specific application and design constraints of the technical solution.

[0076] Based on the above embodiments, this application also provides an image processing apparatus 600, such as... Figure 6 As shown, the image processing device 600 may include an acquisition module 601, a processing module 602, and a display module 603.

[0077] The acquisition module 601 is used to acquire a first image, which is an image corresponding to the video stream of road conditions in front of the vehicle captured by the camera; the processing module 602 is used to process the first image acquired by the acquisition module 601 to obtain a second image; and the display module 603 is used to project the second image processed by the processing module 602 onto the windshield of the vehicle for display.

[0078] In one possible design, the processing module 602 is specifically used to process the first image in the following manner to obtain the second image:

[0079] The video perspective of the first image is adjusted to obtain an image from the driver's perspective; the display range of the image from the driver's perspective is cropped to the size that the head-up display (HUD) can project, and the cropped image is used as the third image; the third image is sharpened to obtain a fourth image; key elements are extracted from the fourth image and processed, and the processed image is used as the second image, wherein the key elements include pedestrians, vehicles, or trees.

[0080] In one possible design, the processing module 602 is specifically used to process the key element in the following manner:

[0081] The key element is virtualized, and the projection position of the key element is determined; the key element and the virtualized key element are displayed at the projection position.

[0082] In one possible design, the processing module 602 is specifically used to virtualize the key element in the following manner:

[0083] Obtain the icon or model corresponding to the key element from the pre-stored material library; overlay the icon or model corresponding to the key element onto the layer of the key element.

[0084] In one possible design, the key element includes the Nth frame image;

[0085] The processing module 602 is specifically used to determine the projection position of the Nth frame image included in the key element in the following manner:

[0086] Obtain the first position of the Nth frame image of the key element, the duration of the Nth frame image, and the image processing delay; wherein, the first position is the position before the Nth frame image is processed; based on the first position of the Nth frame image, the duration of the Nth frame image, and the image processing delay, determine the projection position of the Nth frame image of the key element.

[0087] like Figure 7As shown, some other embodiments of this application disclose an image processing apparatus. See also... Figure 7 As shown, the image processing device 700 includes: a transceiver 701; a processor 702; a memory 703; and one or more computer programs 704 (not shown in the figure). The above devices can be connected through one or more communication buses 705.

[0088] The memory 703 stores one or more computer programs. When the instructions are invoked and executed by the processor 702, the image processing device 700 performs the method steps described in the above embodiments. For example, the transceiver 701 is used to receive an image corresponding to a video stream of road conditions ahead of the vehicle captured by a camera; the processor 702 is used to acquire a first image, which is the image corresponding to the video stream of road conditions ahead of the vehicle captured by the camera, and to process the acquired first image to obtain a second image; the processor 702 is also used to project the second image onto the windshield of the vehicle for display.

[0089] In this embodiment, the processor 702 can be a general-purpose processor, digital signal processor, application-specific integrated circuit, field-programmable gate array (FPGA), or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component, capable of implementing or executing the methods, steps, and logic block diagrams disclosed in this embodiment. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in this embodiment can be directly implemented by the hardware processor, or implemented by a combination of hardware and software modules within the processor. The software modules can be located in the memory 703. The processor 702 reads the program instructions from the memory 703 and, in conjunction with its hardware, completes the steps of the aforementioned methods.

[0090] In this embodiment, the memory 703 can be a non-volatile memory, such as a hard disk drive (HDD) or a solid-state drive (SSD), or it can be volatile memory, such as RAM. The memory can also be any other medium capable of carrying or storing desired program code in the form of instructions or data structures, and accessible by a computer, but is not limited thereto. The memory in this embodiment can also be a circuit or any other device capable of implementing storage functions, used to store instructions and / or data.

[0091] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the above-described apparatus and unit can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0092] Based on the above embodiments, this application also provides a computer storage medium storing a computer program, which, when executed by a computer, causes the computer to perform the method provided in the above embodiments.

[0093] This application also provides a computer program product, including instructions that, when run on a computer, cause the computer to execute the methods provided in the above embodiments.

[0094] This application describes embodiments of methods, apparatus (systems), and computer program products according to embodiments of this application with reference to flowchart illustrations and / or block diagrams. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by instructions. These instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0095] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0096] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

Claims

1. An image processing method, characterized by, include: Acquire a first image, which is an image corresponding to the video stream of road conditions in front of the vehicle captured by the camera; The first image is processed to obtain the second image; The second image is projected onto the windshield of the vehicle for display.

2. The method of claim 1, wherein, The process of processing the first image to obtain the second image includes: The video perspective of the first image is adjusted to obtain an image from the driver's perspective; The image display area from the driver's perspective is cropped to the size that the head-up digital display (HUD) can project, and the cropped image is used as a third image. The third image is optimized to obtain the fourth image; Key elements are extracted from the fourth image and processed. The processed image is used as the second image. The key elements include pedestrians, vehicles, or trees.

3. The method of claim 2, wherein, The processing of the key elements includes: The key element is virtualized, and the projection position of the key element is determined. The key element and the virtualized key element are displayed at the projection position.

4. The method as described in claim 3, characterized in that, Virtualization of the key elements includes: Retrieve the icon or model corresponding to the key element from the pre-stored material library; Display the icon or model corresponding to the key element over the layer of the key element.

5. The method as described in claim 3, characterized in that, The key element includes the Nth frame image; Determining the projection position of the Nth frame image, which includes the key element, includes: Obtain the first position of the Nth frame image of the key element, the duration of the Nth frame image, and the image processing delay; wherein, the first position is the position of the Nth frame image before processing; The projection position of the Nth frame image of the key element is determined based on the first position of the Nth frame image, the duration of the Nth frame image, and the image processing delay.

6. An image processing apparatus, characterized in that, include: The acquisition module is used to acquire a first image, which is an image corresponding to the video stream of road conditions in front of the vehicle captured by the camera; The processing module is used to process the first image acquired by the acquisition module to obtain a second image; The display module is used to project the second image processed by the processing module onto the windshield of the vehicle for display.

7. The apparatus as claimed in claim 6, characterized in that, The processing module is specifically used to process the first image in the following manner to obtain the second image: The video perspective of the first image is adjusted to obtain an image from the driver's perspective; The image display area from the driver's perspective is cropped to the size that the head-up digital display (HUD) can project, and the cropped image is used as a third image. The third image is optimized to obtain the fourth image; Key elements are extracted from the fourth image and processed. The processed image is used as the second image. The key elements include pedestrians, vehicles, or trees.

8. The apparatus as claimed in claim 7, characterized in that, The processing module is specifically used to process the key elements in the following manner: The key element is virtualized, and the projection position of the key element is determined. The key element and the virtualized key element are displayed at the projection position.

9. The apparatus as claimed in claim 7, characterized in that, The processing module is specifically used to virtualize the key elements in the following manner: Retrieve the icon or model corresponding to the key element from the pre-stored material library; Display the icon or model corresponding to the key element over the layer of the key element.

10. The apparatus as claimed in claim 7, characterized in that, The key element includes the Nth frame image; The processing module is specifically used to determine the projection position of the Nth frame image included in the key element in the following manner: Obtain the first position of the Nth frame image of the key element, the duration of the Nth frame image, and the image processing delay; wherein, the first position is the position of the Nth frame image before processing; The projection position of the Nth frame image of the key element is determined based on the first position of the Nth frame image, the duration of the Nth frame image, and the image processing delay.

11. An image processing device, characterized in that, The image processing device includes a transceiver, a processor, a memory, and one or more computer programs. The one or more computer programs are stored in the one or more memories, and the one or more computer programs include instructions that, when invoked and executed by the one or more processors, cause the image processing device to perform the method as described in any one of claims 1 to 5.

12. A computer-readable storage medium storing instructions, characterized in that, When the instructions are executed on an image processing device, the image processing device causes the image processing device to perform the method as described in any one of claims 1 to 5.

Images