Image processing method and device, equipment and storage medium

Abstract

The invention discloses an image processing method and device, equipment and a storage medium. The embodiment of the invention can be applied to various scenes such as a cloud technology. Specifically, the method comprises the following steps: switching current static metadata into updated static metadata corresponding to the currently acquired continuous image frames under the condition that scene switching in the currently acquired continuous image frames is detected; on the basis of the updated static metadata, performing image coding on continuous image frames acquired from the acquisition moment of the updated static metadata to obtain a coded code stream; and in the process of transmitting the encoding code stream to the terminal in real time, sending a metadata switching notification carrying the updated static metadata to the terminal, so that the terminal switches the current static metadata into the updated static metadata, and performs image decoding on the encoding code stream based on the updated static metadata to obtain a target display image frame. By means of the technical scheme, the picture presentation effect of the terminal can be improved under the condition that the terminal does not have the dynamic metadata analysis capacity.

Description

Technical Field

[0001] The present application relates to the field of cloud technology, and in particular to an image processing method, apparatus, device and storage medium. Background Art

[0002] Current high dynamic range (HDR) video resources include two technical formats: static HDR and dynamic HDR. Static HDR uses the same metadata (i.e. static metadata) in a video to control the color and details of each frame, while dynamic HDR allows different metadata (i.e. dynamic metadata) to be specified for each frame or scene in a video.

[0003] However, the dynamic HDR standard has not yet been popularized, and some terminal devices do not support the parsing of dynamic metadata, resulting in poor compatibility. In video display scenarios such as cloud gaming and cloud VR that pursue high-quality picture performance and low processing delay, the existing static HDR technology cannot provide good picture performance while providing low latency, resulting in a poor user experience. Summary of the invention

[0004] The present application provides an image processing method, apparatus, device and storage medium. When a scene switch in a continuous image frame is detected, the dynamic switching of static metadata is realized in the terminal through the means of terminal-cloud collaboration, so that the target display image frame decoded by the terminal can have different dynamic ranges and picture performances in different scenes. Therefore, in a scene where the terminal decoder does not have the ability to parse dynamic metadata, it is also possible to achieve a picture performance effect similar to dynamic metadata while ensuring low picture delay, thereby improving user experience. The technical solution of the present application is as follows:

[0005] In one aspect, an image processing method is provided, the method comprising:

[0006] In the process of encoding the image based on the current static metadata, sequentially acquiring consecutive image frames;

[0007] In case of detecting a scene switch in the currently acquired continuous image frames, switching the current static metadata to updated static metadata corresponding to the currently acquired continuous image frames;

[0008] Based on the updated static metadata, image encoding is performed on the continuous image frames acquired from a target time to obtain an encoded bitstream, wherein the target time is the acquisition time of the updated static metadata;

[0009] In the process of transmitting the encoded code stream to the terminal in real time, a metadata switching notification carrying the updated static metadata is sent to the terminal, so that the terminal switches the current static metadata to the updated static metadata, and based on the updated static metadata, performs image decoding on the encoded code stream to obtain a target display image frame.

[0010] In another aspect, an image processing method is provided, the method comprising:

[0011] In response to a metadata switching notification carrying updated static metadata sent by the cloud during real-time transmission of the encoded bitstream, switching the current static metadata to the updated static metadata;

[0012] Based on the updated static metadata, image decoding is performed on the encoded code stream to obtain a target display image frame;

[0013] Among them, the updated static metadata is the static image metadata corresponding to the currently acquired continuous image frames obtained when the cloud detects a scene switch in the currently acquired continuous image frames; the encoded code stream is the code stream obtained after the cloud encodes the target image frames based on the updated static metadata; the target image frames are the continuous image frames acquired by the cloud from the moment the updated static metadata is acquired; and the continuous image frames are video image frames generated sequentially in real time.

[0014] In another aspect, an image processing device is provided, the device comprising:

[0015] An image frame acquisition module, used to sequentially acquire continuous image frames during image encoding based on current static metadata;

[0016] A first metadata switching module is used to switch the current static metadata to updated static metadata corresponding to the currently acquired continuous image frames when a scene switch in the currently acquired continuous image frames is detected;

[0017] An image encoding module, configured to perform image encoding on the continuous image frames acquired from a target time based on the updated static metadata to obtain an encoded bitstream, wherein the target time is the acquisition time of the updated static metadata;

[0018] The metadata switching notification sending module is used to send a metadata switching notification carrying the updated static metadata to the terminal during the process of transmitting the encoded code stream to the terminal in real time, so that the terminal switches the current static metadata to the updated static metadata, and based on the updated static metadata, performs image decoding on the encoded code stream to obtain a target display image frame.

[0019] In another aspect, an image processing device is provided, the device comprising:

[0020] A second metadata switching module is configured to respond to a metadata switching notification carrying updated static metadata sent by the cloud during real-time transmission of the encoded bitstream, and to switch the current static metadata to the updated static metadata;

[0021] A first image decoding module, configured to perform image decoding on the coded bitstream based on the updated static metadata to obtain a target display image frame;

[0022] Among them, the updated static metadata is the static image metadata corresponding to the currently acquired continuous image frames obtained when the cloud detects a scene switch in the currently acquired continuous image frames; the encoded code stream is the code stream obtained after the cloud encodes the target image frames based on the updated static metadata; the target image frames are the continuous image frames acquired by the cloud from the moment of acquiring the updated static metadata.

[0023] On the other hand, an image processing device is provided, which includes a processor and a memory, wherein the memory stores at least one instruction or at least one program, and the at least one instruction or the at least one program is loaded and executed by the processor to implement the image processing method as described above.

[0024] On the other hand, a computer-readable storage medium is provided, wherein at least one instruction or at least one program is stored in the storage medium, and the at least one instruction or the at least one program is loaded and executed by a processor to implement the image processing method as described above.

[0025] On the other hand, a computer program product or a computer program is provided, the computer program product or the computer program comprising computer instructions, the computer instructions being stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the image processing method as described above.

[0026] The present application provides an image processing method, apparatus, device and storage medium, which have the following technical effects:

[0027] By using the technical solution provided by the present application, when the cloud encoder detects a scene switch in the currently acquired continuous image frames, the updated static metadata corresponding to the currently acquired continuous image frames is used as static metadata that is more suitable for the scene after the switch, and the current static metadata is switched to the updated static metadata, so that the image frames after the scene switch are encoded based on the updated static metadata, and then through the end-cloud collaboration, the terminal decoder is notified to perform the corresponding static metadata switch, thereby improving the timeliness of the dynamic switching of metadata, so that the target display image frame after the terminal decodes can have different dynamic ranges and picture performances in different scenes, so that in the scene where the terminal decoder does not have the ability to parse dynamic metadata, it can also achieve a picture performance effect similar to the dynamic metadata while ensuring low picture latency, thereby improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present application or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0029] Figure 1 is a schematic diagram of an application environment provided by an embodiment of the present application;

[0030] Figure 2 It is a flowchart of an image processing method provided in an embodiment of the present application;

[0031] Figure 3 It is a schematic diagram of a process of switching current static metadata to updated static metadata corresponding to the currently acquired continuous image frames provided by an embodiment of the present application;

[0032] Figure 4a -b is a schematic diagram of a process of obtaining image metadata corresponding to a target image frame in the cloud provided in an embodiment of the present application;

[0033] Figure 5 is a flowchart of another image processing method provided in an embodiment of the present application;

[0034] Figure 6 A flowchart of another image processing method provided in an embodiment of the present application;

[0035] Figure 7 is a schematic diagram of an image processing architecture provided in an embodiment of the present application;

[0036] Figure 8 is a block diagram of an image processing device provided by an embodiment of the present application;

[0037] Fig. 9 is a block diagram of another image processing device provided in an embodiment of the present application;

[0038] Fig.10 It is a structural schematic diagram of an image processing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0039] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0040] It should be noted that the terms "including" and "having" and any variations thereof in the specification and claims of this application and the above-mentioned drawings are intended to cover non-exclusive inclusions. For example, a process, method, system, product or server that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0041] It is understandable that in the specific implementation of this application, related data such as user information is involved. When the following embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions.

[0042] To facilitate understanding of the embodiments of the present application, several concepts are briefly introduced below:

[0043] 1. Cloud technology refers to a hosting technology that unifies hardware, software, network and other resources in a wide area network or local area network to realize data computing, storage, processing and sharing. Cloud technology is a general term for network technology, information technology, integration technology, management platform technology, application technology, etc. based on the cloud computing business model. It can form a resource pool, which is used on demand and flexible and convenient. Cloud computing technology will become an important support. The background services of technical network systems require a large amount of computing and storage resources, such as video websites, picture websites and more portal websites. With the high development and application of the Internet industry, each item may have its own identification mark in the future, and it needs to be transmitted to the background system for logical processing. Data of different levels will be processed separately. All kinds of industry data need strong system backing support, which can only be achieved through cloud computing.

[0044] 2. Cloud gaming, also known as gaming on demand, is an online gaming technology based on cloud computing technology. Cloud gaming technology enables thin clients with relatively limited graphics processing and data computing capabilities to run high-quality games. In a cloud gaming scenario, the game is not played on the player's game terminal, but on a cloud server, which renders the game scene into a video and audio stream and transmits it to the player's game terminal over the network. The player's game terminal does not need to have powerful graphics computing and data processing capabilities, but only needs to have basic streaming media playback capabilities and the ability to obtain player input commands and send them to the cloud server.

[0045] 3. HDR (High-Dynamic Range), as one of the key technologies in the ultra-high-definition audio and video industry, supports a maximum brightness higher than 1000nit and a minimum brightness lower than 0.01nit compared to the traditional standard range, and supports a wider color space (such as Rec.2020 color gamut), etc., which can express more details and colors in the image.

[0046] 4. HDR10 is an open standard announced by the Consumer Technology Association on August 27, 2015. This standard requires the use of wide color gamut Rec.2020 color space, 10-bit color depth, and SMPTE ST 2084 (PQ) image transmission function, but the data elements used by HDR10 are all static metadata. In simple terms, the HDR image effect output by each frame is the same.

[0047] 5. HDR10+, also known as HDR10 Plus, updates HDR10 by adding dynamic metadata. Dynamic metadata can be used to more accurately adjust the brightness level of HDR on a scene-by-scene or frame-by-frame basis, similar to Dolby Vision HDR, but the color depth is still 10bit.

[0048] 6. Static metadata: The entire video uses a single metadata to control the brightness changes of each frame, and will not be adjusted due to differences in scenes in different frames.

[0049] 7. Dynamic metadata: allows you to set different metadata for each frame or scene in a video to maximize the brightness performance of the HDR picture, try to ensure that the picture dynamics of each scene can be fully released, while retaining rich dark and highlight detail levels.

[0050] 8. Image encoding is a method of converting files in the original image format into files in another image format through compression technology. The converted data can be called a code stream.

[0051] 9. Image decoding is the reverse process of image encoding.

[0052] 10. End-cloud collaboration: A method in which the server (cloud) and the user (terminal) work together to improve video quality / smoothness through negotiation, such as collaboratively formulating or adjusting strategies for video encoding / video processing / network transmission. End-cloud collaboration can be used in cloud gaming, cloud rendering, real-time communication and other scenarios.

[0053] In the existing technology, the standard of dynamic HDR has not yet been popularized, and some terminal devices do not support the parsing of dynamic metadata, and have poor compatibility. In video display scenarios such as cloud gaming and cloud VR that pursue high-quality picture performance and low processing delay, the existing static HDR technology cannot provide good picture performance while providing low latency, resulting in a poor user experience.

[0054] In order to solve this technical problem, when the cloud encoder detects a scene switch in the currently acquired continuous image frames, the updated static metadata corresponding to the currently acquired continuous image frames is used as static metadata that is more suitable for the scene after the switch, and the current static metadata is switched to the updated static metadata, so as to encode the image frames after the scene switch based on the updated static metadata, and then through the end-cloud collaboration, the terminal decoder is notified to perform the corresponding static metadata switching, thereby improving the timeliness of the dynamic switching of metadata, so that the target display image frame after the terminal decodes can have different dynamic ranges and picture performances in different scenes. Therefore, in the scene where the terminal decoder does not have the ability to parse dynamic metadata, it can also achieve a picture performance effect similar to the dynamic metadata while ensuring low picture latency, thereby improving the user experience.

[0055] See also Figure 1 , Figure 1is a schematic diagram of an application environment provided by an embodiment of the present application, and the application environment may include a client 10 and a server 20, and the client 10 and the server 20 may be directly or indirectly connected via wired or wireless communication. A user may trigger an image processing request based on the client 10, and the server 20 responds to the image processing request, and in the process of image encoding based on the current static metadata, sequentially acquires continuous image frames, and in the case of detecting a scene switch in the currently acquired continuous image frames, switches the current static metadata to the updated static metadata corresponding to the currently acquired continuous image frames, and based on the updated static metadata, performs image encoding on the continuous image frames acquired from the target moment to obtain an encoded code stream, and the target moment is the acquisition moment of the updated static metadata, and then in the process of transmitting the encoded code stream to the client 10 in real time, sends a metadata switching notification carrying the updated static metadata to the client 10, and the client 10 responds to the metadata switching notification, switches the current static metadata to the updated static metadata, and decodes the encoded code stream based on the updated static metadata to obtain the target display image frame. It should be noted that Figure 1 Just an example.

[0056] The client can be a physical device such as a smart phone, a computer (such as a desktop computer, a tablet computer, a laptop computer), a digital assistant, an intelligent voice interaction device (such as an intelligent speaker), an intelligent wearable device, a vehicle terminal, an aircraft, etc., or it can be software running in a physical device, such as a computer program. The operating system corresponding to the first client can be an Android system (Android system), an iOS system (a mobile operating system developed by Apple), a Linux system (an operating system), a Microsoft Windows system (Microsoft Windows operating system), etc.

[0057] The server side can be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms. The server can include a network communication unit, a processor, and a memory, etc. The server side can provide background services for the corresponding client.

[0058] The client 10 and the server 20 can be used to build a system for image processing, which can be a distributed system. Taking the distributed system as a blockchain system as an example, it is formed by multiple nodes (any form of computing devices in the access network, such as servers, user terminals) and clients, and a peer-to-peer network is formed between the nodes. The peer-to-peer protocol is an application layer protocol running on the Transmission Control Protocol (TCP). In a distributed system, any machine such as a server or a terminal can join and become a node, and the node includes a hardware layer, an intermediate layer, an operating system layer, and an application layer.

[0059] The functions of each node in the above blockchain system include:

[0060] 1) Routing: a basic function of a node, used to support communication between nodes.

[0061] In addition to the routing function, the node can also have the following functions:

[0062] 2) Applications are deployed in the blockchain to implement specific businesses based on actual business needs, record data related to the implementation of functions to form record data, carry digital signatures in the record data to indicate the source of the task data, and send the record data to other nodes in the blockchain system for other nodes to add the record data to a temporary block when they successfully verify the source and integrity of the record data.

[0063] 3) Blockchain, including a series of blocks that are connected to each other in the order of their generation. Once a new block is added to the blockchain, it will not be removed. The block records the record data submitted by the nodes in the blockchain system.

[0064] The following describes a specific embodiment of an image processing method provided by the present application. Figure 2 It is a flowchart of an image processing method provided by an embodiment of the present application. The present application provides method operation steps as described in the embodiment or flowchart, but may include more or fewer operation steps based on conventional or non-creative labor. The order of steps listed in the embodiment is only one way of executing the steps among many orders, and does not represent the only order of execution. When the actual system or product is executed, it can be executed in the order of the method shown in the embodiment or the drawings or in parallel (for example, in a parallel processor or multi-threaded processing environment). Specifically, as Figure 2 As shown, the method may include:

[0065] S201, the cloud obtains continuous image frames in sequence during the process of encoding the image based on the current static metadata.

[0066] In the embodiment of the present specification, the continuous image frames may be multiple video image frames generated sequentially in real time.

[0067] In a specific embodiment, the continuous image frames may be image frames generated based on the HDR (High Dynamic Range) technical standard, and accordingly, each image frame in the continuous image frames is an image in HDR format. Specifically, the HDR technical standard here may include: HDR10, HDR10+, etc.

[0068] In a specific embodiment, the cloud may include: an image generation end and an encoding end. The image generation end may send the currently generated image frame to the encoding end while sequentially generating continuous image frames in real time.

[0069] In a specific embodiment, the image generation end may obtain image metadata corresponding to each image frame in the continuous image frames after generating the image frame. Specifically, the image metadata here may be static metadata or dynamic metadata.

[0070] In the embodiment of the present specification, the current static metadata may be static metadata currently being used for image encoding in the cloud (encoding end).

[0071] In an optional embodiment, the current static metadata may be reference static metadata pre-set in combination with the current business scenario, or may be static metadata determined based on image metadata corresponding to currently acquired continuous image frames, for example, static metadata determined based on image metadata corresponding to the first image frame in currently acquired continuous image frames.

[0072] S202, when the cloud detects a scene switch in the currently acquired continuous image frames, the cloud switches the current static metadata to updated static metadata corresponding to the currently acquired continuous image frames.

[0073] In the embodiment of the present specification, the updated static metadata corresponding to the currently acquired continuous image frames may be static metadata that is more suitable for the dynamic range of the scene after the switching than the current static metadata. Specifically, the dynamic range (DynamicRange) is the ratio of the maximum value to the minimum value of a variable signal (such as sound or light).

[0074] In a specific embodiment, Figure 3 As shown, the above-mentioned continuous image frames may be video image frames generated sequentially in real time, and the above-mentioned switching of the current static metadata to the updated static metadata corresponding to the currently acquired continuous image frames may include:

[0075] S301, when a scene switch is detected in the currently acquired continuous image frames, the cloud obtains image metadata corresponding to a target image frame, where the target image frame is the latest image frame generated when the scene switch is detected.

[0076] In a specific embodiment, the image metadata corresponding to the target image frame may be the image metadata obtained by the image generation end when generating the target image frame. Specifically, the data type of the image metadata here may be static metadata or dynamic metadata.

[0077] Illustratively, the image metadata here may include, but is not limited to: color gamut information and brightness information. Among them, the color gamut information may include color management standards such as DCI-P3 or BT.2020. For example, the color gamut information may include: the x and y coordinates of the three primary colors and the white point. Among them, the brightness information may include the maximum content brightness level (Maximum Content LightLevel, MaxCLL), MaxCLL represents the brightness value of the brightest pixel in the HDR content. The brightness information may also include the maximum average frame brightness level (Maximum Frame-Average Light Level, MaxFALL), and the average frame brightness represents the average brightness of all pixels in an image frame. MaxFALL represents the maximum average frame brightness of all image frames in the HDR video content. It will be understood that the brightness information may also include other information related to brightness.

[0078] Optionally, the scene switching detection here can be performed by the encoding end in the cloud actively detecting the scene switching during the image encoding process, and then requesting the image generating end for the image metadata corresponding to the target image frame; or the image generating end can send a scene switching notification carrying the image metadata for a certain image frame to the encoding end at the same time as sending the image frame to the encoding end, and the encoding end realizes passive detection of the scene switching by receiving the scene switching notification.

[0079] In an optional embodiment, if Figure 4a As shown, the above-mentioned cloud obtains the image metadata corresponding to the target image frame, which may include:

[0080] S3011, the encoding end receives a scene switching notification for the currently acquired continuous image frames sent by the image generating end, where the scene switching notification carries image metadata.

[0081] Specifically, when an image frame is a scene switching frame, the image generation end may send a scene switching notification carrying image metadata at the same time as sending the image frame to the encoding end, wherein the scene switching frame is an image frame in which a scene switching occurs relative to its previous image frame.

[0082] It can be seen from the above embodiments that the encoder implements passive detection of scene switching by receiving scene switching notification, which can improve image processing efficiency and reduce the delay caused by active detection.

[0083] In an optional embodiment, if Figure 4b As shown, the above-mentioned cloud obtains the image metadata corresponding to the target image frame, which may include:

[0084] S3012, the encoding end performs scene switching detection on the current encoded image frame in the process of sequentially encoding the currently acquired continuous image frames based on the current static metadata to obtain a detection result.

[0085] In a specific embodiment, the above-mentioned scene switching detection on the current coded image frame to obtain the detection result may include:

[0086] 1) Determine the prediction block ratio information corresponding to the current coded image frame.

[0087] Specifically, the prediction block ratio information may represent the ratio of the prediction block in the multiple coding blocks corresponding to the current coded image frame.

[0088] 2) When the predicted block ratio information satisfies a preset ratio condition, the current encoded image frame is determined to be a scene switching frame.

[0089] Specifically, the preset proportion condition here may be that the predicted block proportion is greater than a preset proportion threshold. The preset proportion threshold may be preset in combination with the accuracy requirement of scene switching detection in actual applications. Schematically, the preset proportion threshold may be 50%.

[0090] In practical applications, when encoding a certain position of the current encoded image frame, if no similar block (reference block) is found near the same position of the previous image frame, the current encoded image frame can only be compressed with its own information to obtain the image block corresponding to the position. The image block is then used as a prediction block. If the prediction block in the current encoded image frame is larger, it means that the current encoded image frame has fewer similar blocks with the previous frame, and the image picture of the current encoded image frame is very different from that of the previous frame. The current encoded image frame is considered to be a scene switching frame.

[0091] It can be seen from the above embodiments that by judging whether the current coded image frame is a scene switching frame based on the prediction block ratio information, the accuracy of scene switching detection is improved.

[0092] S3013: When the detection result indicates that the current encoded image frame is a scene switching frame, the encoding end sends a metadata update request to the image generating end.

[0093] Specifically, the scene switching frame may be an image frame in which a scene switching occurs relative to a previous image frame.

[0094] In an optional embodiment, since the current encoded image frame is encoded based on the current static metadata, when the current encoded image frame is a scene switching frame, the adaptability of the current static metadata encoding to the scene in the current encoded image frame (i.e., the scene after switching) is low. In order to ensure the picture performance of the image displayed by the subsequent terminal, the encoding end can discard the current encoded image frame while sending a metadata update request to the image generation end.

[0095] S3014, the encoding end receives the image metadata sent by the image generating end in response to the metadata update request.

[0096] It can be seen from the above embodiments that, when the encoder does not receive a scene switching notification, the encoder can improve the accuracy of scene switching detection and reduce misjudgment of the encoder by actively detecting.

[0097] S302, the cloud switches the current static metadata to updated static metadata corresponding to the image metadata.

[0098] In a specific embodiment, the data type of the image metadata can be static metadata or dynamic metadata. When the image metadata is static metadata, the image metadata parameters are directly used as parameters for updating the static metadata; when the image metadata is dynamic metadata, the image metadata is parsed into static metadata and then used as parameters for updating the static metadata.

[0099] Specifically, the cloud reconfigures encoder parameters based on updating static metadata.

[0100] It can be seen from the above embodiments that when a scene switch is detected in the currently acquired continuous image frames, the cloud obtains the image metadata corresponding to the latest generated image frame when the scene switch is detected, and switches the current static metadata to the updated static metadata corresponding to the image metadata, which can improve the matching of the updated static metadata with the scene after the switch.

[0101] In an optional embodiment, the cloud switches the current static metadata to updated static metadata corresponding to the currently acquired continuous image frames, including:

[0102] When the interval between the current moment and the last metadata switching moment satisfies the preset interval, the step of switching the current static metadata to the updated static metadata corresponding to the currently acquired continuous image frames is performed.

[0103] Specifically, the preset interval here may be a time interval or a frame interval, and the specific value of the preset interval may be preset in combination with the image processing delay requirement in actual applications.

[0104] It can be seen from the above embodiments that static metadata switching is performed only when the preset interval is met, which can reduce the image processing delay caused by the switching and further reduce the image display delay.

[0105] S203, based on the updated static metadata, the cloud encodes the continuous image frames acquired from the target time to obtain an encoded bit stream, and the target time is the acquisition time of the updated static metadata.

[0106] Specifically, since the updated static metadata is the static metadata suitable for the scene acquired after the scene switch is detected, the continuous image frames acquired from the moment the updated static metadata is acquired can be considered as image frames showing the scene after the switch. Therefore, based on the updated static metadata, image encoding is performed on the continuous image frames acquired from the target moment.

[0107] S204, in the process of transmitting the encoded code stream to the terminal in real time, the cloud sends a metadata switching notification carrying updated static metadata to the terminal.

[0108] Specifically, in the process of transmitting the encoded code stream to the terminal in real time, the cloud sends a metadata switching notification carrying updated static metadata to the terminal based on the end-cloud collaboration.

[0109] S205: The terminal switches the current static metadata to updated static metadata.

[0110] Specifically, when the terminal performs static metadata switching, it is necessary to restart the decoder and configure the static metadata in the restarted decoder as updated static metadata.

[0111] In an optional embodiment, the terminal switching the current static metadata to updated static metadata may include:

[0112] When the interval between the current time and the last metadata switching time satisfies the preset interval, the decoder is restarted, and the static metadata in the restarted decoder is configured to update the static metadata.

[0113] Specifically, the preset interval here may be a time interval or a frame interval, and the specific value of the preset interval may be preset in combination with the image processing delay requirement in actual applications.

[0114] It can be seen from the above embodiments that since the terminal needs to restart the decoder when switching static metadata, by setting a preset interval and reducing the number of switches, the image processing delay caused by the switch can be reduced, thereby reducing the image display delay.

[0115] S206, based on the updated static metadata, image decoding is performed on the encoded code stream to obtain a target display image frame.

[0116] It can be seen from the above embodiments that when the cloud encoder detects a scene switch in the currently acquired continuous image frames, the updated static metadata corresponding to the currently acquired continuous image frames is used as static metadata that is more suitable for the scene after the switch, and the current static metadata is switched to the updated static metadata, so that the image frames after the scene switch are encoded based on the updated static metadata, and then through the end-cloud collaboration, the terminal decoder is notified to perform the corresponding static metadata switch, thereby improving the timeliness of the dynamic switching of metadata, so that the target display image frame after the terminal decodes can have different dynamic ranges and picture performances in different scenes, so that in the scene where the terminal decoder does not have the ability to parse dynamic metadata, it can also achieve a picture performance effect similar to the dynamic metadata while ensuring low picture latency, thereby improving the user experience.

[0117] In an optional embodiment, the above method may further include:

[0118] When the interval between the current moment and the last metadata switching moment does not satisfy the preset interval, the terminal decodes the coded bitstream based on the current static metadata to obtain a target display image frame.

[0119] It can be seen from the above embodiments that since the terminal needs to restart the decoder when switching static metadata, by setting a preset interval, not switching when the preset interval is not met, the picture processing delay caused by the switching can be reduced, thereby reducing the picture display delay.

[0120] The following introduces another image processing method provided by the embodiment of the present application with the cloud as the execution subject. Figure 5 A flowchart of another image processing method provided for an embodiment of the present application. It should be noted that this specification provides method operation steps as described in the embodiments or flowcharts, but more or fewer operation steps may be included based on conventional or non-creative labor. The order of steps listed in the embodiments is only one way of executing the steps among many orders, and does not represent the only order of execution. When the actual system or product is executed, it can be executed in sequence or in parallel (for example, in a parallel processor or multi-threaded processing environment) according to the method shown in the embodiments or drawings. Specific examples include: Figure 5 As shown, the above method may include:

[0121] S501, in a process of performing image encoding based on current static metadata, sequentially acquiring continuous image frames;

[0122] S502, when a scene switch in the currently acquired continuous image frames is detected, the current static metadata is switched to updated static metadata corresponding to the currently acquired continuous image frames;

[0123] S503, based on the updated static metadata, image encoding is performed on the continuous image frames acquired from the target time to obtain an encoded bitstream, where the target time is the acquisition time of the updated static metadata;

[0124] S504, in the process of transmitting the encoded code stream to the terminal in real time, sending a metadata switching notification carrying updated static metadata to the terminal, so that the terminal switches the current static metadata to the updated static metadata, and based on the updated static metadata, performs image decoding on the encoded code stream to obtain a target display image frame.

[0125] In a specific embodiment, the continuous image frames may be video image frames generated sequentially in real time, and the step of switching the current static metadata to the updated static metadata corresponding to the currently acquired continuous image frames may include:

[0126] S5021, when a scene switch is detected in the currently acquired continuous image frames, acquiring image metadata corresponding to a target image frame, where the target image frame is the latest image frame generated when the scene switch is detected;

[0127] S5022: Switch the current static metadata to updated static metadata corresponding to the image metadata.

[0128] In an optional embodiment, the acquiring of the image metadata corresponding to the target image frame may include:

[0129] A scene switching notification for the currently acquired continuous image frames sent by the image generation end is received, where the scene switching notification carries image metadata.

[0130] In an optional embodiment, the acquiring of the image metadata corresponding to the target image frame may include:

[0131] 1) In the process of sequentially encoding the currently acquired continuous image frames based on the current static metadata, performing scene switching detection on the currently encoded image frames to obtain detection results;

[0132] 2) When the detection result indicates that the current coded image frame is a scene switching frame, a metadata update request is sent to the image generation end;

[0133] 3) Receive image metadata sent by the image generation end in response to the metadata update request.

[0134] In a specific embodiment, the above-mentioned scene switching detection on the current coded image frame to obtain the detection result may include:

[0135] 1) Determine the prediction block ratio information corresponding to the current coded image frame;

[0136] 2) When the predicted block ratio information satisfies a preset ratio condition, the current encoded image frame is determined to be a scene switching frame.

[0137] In an optional embodiment, switching the current static metadata to updated static metadata corresponding to the currently acquired continuous image frames may include:

[0138] When the interval between the current moment and the last metadata switching moment satisfies the preset interval, the step of switching the current static metadata to the updated static metadata corresponding to the currently acquired continuous image frames is performed.

[0139] The specific detailed steps of the embodiment of the above-mentioned image processing method written from the cloud side can be referred to the embodiment of the above-mentioned interactive side image processing method, which will not be repeated here.

[0140] Another image processing method provided by an embodiment of the present application is introduced below with a terminal as the execution subject. Figure 6 A flowchart of another image processing method provided for an embodiment of the present application. It should be noted that this specification provides method operation steps as described in the embodiments or flowcharts, but more or fewer operation steps may be included based on conventional or non-creative labor. The order of steps listed in the embodiments is only one way of executing the steps among many orders, and does not represent the only order of execution. When the actual system or product is executed, it can be executed in sequence or in parallel (for example, in a parallel processor or multi-threaded processing environment) according to the method shown in the embodiments or drawings. Specific examples include: Figure 6 As shown, the above method may include:

[0141] S601, in response to a metadata switching notification carrying updated static metadata sent by the cloud during real-time transmission of an encoded bitstream, switching current static metadata to updated static metadata;

[0142] S602, based on the updated static metadata, decoding the coded bitstream to obtain a target display image frame;

[0143] Among them, updating static metadata is the static image metadata corresponding to the currently acquired continuous image frames obtained when the cloud detects scene switching in the currently acquired continuous image frames; the encoded code stream is the code stream obtained after the cloud encodes the target image frames based on the updated static metadata; the target image frames are the continuous image frames acquired by the cloud from the moment of acquiring the updated static metadata; the continuous image frames are video image frames generated sequentially in real time.

[0144] In an optional embodiment, switching the current static metadata to updated static metadata may include:

[0145] When the interval between the current time and the last metadata switching time satisfies the preset interval, the decoder is restarted, and the static metadata in the restarted decoder is configured to update the static metadata.

[0146] In an optional embodiment, the above method may further include:

[0147] When the interval between the current moment and the last metadata switching moment does not satisfy the preset interval, the coded bitstream is image decoded based on the current static metadata to obtain a target display image frame.

[0148] The specific detailed steps of the embodiment of the above-mentioned image processing method written from the terminal side can be referred to the embodiment of the above-mentioned interactive side image processing method, which will not be repeated here.

[0149] It can be seen from the technical solution provided by the above embodiments of the present application that when the cloud encoder detects a scene switch in the currently acquired continuous image frames, the updated static metadata corresponding to the currently acquired continuous image frames is used as static metadata that is more suitable for the scene after the switch, and the current static metadata is switched to the updated static metadata, so that the image frames after the scene switch are encoded based on the updated static metadata, and then through the end-cloud collaboration, the terminal decoder is notified to perform the corresponding static metadata switching, thereby improving the timeliness of the dynamic switching of metadata, so that the target display image frame after the terminal decodes can have different dynamic ranges and picture performances in different scenes, so that in the scene where the terminal decoder does not have the ability to parse dynamic metadata, it can also achieve a picture performance effect similar to the dynamic metadata while ensuring low picture latency, thereby improving the user experience.

[0150] See also Figure 7 , Figure 7 1 is a schematic diagram of an image processing architecture in a cloud gaming scenario provided by an embodiment of the present application. Specifically, the image processing architecture may include: a cloud and a terminal, the cloud includes an image rendering terminal and an encoding terminal. The terminal includes: a decoding terminal.

[0151] In the image processing scheme of the cloud game scene, the game screen data rendered by the image rendering end will be sent to the encoding end for further encoding. At this time, the encoding end needs to determine whether the scene has switched. If the scene has switched, the HDR static metadata used in the previous scene is not applicable to the new scene, and it is necessary to encode the static metadata applicable to the new scene characteristics. Once the encoding end detects the scene switch, it resets the static metadata of the encoder according to the latest HDR metadata provided by the image rendering end, realizes the switching of static metadata, and thus generates the switched HDR code stream. At this time, the switched static metadata and the switched HDR code stream will be notified to the decoding end through the specific field in the end-cloud collaboration parameters. The decoding end needs to be restarted and reset with the switched static metadata, and then decode the switched HDR code stream to obtain the target display screen. In the process of cloud game screen production, the present invention detects that the game scene has changed, and realizes the switching of static metadata and code stream through the end-cloud collaboration method, and obtains the effect of dynamic metadata, so that the entire video has different dynamic ranges and picture performances in different scenes.

[0152] The present application embodiment provides an image processing device with the cloud as the execution subject, such as Figure 8 As shown, the image processing device may include:

[0153] The image frame acquisition module 810 is used to sequentially acquire continuous image frames during the image encoding process based on the current static metadata;

[0154] A first metadata switching module 820 is configured to switch the current static metadata to updated static metadata corresponding to the currently acquired continuous image frames when a scene switch in the currently acquired continuous image frames is detected;

[0155] The image encoding module 830 is used to perform image encoding on the continuous image frames acquired from the target time based on the updated static metadata to obtain an encoded bitstream, where the target time is the acquisition time of the updated static metadata;

[0156] The metadata switching notification sending module 840 is used to send a metadata switching notification carrying updated static metadata to the terminal during the process of transmitting the encoded code stream to the terminal in real time, so that the terminal switches the current static metadata to the updated static metadata, and decodes the encoded code stream based on the updated static metadata to obtain the target display image frame.

[0157] In a specific embodiment, the continuous image frames may be video image frames generated sequentially in real time, and the first metadata switching module 820 may include:

[0158] An image metadata acquisition unit, configured to acquire image metadata corresponding to a target image frame when a scene switch is detected in the currently acquired continuous image frames, the target image frame being the latest image frame generated when the scene switch is detected;

[0159] The updated static metadata switching unit is used to switch the current static metadata to the updated static metadata corresponding to the image metadata.

[0160] In an optional embodiment, the image metadata acquisition unit may include:

[0161] The scene switching notification receiving unit is used to receive a scene switching notification for the currently acquired continuous image frames sent by the image generating end, where the scene switching notification carries image metadata.

[0162] In an optional embodiment, the image metadata acquisition unit may include:

[0163] A scene switching detection unit, configured to perform scene switching detection on a currently encoded image frame in a process of sequentially encoding the currently acquired continuous image frames based on the current static metadata, and obtain a detection result;

[0164] A metadata update request sending unit, configured to send a metadata update request to an image generation end when a detection result indicates that the current encoded image frame is a scene switching frame;

[0165] The image metadata receiving unit is used to receive the image metadata sent by the image generating end in response to the metadata update request.

[0166] In a specific embodiment, the scene switching detection unit may include:

[0167] A prediction block ratio information determination unit, used to determine the prediction block ratio information corresponding to the current coded image frame;

[0168] The scene switching frame determining unit is used to determine that the current coded image frame is a scene switching frame when the prediction block proportion information meets a preset proportion condition.

[0169] In an optional embodiment, the first metadata switching module 820 may include:

[0170] The interval execution unit is used to execute the step of switching the current static metadata to the updated static metadata corresponding to the currently acquired continuous image frames when the interval between the current moment and the last metadata switching moment meets the preset interval.

[0171] It should be noted that the device in the device embodiment and the method embodiment are based on the same inventive concept.

[0172] The present application embodiment provides an image processing device with a terminal as an execution subject, such as Fig. 9 As shown, the image processing device may include:

[0173] The second metadata switching module 910 is used to switch the current static metadata to the updated static metadata in response to the metadata switching notification carrying the updated static metadata sent by the cloud during the real-time transmission of the encoded bitstream;

[0174] A first image decoding module 920, configured to decode the coded bitstream based on the updated static metadata to obtain a target display image frame;

[0175] Among them, updating static metadata is the static image metadata corresponding to the currently acquired continuous image frames obtained when the cloud detects a scene switch in the currently acquired continuous image frames; the encoded code stream is the code stream obtained after the cloud encodes the target image frames based on the updated static metadata; the target image frames are the continuous image frames acquired by the cloud from the moment of acquiring the updated static metadata.

[0176] In an optional embodiment, the second metadata switching module 910 may include:

[0177] The decoder restart unit is used to restart the decoder when the interval between the current moment and the last metadata switching moment meets the preset interval, and configure the static metadata in the restarted decoder to update the static metadata.

[0178] In an optional embodiment, the above method may further include:

[0179] The second image decoding module is used to decode the coded code stream based on the current static metadata to obtain a target display image frame when the interval between the current moment and the last metadata switching moment does not meet the preset interval.

[0180] It should be noted that the device in the device embodiment and the method embodiment are based on the same inventive concept.

[0181] An embodiment of the present application provides an image processing device, which includes a processor and a memory, wherein the memory stores at least one instruction or at least one program, and the at least one instruction or the at least one program is loaded and executed by the processor to implement the image processing method provided in the above method embodiment.

[0182] Further, Fig.10A hardware structure diagram of an image processing device for implementing the image processing method provided in the embodiment of the present application is shown, and the image processing device may participate in or include the image processing apparatus provided in the embodiment of the present application. Fig.10 As shown, the image processing device 100 may include one or more (1002a, 1002b, ..., 1002n are used to illustrate) processors 1002 (the processor 1002 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 1004 for storing data, and a transmission device 1006 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the I / O interface), a network interface, a power supply and / or a camera. It can be understood by those skilled in the art that Fig.10 The structure shown is only for illustration and does not limit the structure of the above electronic device. Fig.10 More or fewer components as shown, or with Fig.10 Different configurations are shown.

[0183] It should be noted that the one or more processors 1002 and / or other data processing circuits described above may generally be referred to herein as "data processing circuits". The data processing circuits may be embodied in whole or in part as software, hardware, firmware, or any other combination thereof. In addition, the data processing circuit may be a single independent processing module, or may be incorporated in whole or in part into any of the other components in the image processing device 100 (or mobile device). As involved in the embodiments of the present application, the data processing circuit acts as a processor control (e.g., selection of a variable resistor terminal path connected to an interface).

[0184] The memory 1004 can be used to store software programs and modules of application software, such as program instructions / data storage devices corresponding to the image processing method described in the embodiment of the present application. The processor 1002 executes various functional applications and data processing by running the software programs and modules stored in the memory 1004, that is, realizing the above-mentioned image processing method. The memory 1004 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 1004 may further include a memory remotely arranged relative to the processor 1002, and these remote memories may be connected to the image processing device 100 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0185] The transmission device 1006 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the image processing device 100. In one example, the transmission device 1006 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one embodiment, the transmission device 1006 can be a radio frequency (Radio Frequency, RF) module, which is used to communicate with the Internet wirelessly.

[0186] The display may be, for example, a touch screen type liquid crystal display (LCD), which may enable a user to interact with a user interface of the image processing device 100 (or mobile device).

[0187] An embodiment of the present application also provides a computer-readable storage medium, which can be set in an image processing device to store at least one instruction or at least one program related to the image processing method in the method embodiment. The at least one instruction or the at least one program is loaded and executed by the processor to implement the image processing method provided by the above-mentioned method embodiment.

[0188] Optionally, in this embodiment, the storage medium may be located in at least one of the multiple network servers of the computer network. Optionally, in this embodiment, the storage medium may include, but is not limited to, various media that can store program codes, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk, or an optical disk.

[0189] The embodiment of the present application also provides a computer program product or a computer program, which includes a computer instruction stored in a computer-readable storage medium. The processor of the computer device reads the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device performs the image processing method provided in the method embodiment.

[0190] It should be noted that the above-mentioned sequence of the embodiments of the present application is for description only and does not represent the advantages and disadvantages of the embodiments. The above-mentioned specific embodiments of the present application are described. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0191] Each embodiment in this application is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device and equipment embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.

[0192] A person skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware or by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.

[0193] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application should be included in the protection scope of the present application.

Claims

1. An image processing method, characterized in that: The method comprises: In the process of encoding the image based on the current static metadata, sequentially acquiring consecutive image frames; In case of detecting a scene switch in the currently acquired continuous image frames, switching the current static metadata to updated static metadata corresponding to the currently acquired continuous image frames; Based on the updated static metadata, image encoding is performed on the continuous image frames acquired from a target time to obtain an encoded bitstream, wherein the target time is the acquisition time of the updated static metadata; In the process of transmitting the encoded code stream to the terminal in real time, a metadata switching notification carrying the updated static metadata is sent to the terminal, so that the terminal switches the current static metadata to the updated static metadata, and based on the updated static metadata, performs image decoding on the encoded code stream to obtain a target display image frame.

2. The method according to claim 1, characterized in that The continuous image frames are video image frames generated sequentially in real time, and the step of switching the current static metadata to updated static metadata corresponding to the currently acquired continuous image frames includes: In the case of detecting a scene switch in the currently acquired continuous image frames, acquiring image metadata corresponding to a target image frame, the target image frame being the most recently generated image frame when the scene switch is detected; The current static metadata is switched to updated static metadata corresponding to the image metadata.

3. The method according to claim 2, characterized in that The step of acquiring image metadata corresponding to the target image frame includes: A scene switching notification for the currently acquired continuous image frames is received from an image generation end, where the scene switching notification carries the image metadata.

4. The method according to claim 2, characterized in that: The step of acquiring image metadata corresponding to the target image frame includes: In the process of sequentially encoding the currently acquired continuous image frames based on the current static metadata, performing scene switching detection on the currently encoded image frames to obtain detection results; When the detection result indicates that the current encoded image frame is a scene switching frame, sending a metadata update request to the image generation end; The image metadata sent by the image generating end in response to the metadata update request is received.

5. The method according to claim 4, characterized in that The performing scene switching detection on the current coded image frame to obtain the detection result comprises: Determine the prediction block ratio information corresponding to the current coded image frame; When the prediction block proportion information satisfies a preset proportion condition, it is determined that the current encoded image frame is a scene switching frame.

6. According to any one of the methods of claims 1 to 5, switching the current static metadata to updated static metadata corresponding to the currently acquired continuous image frames comprises: When the interval between the current moment and the last metadata switching moment satisfies the preset interval, the step of switching the current static metadata to updated static metadata corresponding to the currently acquired continuous image frames is performed.

7. An image processing method, characterized in that: The method comprises: In response to a metadata switching notification carrying updated static metadata sent by the cloud during real-time transmission of the encoded bitstream, switching the current static metadata to the updated static metadata; Based on the updated static metadata, image decoding is performed on the encoded code stream to obtain a target display image frame; Among them, the updated static metadata is the static image metadata corresponding to the currently acquired continuous image frames obtained when the cloud detects a scene switch in the currently acquired continuous image frames; the encoded code stream is the code stream obtained after the cloud encodes the target image frames based on the updated static metadata; the target image frames are the continuous image frames acquired by the cloud from the moment the updated static metadata is acquired; and the continuous image frames are video image frames generated sequentially in real time.

8. The method according to claim 7, characterized in that The switching of the current static metadata to the updated static metadata comprises: When the interval between the current moment and the last metadata switching moment meets the preset interval, the decoder is restarted, and the static metadata in the restarted decoder is configured as the updated static metadata.

9. The method according to claim 8, characterized in that The method further comprises: When the interval between the current moment and the last metadata switching moment does not satisfy the preset interval, the encoded code stream is image decoded based on the current static metadata to obtain the target display image frame.

10. An image processing device, characterized in that: The device comprises: An image frame acquisition module, used to sequentially acquire continuous image frames during image encoding based on current static metadata; A first metadata switching module is used to switch the current static metadata to updated static metadata corresponding to the currently acquired continuous image frames when a scene switch in the currently acquired continuous image frames is detected; An image encoding module, configured to perform image encoding on the continuous image frames acquired from a target time based on the updated static metadata to obtain an encoded bitstream, wherein the target time is the acquisition time of the updated static metadata; The metadata switching notification sending module is used to send a metadata switching notification carrying the updated static metadata to the terminal during the process of transmitting the encoded code stream to the terminal in real time, so that the terminal switches the current static metadata to the updated static metadata, and based on the updated static metadata, performs image decoding on the encoded code stream to obtain a target display image frame.

11. An image processing device, characterized in that: The device comprises: A second metadata switching module is configured to respond to a metadata switching notification carrying updated static metadata sent by the cloud during real-time transmission of the encoded bitstream, and to switch the current static metadata to the updated static metadata; A first image decoding module, configured to perform image decoding on the coded bitstream based on the updated static metadata to obtain a target display image frame; Among them, the updated static metadata is the static image metadata corresponding to the currently acquired continuous image frames obtained when the cloud detects a scene switch in the currently acquired continuous image frames; the encoded code stream is the code stream obtained after the cloud encodes the target image frames based on the updated static metadata; the target image frames are the continuous image frames acquired by the cloud from the moment of acquiring the updated static metadata.

12. An image processing device, characterized in that: The device includes a processor and a memory, wherein the memory stores at least one instruction or at least one program, and the at least one instruction or the at least one program is loaded and executed by the processor to implement the image processing method as described in any one of claims 1 to 6 or the image processing method as described in any one of claims 7 to 9.

13. A computer-readable storage medium, characterized in that: The storage medium stores at least one instruction or at least one program, and the at least one instruction or the at least one program is loaded and executed by the processor to implement the image processing method as described in any one of claims 1 to 6 or the image processing method as described in any one of claims 7 to 9.

14. A computer program product, characterized in that The computer program product comprises at least one instruction or at least one program, and the at least one instruction or the at least one program is loaded and executed by a processor to implement the image processing method as described in any one of claims 1 to 6 or the image processing method as described in any one of claims 7 to 9.

Images