Construction method of training sample, training method of model, electronic equipment and medium

Abstract

The invention provides a training sample construction method, a model training method, an electronic device and a medium, and the training sample construction method comprises the steps: obtaining a first image having a first graph compared with a basic image, the first graph having analog distortion sawteeth; the first image and the second image serve as training samples, the position of the second image in the second image corresponds to the position of the first image in the first image, and the difference between the shape of the second image and the shape of the first image comprises the difference between the first image and the second image. The second pattern does not have analog distortion sawteeth. According to the technical scheme, the damage to the image quality of other areas in the training sample in the process of constructing the analog distortion sawteeth can be reduced, and the quality of the training sample is improved.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a method for constructing training samples, a method for training a model, an electronic device, and a medium. Background Art

[0002] In the field of image processing, some processes can cause aliasing in images. For example, aliasing can occur in rendered images, as can aliasing in encoded and decoded images. The presence of aliasing in images can degrade the user's viewing experience. Using existing anti-aliasing methods to remove aliasing can negatively impact the image, such as reducing overall image quality and causing image distortion. Summary of the Invention

[0003] In view of this, the embodiments of the present application provide a method for constructing a training sample, a method for training a model, and a device, which can reduce the damage to the image quality of other areas in the training sample caused by the process of constructing simulated distortion aliasing, and improve the quality of the training sample.

[0004] In a first aspect, an embodiment of the present application provides a method for constructing a training sample, comprising: obtaining a first image having a first graphic compared to a base image, wherein the first graphic has simulated distortion aliasing; obtaining a second image having a second graphic compared to the base image, wherein the first image and the second image serve as training samples, the position of the second graphic in the second image corresponds to the position of the first graphic in the first image, and the difference between the shape of the second graphic and the shape of the first graphic includes that the second graphic does not have simulated distortion aliasing.

[0005] In a second aspect, an embodiment of the present application provides a model training method, comprising: constructing a training sample according to the training sample construction method described in the first aspect; and using the training sample to train an initial model to obtain a target model.

[0006] In the third aspect, an embodiment of the present application provides a model deployment method, including: sending a model acquisition request to a server, the model acquisition request carries distortion sawtooth type information, the distortion sawtooth type information is used to characterize a specified type of distortion sawtooth, the server is used to obtain a target model based on the distortion sawtooth type information, the target model is used to remove the specified type of distortion sawtooth; receiving the target model sent by the server to deploy the target model on the target terminal.

[0007] In a fourth aspect, an embodiment of the present application provides a model deployment method, including: receiving a model acquisition request sent by a target terminal, the model acquisition request carries distortion sawtooth type information, and the distortion sawtooth type information is used to characterize a specified type of distortion sawtooth; obtaining a target model according to the distortion sawtooth type information, wherein the target model is used to remove the specified type of distortion sawtooth; and sending the target model to the target terminal to deploy the target model on the target terminal.

[0008] In a fifth aspect, an embodiment of the present application provides a device for constructing a training sample, comprising: a first acquisition module for acquiring a first image having a first graphic compared to a base image, wherein the first graphic has simulated distortion aliasing; a second acquisition module for acquiring a second image having a second graphic compared to the base image, wherein the first image and the second image serve as training samples, the position of the second graphic in the second image corresponds to the position of the first graphic in the first image, and the difference between the shape of the second graphic and the shape of the first graphic includes that the second graphic does not have simulated distortion aliasing.

[0009] In a sixth aspect, an embodiment of the present application provides a model training device, comprising: a construction module for constructing training samples according to the training sample construction method described in the first aspect; and a training module for training an initial model using the training samples to obtain a target model.

[0010] In the seventh aspect, an embodiment of the present application provides a model deployment device, including: a sending module for sending a model acquisition request to a server, the model acquisition request carries distortion sawtooth type information, the distortion sawtooth type information is used to characterize a specified type of distortion sawtooth, the server is used to obtain a target model based on the distortion sawtooth type information, and the target model is used to remove the specified type of distortion sawtooth; the receiving module is used to receive the target model sent by the server to deploy the target model on the target terminal.

[0011] In the eighth aspect, an embodiment of the present application provides a model deployment device, including: a receiving module for receiving a model acquisition request sent by a target terminal, the model acquisition request carries distortion sawtooth type information, and the distortion sawtooth type information is used to characterize a specified type of distortion sawtooth; an acquisition module is used to obtain a target model according to the distortion sawtooth type information, wherein the target model is used to remove the specified type of distortion sawtooth; a sending module is used to send the target model to the target terminal to deploy the target model on the target terminal.

[0012] In a ninth aspect, an embodiment of the present application provides an electronic device comprising: a processor; and a memory for storing processor-executable instructions, wherein the processor is used to execute the above-mentioned training sample construction method, model training method, or model deployment method.

[0013] In a tenth aspect, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program, and the computer program is used to execute the above-mentioned training sample construction method, model training method, or model deployment method.

[0014] On the eleventh aspect, an embodiment of the present application provides a computer program product, which includes instructions. When the instructions are executed by a processor of a computer device, the computer device is able to execute the above-mentioned training sample construction method, model training method, or model deployment method.

[0015] In the twelfth aspect, an embodiment of the present application provides a chip comprising: a processor; and a memory for storing processor-executable instructions, wherein the processor is used to execute the above-mentioned training sample construction method, model training method, or model deployment method.

[0016] The embodiments of the present application provide a method for constructing a training sample, a method for training a model, and an apparatus. By obtaining a first image having a first graphic compared to a base image, and obtaining a second image having a second graphic compared to the base image, the first image and the second image can be used as training samples, wherein the first graphic has simulated distortion aliasing and the second graphic does not have simulated distortion aliasing. Since the first graphic can be located in a specified area in the first image and the second graphic can be located in a specified area in the second image, the damage to the image quality of other areas in the training sample caused by the process of constructing the simulated distortion aliasing can be reduced, and the quality of the training sample can be improved, that is, a targeted training sample can be obtained. Using the training sample to train the model can improve the effect of the model in removing distortion aliasing and reduce the impact of the model on the overall image quality while removing distortion aliasing. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 Shown is a schematic diagram of the system architecture of a model training system provided by an exemplary embodiment of the present application.

[0018] Figure 2 Shown is an interactive schematic diagram of the interactive end of the model training method provided by an exemplary embodiment of the present application.

[0019] Figure 3 Shown is a flow chart of a method for constructing training samples provided by an exemplary embodiment of the present application.

[0020] Figure 4A Shown is a schematic diagram of a second image mask provided by an exemplary embodiment of the present application.

[0021] Figure 4B Shown is a schematic diagram of a first image mask provided by an exemplary embodiment of the present application.

[0022] Figure 4C Shown is a schematic diagram of a second image mask provided by another exemplary embodiment of the present application.

[0023] Figure 4D Shown is a schematic diagram of a first image mask provided by another exemplary embodiment of the present application.

[0024] Figure 4E Shown is a schematic diagram of a second image mask provided by another exemplary embodiment of the present application.

[0025] Figure 4F Shown is a schematic diagram of a first image mask provided by another exemplary embodiment of the present application.

[0026] Figure 4G Shown is a schematic diagram of a first image mask provided by another exemplary embodiment of the present application.

[0027] Figure 4H Shown is a schematic diagram of a first image mask provided by another exemplary embodiment of the present application.

[0028] Figure 4I Shown is a schematic diagram of a second image mask provided by another exemplary embodiment of the present application.

[0029] Figure 4J Shown is a schematic diagram of a second image mask provided by another exemplary embodiment of the present application.

[0030] Figure 4K Shown is a schematic diagram of a fourth image mask provided by an exemplary embodiment of the present application.

[0031] Figure 4L Shown is a schematic diagram of a first image mask provided by another exemplary embodiment of the present application.

[0032] Figure 4M FIG2 is a schematic diagram of an image mask obtained after staggering and adjusting the blocks in the fifth image mask provided by an exemplary embodiment of the present application.

[0033] Figure 4N Shown is a schematic diagram of a second image mask provided by another exemplary embodiment of the present application.

[0034] Figure 4O Shown is a schematic diagram of a 4x3 sawtooth unit structure provided by an exemplary embodiment of the present application.

[0035] Figure 4P Shown are schematic diagrams of a sawtooth unit structure before and after repair provided by an exemplary embodiment of the present application.

[0036] Figure 4QShown is a schematic diagram of a first image mask provided by another exemplary embodiment of the present application.

[0037] Figure 4Q' Schematic diagrams of a first image mask without extension lines and a first image mask with extension lines provided by an exemplary embodiment of the present application are shown.

[0038] Figure 4Q ” is a schematic diagram of a second image mask with an extension line added provided by an exemplary embodiment of the present application.

[0039] Figure 4R Shown is a schematic diagram of a random text mask provided by an exemplary embodiment of the present application.

[0040] Figure 4S Shown is a schematic diagram of an image obtained after adding color information to a random text mask provided by an exemplary embodiment of the present application.

[0041] Figure 4T Shown is a schematic diagram of an image obtained after random text is embedded in a basic image provided by an exemplary embodiment of the present application.

[0042] Figure 4U FIG2 is a schematic diagram of an image obtained after adding rendering noise to an image to be added with noise, provided by an exemplary embodiment of the present application.

[0043] Figure 4V FIG2 is a schematic diagram showing the location of a rendering hole provided by an exemplary embodiment of the present application.

[0044] Figure 4W Shown is a schematic diagram of a basic image provided by an exemplary embodiment of the present application.

[0045] Figure 4X Shown is a schematic diagram of a second image provided by an exemplary embodiment of the present application.

[0046] Figure 4Y Shown is a schematic diagram of a first image provided by an exemplary embodiment of the present application.

[0047] Figure 4Z Shown is a schematic diagram of a first image provided by another exemplary embodiment of the present application.

[0048] Figure 5 Shown is a flowchart of a model training method provided by an exemplary embodiment of the present application.

[0049] Figure 6 Shown is a flow chart of a model deployment method provided by an exemplary embodiment of the present application.

[0050] Figure 7Shown is a flow chart of a model deployment method provided by another exemplary embodiment of the present application.

[0051] Figure 8 Shown is a schematic structural diagram of a training sample construction device provided by an exemplary embodiment of the present application.

[0052] Figure 9 Shown is a schematic structural diagram of a model training device provided by an exemplary embodiment of the present application.

[0053] Figure 10 Shown is a schematic structural diagram of a deployment device for a model provided by an exemplary embodiment of the present application.

[0054] Figure 11 Shown is a structural schematic diagram of a deployment device of a model provided by another exemplary embodiment of the present application.

[0055] Figure 12 Shown is a block diagram of an electronic device for executing a training sample construction method, a model training method, or a model deployment method provided by an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0056] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0057] Application Overview

[0058] In the field of video processing, it is necessary to upsample a low-resolution video and then render it to obtain the final displayed video. Specifically, for example, in scenes such as videos in the field of sports and game videos, the lines in the final displayed video may be accompanied by jagged images that are difficult to see. Generally speaking, encoding, decoding or rendering a video will cause the appearance of jagged images. Taking rendering jagged images as an example, the causes of rendering jagged images include: the rendering calculation coordinates of the lines are real numbers, but when mapped to the pixel space, the coordinates can only be integers, thus introducing jagged images in the video or image.

[0059] In the related technology, methods for removing jagged edges include: super sampling / multisampling anti-aliasing (SSAA / MSAA) method, temporal anti-aliasing (TAA) method, subpixel morphological anti-aliasing (SMAA) method, fast approximate anti-aliasing (FXAA) method, and Nvidia's AI method, etc.

[0060] However, using the aforementioned anti-aliasing methods to remove aliasing in an image can negatively impact the overall image quality. For example, this can cause distortion in areas other than the aliasing region. Distortion in this context can mean blurring or other degradation of image quality.

[0061] In response to the above technical problems, an embodiment of the present application provides a method for constructing training samples. A first image is obtained by adding a first graphic with simulated distortion aliasing to a specified area of ​​a base image, and a second image is obtained by adding a second graphic without simulated distortion aliasing to a specified area of ​​the base image. In this way, a training sample can be determined based on the first image and the second image for training a model. Because the graphic is added to a specified area of ​​the base image, it will not affect the quality of other areas of the base image, thereby improving the quality of the training sample. Furthermore, the model trained using the training sample can specifically remove the distortion aliasing in the image, reducing the impact on the overall image quality.

[0062] Exemplary Systems

[0063] Figure 1 FIG. 1 is a schematic diagram of the system architecture of a model training system provided by an exemplary embodiment of the present application. Figure 1 As shown, the training system 100 includes a server 110 and a target terminal 120. The server 110 can be used to execute the model training method.

[0064] Exemplarily, the server 110 may be an electronic device with a certain computing and processing capability, which may have a network communication module, a processor, a memory, etc. Of course, the server 110 may also refer to software running in the electronic device. The server 110 may also be a distributed server, which may be a system having multiple processors, memories, network communication modules, etc. operating in collaboration. Alternatively, the server 110 may also be a server cluster formed by several servers. Alternatively, with the development of science and technology, the server 110 may also be a new technical means capable of realizing the corresponding functions of the embodiments of the specification, for example, it may be a new form of "server" based on quantum computing. In an exemplary scenario, the server 110 may be expressed as a cloud platform.

[0065] The target terminal 120 can provide a model training settings page and a page for receiving the trained model sent by the server 110. The target terminal 120 can be an electronic device with network access capabilities, such as a mobile phone, a server, or a computer. The target terminal 120 can be connected to the server 110 for communication.

[0066] In an application scenario example, Figure 2 As shown, the server 110 and the target terminal 120 can serve as interactive terminals of the model training method. The user of the target terminal 120 may be a person who has a model training requirement and hopes to train the model through the training system 100 to obtain a target model with the function of removing distortion and aliasing.

[0067] Specifically, the target terminal 120 may receive the function information and the distortion sawtooth type information of the model selected by the user, and send the function information and the distortion sawtooth type information of the model to the server 110 .

[0068] For example, the model's functional information may represent other functions of the model that the user desires to train, in addition to removing distortion artifacts. The model's functional information may include information indicating at least one of the following: denoising, deblurring, detail generation, compression artifact removal, and super-resolution. The distortion artifact type information may include information indicating at least one of the following: warping artifacts, residual deinterlacing artifacts, and rendering artifacts.

[0069] Server 110 can embed random text in the base image and, based on the aliasing type information, add a first graphic with simulated aliasing to the base image to obtain a first intermediate image. Similarly, server 110 can embed random text in the base image and add a second graphic without simulated aliasing to the base image to obtain a second image.

[0070] The server 110 may perform image processing on the first intermediate image according to the functional information of the model to obtain a first image.

[0071] Exemplarily, the position of the random text in the first image is the same as the position of the random text in the second image, the position of the first graphic in the first image corresponds to the position of the second graphic in the second image, and the difference between the shape of the first graphic and the shape of the second graphic includes that the second graphic does not have simulated distortion jaggedness.

[0072] Image processing may include at least one of noise addition, blurring, image quality degradation, encoding and compression, and downsampling. For example, the model's functional information may include information indicating a denoising function. Based on the model's functional information, server 110 may perform noise addition on the first intermediate image, resulting in an image with simulated distortion, jagged edges, and noise, namely, the first image. The model is trained using the first and second images, resulting in a target model that can remove distortion, jagged edges, and perform denoising. Similarly, if the model's functional information includes information corresponding to multiple functions, server 110 may perform multiple image processing operations on the first intermediate image based on the model's functional information.

[0073] The server 110 may select a model according to the function information of the model and the distortion and aliasing type information.

[0074] For example, the server 110 may select a model with a suitable network structure based on the functional information of the model and the distortion sawtooth type information. The network structure can meet the training expectations, but the network complexity is as low as possible, thereby optimizing the cost.

[0075] The server 110 may use the first image and the second image as training samples to train the model to obtain a target model. The server 110 may send the target model to the target terminal 120. The target terminal 120 may automatically configure the target model for the user to use.

[0076] Furthermore, during the use of the target model, the target model may process the input image to obtain an output image, and the output image may contain distorted areas circled by the user. For example, the output image of the target model may contain distorted areas, and the distorted areas in the output image may be circled by the user. The target terminal 120 may send the input image and output image of the target model to the server 110 in accordance with user instructions. The server 110 may replace the circled distorted areas in the output image with the corresponding areas in the input image to obtain a modified image, and store the input image and the modified image as a set of training samples. The server 110 may use this set of training samples to perform secondary training on the target model and send the secondary trained model to the target terminal 120.

[0077] It should be understood that the above application scenario examples are only provided to facilitate understanding of the spirit and principles of the present application, and the embodiments of the present application are not limited thereto. On the contrary, the embodiments of the present application can be applied to any applicable scenario.

[0078] Exemplary Methods

[0079] Figure 3 Shown is a flow chart of a method for constructing training samples provided by an exemplary embodiment of the present application. Figure 3 The method can be Figure 1 In other embodiments, the server executes Figure 3 The method can also be Figure 1 The target terminal in the execution. Figure 3 As shown, the method for constructing the training sample may include the following content.

[0080] 310: Acquire a first image having a first pattern compared to a base image, wherein the first pattern has simulated distortion aliasing.

[0081] The first figure may be a regular figure or an irregular figure composed of lines. For example, the first figure may be a straight line, a curve, a circle, or a polygon.

[0082] Some or all of the lines in the first graphic exhibit simulated aliasing. Aliasing can indicate a rough line or discontinuous line edges. It typically occurs in images that have undergone rendering, encoding, decoding, or other image processing. It should be understood that in some cases, aliasing in lines is also referred to as aliasing. However, to avoid ambiguity, that is, to avoid understanding aliasing as a jagged pattern constructed from lines, embodiments of the present application refer to aliasing that causes rough or discontinuous lines as aliasing. For example, aliasing can be at least one of distortion aliasing, residual deinterlacing aliasing, and rendering aliasing. Simulated aliasing can be aliasing simulated using lines as a carrier through a simulation method. Specifically, in some embodiments, an image containing the graphic can be rendered, encoded, decoded, or otherwise processed to impart simulated aliasing to the image, thereby obtaining the first graphic. It should be understood that the type of simulated aliasing can correspond to the type of real aliasing, such as distortion aliasing, residual deinterlacing aliasing, and rendering aliasing.

[0083] The first graphic may be located in a designated area, such as a local area, in the first image. Of course, in other cases, the first graphic may also cover the entire area of ​​the first image.

[0084] The base image can be an image composed of multiple pixel arrangements and can be an image from any scenario, such as an image in a gaming setting, a medical setting, or a sports setting. In some embodiments, the base image can be an image without aliasing. By adding a first graphic having simulated aliasing to the base image, simulated aliasing is introduced into the base image, thereby obtaining the first image. In other embodiments, the base image itself may have aliasing. By adding an additional first graphic having simulated aliasing to the base image, the first image can be obtained.

[0085] 320: Obtain a second image having a second graphic compared to the base image, wherein the first image and the second image serve as training samples, a position of the second graphic in the second image corresponds to a position of the first graphic in the first image, and a difference between a shape of the second graphic and a shape of the first graphic includes that the second graphic does not have simulated distortion jaggedness.

[0086] The first image and the second image are a set of comparison images. The line trends of the first graphic and the line trends of the second graphic can be the same, and the difference between the shapes of the first graphic and the second graphic includes that the second graphic does not have simulated distortion jagged edges.

[0087] In some embodiments, the base image can be regarded as a high-quality original image. For example, a first graphic with simulated distortion aliasing can be added to the base image to obtain a first image; and a second graphic without simulated distortion aliasing can be added to the base image to obtain a second image. The model is trained using the first image and the second image as training samples, and the obtained model can have the function of removing distortion aliasing. Furthermore, since the simulated distortion aliasing is introduced into the base image through the first graphic, rather than by image processing the entire base image, the impact on the image quality of the area outside the first graphic in the first image can be reduced. Training the model with this training sample can improve the model's effect in removing distortion aliasing and reduce the impact of the model on the overall image quality while removing distortion aliasing.

[0088] In some embodiments, the method of adding the first graphic to the basic image can be to directly draw the first graphic on the basic image. For example, the first graphic can be drawn on the basic image in the form of code to obtain the first image; similarly, the second graphic can be drawn on the basic image in the form of code to obtain the second image.

[0089] An embodiment of the present application provides a method for constructing a training sample, by obtaining a first image having a first graphic compared to a base image, and obtaining a second image having a second graphic compared to the base image, the first image and the second image can be used as training samples, wherein the first graphic has simulated distortion aliasing and the second graphic does not have simulated distortion aliasing. Since the first graphic can be located in a specified area in the first image and the second graphic can be located in a specified area in the second image, the damage to the image quality of other areas in the training sample caused by the process of constructing the simulated distortion aliasing can be reduced, and the quality of the training sample can be improved, that is, a targeted training sample is obtained. Using the training sample to train the model can improve the effect of the model in removing distortion aliasing and reduce the impact of the model on the overall image quality while removing distortion aliasing.

[0090] According to one embodiment of the present application, training samples are used to train a target model, which is used to remove a specified type of distortion aliasing. The method for constructing the training samples further includes: selecting a target graphic set from multiple graphic sets based on information indicating the specified type of distortion aliasing; and determining a first graphic from the target graphic set, wherein the simulated distortion aliasing of the first graphic belongs to the specified type.

[0091] There are multiple types of aliasing, including distortion aliasing, deinterlacing residual aliasing, rendering aliasing, etc. The aliasing type information may represent a specific type of aliasing, which may be at least one of the above types.

[0092] A graphic set can be a set of graphics with the same type of simulated distortion aliasing. The graphics in different graphic sets correspond to different types of simulated distortion aliasing. For a type of simulated distortion aliasing, at least one graphic can be created, and at least one graphic constitutes a graphic set. When a graphic set includes multiple graphics, the multiple graphics can differ in shape and / or size. For example, the shape of any graphic in a graphic set can be a straight line, a curve, a circle, or a polygon; the color of any graphic can be a solid color such as black, white, or red, or any graphic can have a gradient color.

[0093] The distortion sawtooth type information may be determined by the server according to user input, or may be determined randomly by the server.

[0094] A target graphic set can be selected from multiple graphic sets based on the distortion aliasing type information, and a first graphic can be selected from the target graphic set. The selection of the first graphic can be determined by the server based on user input or randomly determined by the server. Each first graphic can have a corresponding second graphic. When the first graphic is obtained based on the distortion aliasing type information, the second graphic can be obtained based on the first graphic.

[0095] In some embodiments, the specified type of distortion aliasing represented by the distortion aliasing type information can be one type or multiple types. When the specified type is multiple types, multiple target graphic sets can be selected from multiple graphic sets based on the distortion aliasing type information, and a first graphic with simulated distortion aliasing is obtained from each target graphic set and added to the base image to obtain a first image with multiple simulated distortion aliasing. That is, the first image includes multiple first graphics, each of which has one type of simulated distortion aliasing. Similarly, a second graphic without simulated distortion aliasing corresponding to each of the multiple first graphics is added to the base image to obtain a second image. The model is trained using the first image and the second image, so that the trained target model can have the function of removing multiple types of distortion aliasing.

[0096] In this embodiment, by providing multiple graphic sets, it is possible to easily produce a first image with one or more types of simulated distortion aliasing. This can provide users with a variety of options, meet their diverse needs, improve the richness of training samples, and enable the target model to have the function of removing multiple types of distortion aliasing.

[0097] According to an embodiment of the present application, step 310 may include combining the base image with a first image mask representing the first graphic to embed the first graphic in the base image, thereby obtaining the first image. Step 320 may include combining the base image with a second image mask representing the second graphic to embed the second graphic in the base image, thereby obtaining the second image, wherein the size of the second image mask is the same as the size of the first image mask.

[0098] The first graphic and the second graphic can be represented by an image mask. Specifically, the first graphic can be represented by the first image mask. For example, the first image mask can be a first graphic produced on a background image, and the background image can be regarded as a carrier of the first graphic. The first image mask can also be called a simulated distortion sawtooth mask. For different application scenarios, customized simulated distortion sawtooth masks can be generated, that is, different types of simulated distortion sawtooth masks can be generated. Therefore, the graphic set can also be called a mask set. Each mask set can correspond to a type of simulated distortion sawtooth, and can include at least one simulated distortion sawtooth mask. The generation process of the simulated distortion sawtooth mask can be executed by a server, or it can be executed by other electronic devices, and the generated simulated distortion sawtooth mask is stored on the server.

[0099] Similarly, the second graphic can be represented by a second image mask. For example, the second image mask can be a second graphic produced on a background image, and the background image can be considered as a carrier of the second graphic. The second image mask can also be referred to as a non-simulated distortion serration mask. Each simulated distortion serration mask can correspond to a non-simulated distortion serration mask, and the position of the first graphic in the simulated distortion serration mask corresponds to the position of the second graphic in the non-simulated distortion serration mask. The non-simulated distortion serration mask can be stored together with the corresponding simulated distortion serration mask in the mask set where the simulated distortion serration mask is located, or the non-simulated distortion serration mask can be stored separately from the corresponding simulated distortion serration mask.

[0100] In some embodiments, a target mask set can be selected from multiple mask sets based on the distortion aliasing type information, and a first image mask corresponding to the first graphic, i.e., the simulated distortion aliasing mask, can be obtained from the target mask set. By combining the first image mask with the base image, the first graphic with simulated distortion aliasing can be added to the base image to obtain the first image. Similarly, by combining the second image mask corresponding to the first image mask (i.e., the mask without simulated distortion aliasing) with the base image, the second graphic can be added to the base image to obtain the second image.

[0101] For example, the base image may be represented as H0, the first image may be represented as L, the second image may be represented as H, the first image mask may be represented as maskL, and the second image mask may be represented as maskH.

[0102] The combination of the base image H0 and the first image mask maskL can be achieved by the following formula:

[0103] L = H0 * (E - maskL / 255.0) + colormap * maskL / 255.0 (1)

[0104] Similarly, the combination of the base image H0 and the second image mask maskH can be achieved by the following formula:

[0105] H = H0 * (E - maskH / 255.0) + colormap * maskH / 255.0 (2)

[0106] In the above formulas (1) and (2), colormap can be a random solid color map or a gradient color map, which is used to determine the color and brightness of the first graphic / second graphic, which is equivalent to determining the color and brightness of the first graphic / second graphic embedded in the basic image. The sizes of colormap, H0, H, L, maskH and maskL can be consistent. Colormap, H0, H, L, maskH and maskL can be represented by a matrix, where E can represent a matrix with all elements being 1. Specifically, the background image corresponding to maskH and maskL can be black, and the pixel value of black can be 0; the color of the second graphic in maskH and the first graphic in maskL can be white, and the pixel value corresponding to white can be 255. The first term H0*(E-maskL / 255.0) in the above formula (1) can represent the partial area that needs to be subtracted from the basic image, and the second term colormap*maskL / 255.0 in the formula (1) can represent the colored first graphic that needs to be added to the subtracted partial area. Similarly, the first term H0*(E-maskH / 255.0) in the above formula (2) can represent the partial area to be subtracted from the base image, and the second term colormap*maskH / 255.0 in the above formula (2) can represent the colored second graphic to be added to the subtracted partial area. It should be understood that the above formula is only an example, and the coefficients of the various parameters in the above formula can be adjusted according to the color of the background image and the color of the first graphic / second graphic.

[0107] In this embodiment, by combining the base image with the image mask and adding the first graphic / second graphic to the base image, the consistency between the position of the first graphic in the first image and the position of the second graphic in the second image can be improved, so that the difference between the first image and the second image focuses on the presence or absence of simulated distortion jaggedness, thereby improving the quality of the training samples.

[0108] Different methods can be used to generate a first image mask and a corresponding second image mask for different types of simulated aliasing. The generation of the first and second image masks can be performed by a server or other electronic device. The following describes various methods for generating the first and second image masks.

[0109] According to one embodiment of the present application, the method for constructing the training sample also includes: adding a second graphic to the background image to obtain a second image mask; dividing the second graphic in the second image mask into multiple segments, and rotating at least one of two adjacent segments in the multiple segments by a specified angle according to the geometric center of the segment to obtain a first image mask.

[0110] In some embodiments, the size of the background image can be the same as the size of the base image. The color of the background image can be a solid color, such as black. The process of adding the second graphic to the background image can be implemented through code, a network model, or manual means. The shape of the second graphic can be a straight line, and the color of the second graphic can be a solid color, such as white. Figure 4A A second image mask is shown in which the background image is black and the second graphics are white line segments.

[0111] Of course, in other embodiments, the color of the background image can be a solid color other than black, or can be a gradient color. The shape of the second graphic can be a regular shape or an irregular shape other than a straight line. The color of the second graphic can be a solid color other than white, or can be a gradient color.

[0112] Adding a second graphic to the background image can obtain a second image mask, and adjusting the second image mask can obtain a first image mask. Specifically, the second graphic in the second image mask can be divided into multiple segments, and two adjacent segments can be in contact with each other but have no overlapping parts, or there can be overlapping parts between the two adjacent segments. In the extension direction of the second graphic, for any two adjacent segments, the latter of the two segments can be rotated by a specified angle according to the geometric center of the segment to obtain the first image mask. Optionally, the two adjacent segments can be rotated by a specified angle according to the geometric center of the two segments to obtain the first image mask. Here, the specified angles for the rotation of the two adjacent segments can be the same or different. The specified angle can be set according to actual needs. After rotating by the specified angle, the two adjacent segments can be in contact with each other, and the type of simulated distortion aliasing possessed by the first graphic obtained in this way is twisted aliasing.

[0113] For example, Figure 4A The second figure in is divided into multiple segments, and some segments (except the first segment) are rotated so that two adjacent segments are staggered by a certain angle, so that Figure 4B The first image mask is shown.

[0114] The specified angle that each segment is rotated relative to the line connecting the geometric centers of the segments can be expressed as θ, the length of each segment can be expressed as L, and the line width (also called pixel width) of the second figure can be expressed as D. In order to make the simulated distortion aliasing of the first figure more consistent with the actual distortion aliasing and reduce the difficulty of learning the model, it can be set that after a segment is rotated by a specified angle, it still intersects with the adjacent segments. That is, the specified angle θ, the segment length L, and the line width D can satisfy the following formula:

[0115] tan θ * L / 2 ≤ D / 2 (3)

[0116] Or, θ*L / 2≤D / 2(4)

[0117] It should be understood that L, θ and D in the above formulas (3) and (4) can be set according to the specific size requirements of the sawtooth shape, such as being randomly selected within a data range.

[0118] In this embodiment, by dividing the second graphic into multiple segments and rotating some or all of the segments, distorted sawtooth of the twisted sawtooth type can be simulated, that is, a first graphic with simulated distorted sawtooth is obtained.

[0119] According to one embodiment of the present application, the method for constructing the training sample also includes: adding a second graphic to the background image to obtain a second image mask; performing downsampling and upsampling on the second image mask to obtain an intermediate image mask, the size of the intermediate image mask being the same as the size of the second image mask; and adjusting the pixel value of the intermediate image mask according to specified pixel value adjustment requirements to obtain a first image mask.

[0120] The size and color of the background image, the shape and color of the second graphic, and the process of adding the second graphic to the background image can be found in the description of the above embodiment, and will not be described here in detail to avoid repetition. Figure 4C A second image mask is shown in which the background image is black and the second graphic is a white curve.

[0121] The second image mask is downsampled and upsampled to obtain an intermediate image mask, which has the same size as the second image mask. Specifically, downsampling the second image mask compresses the image information in the second image mask, resulting in a downsampled image of reduced size. Upsampling the downsampled image yields the intermediate image mask. Although the intermediate image mask has the same size as the second image mask, it contains less image information than the second image mask. Therefore, after downsampling and upsampling, the second pattern in the second image mask is transformed into a pattern with simulated aliasing in the intermediate image mask. The greater the size difference between the downsampled image and the second image mask, the greater the simulated aliasing in the intermediate image mask.

[0122] In some embodiments, the intermediate image mask can be used directly as the first image mask, and the pattern with simulated aliasing within the intermediate image mask is the first pattern. The simulated aliasing of the first pattern can simulate the aliasing that occurs on lines in an image due to downsampling and upsampling in real scenes. This type of aliasing can also be considered distortion aliasing.

[0123] In some practical situations, the formation of distortion jagged edges in some images is caused by downsampling and upsampling processing as well as some other image processing, which may be at least one of normalization processing, sharpening processing, color adjustment processing, contrast adjustment processing, etc.

[0124] Therefore, in other embodiments, the pixel values ​​of each pixel in the intermediate image mask can be adjusted, and the adjusted mask can be used as the first image mask. By adjusting the pixel values, the simulated aliasing of the first pattern in the first image mask can be made more realistic and more consistent with the actual aliasing.

[0125] Specifically, the pixel values ​​of the intermediate image mask can be adjusted according to the specified pixel value adjustment requirement to obtain the first image mask. The specified pixel value adjustment requirement can be set according to actual needs. For example, the pixel values ​​of pixels within a specified pixel distance from the outline of the graphic in the intermediate image mask can be adjusted. The specified pixel distance and the adjusted pixel values ​​can be set as needed.

[0126] For example, the second image mask can be expressed as maskH, and the intermediate image mask can be expressed as maskI. The specified pixel value adjustment requirement can be expressed by the following formula:

[0127] mask_adjust1 = 255 * a * func(maskI, coef) / (max(func(maskI, coef))+ 1e-6) (5)

[0128] mask_adjust2 = mask_adjust1 * beta + maskH * (1.0 - beta) (6)

[0129] In the above formula (5), func can be a function representing sharpening / smoothing adjustment processing, contrast adjustment processing, color adjustment processing, or other adjustment processing, and coef can be an adjustment parameter. Coef equal to 0 is equivalent to no adjustment, and the value range of coef can be [0, 1] or other appropriate ranges. Formula (5) is equivalent to normalizing func(maskI, coef) to obtain mask_adjust1, where max(func(maskI, coef)) represents the maximum pixel value on func(maskI, coef); a can be a normalization coefficient, specifically a random number in the range of 0.8 to 1; func(maskI, coef) can be replaced by multiple nested functions, for example, func1(func2(…(funcN(maskI, coefN),…), coef2), coef1), where funci can be a function representing various adjustment processing, and coefi can be an adjustment parameter corresponding to various adjustment processing, where i = 1, 2, …, N. Formula (6) represents the fusion of mask_adjust1 after adjustment and maskH before adjustment, which is used to represent the information of the original undistorted image mixed with the aliased distortion generated in real scenes. By introducing formula (6), the robustness of the aliased distortion removal can be enhanced. Beta in formula (6) can be a random number in the range of 0.5 to 1.

[0130] In some cases, mask_adjust1 can represent the image obtained by multiplying the pixel value of each pixel on maskI by a certain coefficient; mask_adjust2 can represent the first image mask, that is, maskL. Through formulas (5) and (6), distortion and aliasing effects mixed with other image processing effects can be simulated.

[0131] Figure 4D The first image mask obtained by using the above formulas (5) and (6) is shown. Figure 4C Compared with the second figure in Figure 4D The first graphic in the image has simulated aliasing.

[0132] In this embodiment, by performing downsampling and upsampling on the second image mask to obtain an intermediate image mask, and adjusting the pixel values ​​of the intermediate image mask according to the specified pixel value adjustment requirements, the distortion aliasing caused by the execution of the downsampling and upsampling processes in the actual scene can be simulated, thereby improving the authenticity of the simulated distortion aliasing.

[0133] According to one embodiment of the present application, the method for constructing the training sample also includes: adding a second graphic to the background image to obtain a second image mask; dividing the second image mask into multiple blocks, each of the multiple blocks includes multiple adjacent pixel rows, and the multiple pixel rows include an upper pixel row and a lower pixel row; swapping the lower pixel row with the upper pixel row to obtain the first image mask.

[0134] The size and color of the background image, the shape and color of the second graphic, and the process of adding the second graphic to the background image can be found in the description of the above embodiment, and will not be described here in detail to avoid repetition. Figure 4E A second image mask is shown in which the background image is black and the second graphics are white line segments.

[0135] The second image mask is divided into a plurality of blocks, each block including a plurality of adjacent pixel rows. The plurality of pixel rows may include an upper portion of pixel rows and a lower portion of pixel rows, and the lower portion of pixel rows are swapped with the upper portion of pixel rows to obtain the first image mask. Alternatively, the plurality of pixel rows may include an upper portion of pixel rows, a middle portion of pixel rows, and a lower portion of pixel rows, and the lower portion of pixel rows are swapped with the upper portion of pixel rows, with the middle portion of pixel rows remaining unchanged or replaced with pixel rows from another block to obtain the first image mask.

[0136] For example, the number of pixel rows in each block can be expressed as dh; the second image mask can be expressed as maskH, and the width of the second image mask is w and the height is h; the first image mask can be expressed as maskL. The first image mask can be obtained by swapping the pixel rows in the lower half and the pixel rows in the upper half of each block using the following formula:

[0137] maskL[i * dh:i * dh + int(dh / 2), 0:w] = maskH[i * dh + dh – int(dh / 2):i * dh + dh, 0:w] (7)

[0138] maskL[i * dh + dh – int(dh / 2):i * dh + dh, 0:w] = maskH[i * dh:i *dh + int(dh / 2), 0:w] (8)

[0139] In the above formulas (7) and (8), i = 0, 1, 2, ..., int (h / dh) - 1, where int is rounded. The j-th pixel row in the i-th block can be expressed as maskH [i * dh + j, 0: w], where j = 0, 1, ..., dh - 1. When dh = 2, the 0-th block includes the 0-th pixel row and the 1-th pixel row, and the 1-th block includes the 2-th pixel row and the 3-th pixel row. Using the above formulas (7) and (8), the 0-th pixel row can be interchanged with the 1-th pixel row, the 2-th pixel row can be interchanged with the 3-th pixel row, and so on, to obtain Figure 4F The first image mask shown. Here, the 0th pixel row / the 2nd pixel row can be regarded as the upper pixel row, and the 1st pixel row / the 3rd pixel row can be regarded as the lower pixel row. When dh=3, the 0th block includes the 0th pixel row to the 2nd pixel row, and the 1st block includes the 3rd pixel row and the 5th pixel row. Using the above formulas (7) and (8), the 0th pixel row and the 2nd pixel row can be interchanged, and the 1st pixel row can be replaced by the 3rd pixel row; the 3rd pixel row and the 5th pixel row can be interchanged, and the 4th pixel row can be replaced by the 6th pixel row, and so on. Figure 4G Here, the 0th pixel row / the 3rd pixel row can be regarded as the upper pixel row, the 1st pixel row / the 4th pixel row can be regarded as the middle pixel row, and the 2nd pixel row / the 5th pixel row can be regarded as the lower pixel row.

[0140] Depend on Figure 4F and Figure 4G It can be seen that the larger the value of dh, the larger the simulated distortion aliasing, that is, the more obvious the simulated distortion aliasing. In other embodiments, dh can also be other integers. In addition, the above-mentioned method of swapping pixel rows is only an example, and the method of swapping pixel rows can also be implemented in other ways.

[0141] It should be understood that the pixel rows in this embodiment can be replaced by pixel columns. Each block may include multiple adjacent pixel columns, and the multiple pixel columns may include at least one first pixel column and at least one second pixel column. By exchanging the first pixel column and the second pixel column, a first image mask can be obtained. Furthermore, the multiple pixel columns may include an intermediate pixel column located between the first pixel column and the second pixel column. The specific process of exchanging pixel columns can refer to the specific process of exchanging pixel rows mentioned above, and will not be repeated here. It should be understood that the concept of pixel rows / pixel columns in this embodiment can be determined according to a specified direction. When the specified direction changes, the pixel rows can be understood as pixel columns.

[0142] In this embodiment, by swapping some pixel rows in the second image mask, aliasing of the type of residual deinterlacing aliasing can be simulated, thereby obtaining a first pattern having simulated aliasing. Here, the residual deinterlacing aliasing may be the aliasing that occurs on lines in an image due to incomplete video deinterlacing in real scenes.

[0143] According to one embodiment of the present application, the method for constructing the training sample also includes: adding a first graphic to the background image to obtain a first image mask; adding a third graphic to the enlarged background image to obtain a third image mask, wherein the third graphic is an enlarged graphic of the first graphic; and downsampling the third image mask to obtain a second image mask.

[0144] In this embodiment, the size and color of the background image, as well as the shape and color of the second graphic, can be found in the description of the above embodiment. The difference between the shapes of the first graphic and the second graphic is that the second graphic does not have simulated distortion jagged edges. The following uses the example of a straight line as the first graphic to illustrate the method of this embodiment. The same is true for a curved line.

[0145] Specifically, the background image can have a width of w and a height of h. Two points (x1, y1) and (x2, y2) are randomly selected on the background image. Connecting these two points yields a line segment, which is the first graphic. Because the coordinates of points in the background image (i.e., pixel space) can only be integers, the first graphic can be a diagonal line, rather than a purely horizontal or vertical line segment. This allows the first graphic to exhibit simulated aliasing, a type of rendering artifact. Figure 4H A first image mask is shown in which the background image is black and the first graphic is a white line segment. The line width of the first graphic may be D.

[0146] By properly filling non-zero pixels between simulated distortion teeth, the simulated distortion teeth can be smoothed or eliminated.

[0147] Specifically, in one embodiment, two points (x1*tile, y1*tile) and (x2*tile, y2*tile) can be selected on an enlarged background image with a width of (w*tile) and a height of (h*tile). A line segment can be obtained by connecting the two points. The line segment can be referred to as a third graphic. The image obtained by adding the third graphic to the enlarged background image is recorded as a third image mask. Similar to the first graphic, the third graphic also has simulated distortion aliasing of the rendering aliasing type. The width / height of the enlarged background image can be tile times the width / height of the background image, and tile can be a number greater than 1. Since the enlarged background image is an enlarged image of the background image, the line width of the third graphic can also be an enlarged image of the first graphic.

[0148] The third image mask is downsampled to obtain a second image mask. The downsampling process here can use the Lanczos algorithm. In some embodiments, the third image mask can be first downsampled to int(w*tile / 2)×int(h*tile / 2) size, and then further downsampled to w×h size, so as to obtain a second image mask, that is, the third figure in the third image mask can be converted into the second figure in the second image mask without simulated distortion aliasing, such as Figure 4I In other embodiments, the third image mask may be downsampled once, three times, or more than three times, as long as the resulting second image mask has a size of w x h. It should be understood that when the number of downsampling processes is multiple, the mask reduction factor corresponding to each downsampling process may be the same or different.

[0149] To ensure that the line widths of the first and second graphics are consistent, the line width of the third graphic in the third image mask can be set to tile*D+alpha, where alpha is an undetermined integer (positive or negative, or even 0). For example, alpha can be 0, meaning that the magnification of the third graphic relative to the first graphic is the same as the magnification of the enlarged background image relative to the background image. In some cases, because the number of downsampling processes can have varying degrees of impact on the line width of the third graphic, the line width of the third graphic can be adjusted using alpha to ensure that the line width of the second graphic, derived from the third graphic, is equal to or approximately equal to the line width of the first graphic.

[0150] The value of alpha can be set according to actual needs. For example, by adjusting the value of alpha, the difference between the area occupied by the first and second shapes can be minimized. Generally, alpha = 0 and tile = 4 can be used. Of course, alpha can also take other appropriate values.

[0151] It should be understood that the above Figure 4A 、 Figure 4C and Figure 4E The second image mask in the corresponding embodiment can be acquired by the method of acquiring the second image mask based on the third image mask in this embodiment.

[0152] In this embodiment, by adding non-pure horizontal or non-pure vertical lines to the background image, rendering aliasing can be simulated, thereby obtaining a first graphic with simulated aliasing. Furthermore, by adding a third graphic to the expanded background image to obtain a third image mask, and then downsampling the third image mask, a second graphic without simulated aliasing can be obtained.

[0153] According to one embodiment of the present application, the method for constructing the training sample also includes: adding a second graphic to the background image to obtain a second image mask; dividing the second graphic in the second image mask into multiple segments, rotating at least one of two adjacent segments in the multiple segments by a specified angle according to the geometric center of the segment to obtain a fourth image mask; binarizing the pixel values ​​of the pixels in the fourth image mask to obtain the first image mask.

[0154] The size and color of the background image, the shape and color of the second graphic, and the process of adding the second graphic to the background image can be found in the description of the above embodiment, and will not be described here in detail to avoid repetition. Figure 4A The method in the corresponding embodiment is used to prepare the second image mask in this embodiment, and Figure 4A The first image mask in the corresponding embodiment serves as the fourth image mask in this embodiment. Therefore, the pattern in the fourth image mask has simulated distorted aliasing of the distorted aliasing type. It should be understood that other methods can also be used to distort the lines of the second pattern to obtain the fourth image mask. Figure 4J A second image mask is shown in which the background image is black and the second graphic is a white line segment. Figure 4K A fourth image mask is shown.

[0155] Since rendering aliasing in real scenes primarily manifests as "pixel steps," each pixel step is typically 1 to 2 pixels high. Due to the cause of rendering aliasing, the error caused by integer coordinates is also 1 to 2 pixels. Therefore, the pixel values ​​of the pixels in the fourth image mask are binarized to obtain the first image mask. The first pattern in the first image mask exhibits pixel steps, i.e., simulated distortion aliasing of the rendering aliasing type.

[0156] The process of binarizing the pixel values ​​of the pixels in the fourth image mask may also be referred to as a pixel corrosion process, and “pixel steps” may be created by pixel corrosion.

[0157] Specifically, Figure 4J and Figure 4K For example, the fourth image mask can be obtained by slightly distorting the lines of the second graphic in the second image mask. Therefore, the pixel values ​​near the outline of the graphic in the fourth image mask may be in the range [0, 255]. The values ​​of the pixels in the fourth image mask can be converted to 0 or 255 through binarization.

[0158] For example, the second image mask may be represented as maskH, the fourth image mask may be represented as maskM, and the first image mask may be represented as maskL. The pixel value of any pixel in maskL may be represented as PL, and the pixel value of any pixel in maskM may be represented as PM.

[0159] For the pixel value PM of any pixel in maskM, a threshold Thr can be pre-set, and pixel erosion can be achieved by the following formula:

[0160] PL = (PM ≥ Thr ? 255 : 0) (9)

[0161] According to the above formula (9), when PM is greater than or equal to Thr, the pixel value PL of the corresponding pixel can be set to 255; when PM is less than Thr, the pixel value PL of the corresponding pixel can be set to 0. All the obtained PLs constitute the maskL, as shown in Figure 4L Here, Thr can be a number between 0 and 255, for example, a number between 100 and 200.

[0162] In this embodiment, the first image mask is obtained by binarizing the pixel values ​​of the pixels in the fourth image mask. This allows for the generation of simulated distortion aliasing of the rendering aliasing type based on the simulated distortion aliasing of the warping aliasing type. This reduces the difficulty of simulating the rendering aliasing and improves the realism of the simulated distortion aliasing of the rendering aliasing type. Furthermore, by processing the second image mask to obtain the first image mask, the consistency of length and line width between the first and second graphics can be improved. Furthermore, by training the model using the first image generated based on the first image mask, the trained target model can achieve smoother processing of rendering aliasing.

[0163] According to one embodiment of the present application, the method for constructing the training sample also includes: adding a fourth graphic to the background image to obtain a fifth image mask; dividing the fifth image mask into multiple blocks, each of the multiple blocks including at least one adjacent pixel row or at least one adjacent pixel column; staggering two adjacent blocks by a specified pixel distance to form a sawtooth step between the two adjacent blocks to obtain a first image mask; performing regular pixel value filling on the pixels involved in the triangular area formed by the sawtooth step in the first image mask to obtain a second image mask, wherein the pixel value of the fourth graphic is represented as P1, the pixel value of the pixel after the regular pixel value filling is represented as P2, and the ratio of the area of ​​the pixel in the triangular area to the area of ​​the pixel is represented as R, P2=P1*R.

[0164] The size and color of the background image can be found in the description of the above embodiment. The process of adding the fourth graphic to the background image can be implemented through code, a network model, or manual means. The shape of the fourth graphic can be a straight line, other regular shapes, or irregular shapes, and the color of the fourth graphic can be a solid color or a gradient color.

[0165] In some embodiments, the fourth figure can be a vertical line or a horizontal line, so that the coordinates corresponding to the points on the fourth figure can all be integers, that is, the lines of the fourth figure can be smooth. The fifth image mask can be divided into multiple blocks in the form of pixel rows or pixel columns. For example, in the multiple blocks obtained by dividing the fifth image mask in the form of pixel rows, the number of pixel rows included in each block can be different or the same, such as random. By staggering two adjacent blocks along the direction of the pixel rows by a specified pixel distance, a serrated step can be formed between the two adjacent blocks, that is, the fourth figure is transformed into a figure with simulated distorted serrations (rendering serration type). The specified pixel distances staggered between different blocks can be the same or different. The image mask maskf obtained after staggering the blocks in the fifth image mask is as follows Figure 4M As shown. maskf can serve as the first image mask, where the background image corresponding to the first image mask is black and the first graphic is white lines. Similarly, the number of pixel columns included in each block obtained by dividing the fifth image mask by pixel columns can be different or the same, for example, random. By staggering two adjacent blocks by a specified pixel distance along the pixel columns, a sawtooth step can be formed between the two adjacent blocks, thereby obtaining maskf.

[0166] In other embodiments, the line corresponding to the fourth graphic may be a slant line or a curve. In order to make the line of the fourth graphic smooth, the above Figure 4I The fifth image mask in this embodiment is obtained by using the method of obtaining the second image mask based on the third image mask in the corresponding embodiment, that is, a graphic can be first added to the enlarged background image, and the enlarged background image after adding the graphic can be downsampled to obtain the fifth image mask.

[0167] right Figure 4M The pixel blocks half-surrounded by the sawtooth steps in the image mask maskf shown in the figure are filled with regular pixel values, and the following can be obtained: Figure 4N The second image mask is shown.

[0168] Specifically, Figure 4M The sawtooth steps formed between two adjacent blocks can be called a sawtooth unit structure, and the sawtooth unit structure can be represented by n x r, where n represents the number of pixels of the sawtooth unit structure in the column direction (the direction in which the lines extend), and r represents the number of pixels of the sawtooth unit structure in the row direction (the direction perpendicular to the direction in which the lines extend).

[0169] For example, Figure 4O A 4×3 sawtooth unit structure is shown. For ease of explanation, Figure 4O The 4×3 sawtooth unit structure is shown in gray pixels. The sawtooth unit structure (sawtooth steps) can form a triangular area, such as Figure 4O The pixels involved in the triangular area are the pixels in the 3×2 pixel block half-surrounded by the zigzag unit structure.

[0170] Connect the upper left vertex and the lower right vertex of the 3×2 pixel block into a line, and the line and the sawtooth step can form a triangular area. After regular pixel value filling, the pixel value (filling value) of each pixel in the 3×2 pixel block can be expressed as P2; the pixel value of the fourth figure, that is, Figure 4O The pixel value of a medium gray pixel is denoted as P1; the ratio of the area of ​​the pixel within the triangular region to the area of ​​the pixel, that is, the ratio of the area corresponding to the intersection of the pixel and the triangular region to the area of ​​the single pixel, is denoted as R. Regular pixel value filling can mean that the pixel value P2 after filling satisfies P2 = P1 * R.

[0171] The n and r of the zigzag unit structure can be set according to actual needs. Generally, r can be 2, 3, or other integers. Whether the zigzag unit structure is n×2, n×3, or other sizes, the method of filling the pixel block half-enclosed by the zigzag unit structure is the same.

[0172] The above process of regularly filling pixel values ​​in the pixel blocks half-enclosed by the nxr sawtooth unit structure can be considered as a process of filling non-zero pixels. By properly filling non-zero pixels, various microscopic sawtooth unit structures can be repaired and sawtooth steps can be smoothed.

[0173] Figure 4P Schematic diagrams of the sawtooth unit structures of nx2 and nx3 before and after repair are shown, where n = 1, 2, ..., m, and m = 20. When n is large enough, such as n>m, the sawtooth unit structure can be repaired according to n = m, and the sawtooth steps after repair are no longer obvious. It should be understood that the value of m here can be set according to actual needs and is not limited in this embodiment of the application.

[0174] Furthermore, any zigzag unit structure can be rotated 90 degrees left / right / up / down, as well as flipped left / right, resulting in seven structural variations. In other words, for any size of zigzag unit structure, eight specific representations are possible. Rendering aliasing in real-world scenarios can be characterized at the microscopic level using nxr zigzag unit structures.

[0175] In this embodiment, by dividing the fifth image mask into multiple blocks and staggering adjacent blocks by a specified pixel distance, simulated aliasing can be constructed pixel by pixel, thereby improving the fineness of the simulated aliasing of the first graphic in the first image mask. Furthermore, regular pixel value padding can improve the line smoothness of the second graphic. Furthermore, by training a model using the first image generated based on the first image mask, the trained target model can remove rendering aliasing while reducing damage to fine textures, such as blurring, thereby improving the target model's preservation of fine textures.

[0176] According to one embodiment of the present application, the method for constructing the training sample also includes: adding a fourth graphic to the background image to obtain a fifth image mask; dividing the fifth image mask into multiple blocks, each of the multiple blocks including at least one adjacent pixel row or at least one pixel column; staggering two adjacent blocks by a specified pixel distance to form a sawtooth step between the two adjacent blocks to obtain a sixth image mask; performing regular pixel value filling on the pixels involved in the triangular area formed by the sawtooth step in the sixth image mask to obtain a second image mask, wherein the pixel value of the fourth graphic is represented as P1, the pixel value of the pixel after regular pixel value filling is represented as P2, and the ratio of the area of ​​the pixel in the triangular area to the area of ​​the pixel is represented as R, P2=P1*R; performing random pixel value filling on the pixels involved in the triangular area formed by the sawtooth step in the sixth image mask to obtain the first image mask, or performing random pixel value filling on the pixels involved in the triangular area formed by the sawtooth step in the sixth image mask and the pixel neighborhood around the triangular area to obtain the first image mask.

[0177] This embodiment is similar to the previous embodiment. For the similarities, reference may be made to the description of the previous embodiment. The differences are described emphatically here.

[0178] In this embodiment, the image mask maskf obtained by staggering and adjusting the blocks in the fifth image mask is as follows: Figure 4M As shown. maskf can be recorded as the sixth image mask. The pixels involved in the triangular area formed by the sawtooth steps in the sixth image mask are randomly filled with pixel values ​​to obtain the first image mask. Alternatively, the pixels involved in the triangular area formed by the sawtooth steps in the sixth image mask and the pixel neighborhood surrounding the triangular area are randomly filled with pixel values ​​to obtain the first image mask. The pixel neighborhood surrounding the triangular area can be an area formed by expanding N pixels outward from the pixels involved in the triangular area, where N can be a positive integer.

[0179] Specifically, in actual application scenarios, after using some existing anti-aliasing algorithms to process rendering aliasing, there is a problem of incomplete aliasing. This problem can be manifested in that the pixel blocks half-enclosed by each aliasing unit structure, or even the neighborhood of the pixel block, may be filled with pixels, but not necessarily the ideal value.

[0180] To make the simulated distortion aliasing of the first graphic more consistent with the rendering aliasing in an actual scene, the pixel blocks half-enclosed by the zigzag unit structure of the graphic in maskf can be filled with random pixel values, i.e., non-ideal pixel values, to obtain the first image mask. The random pixel values ​​(i.e., non-ideal pixel values) can be randomly determined and can be in the range [0, 255]. Alternatively, the pixel blocks half-enclosed by the zigzag unit structure of the graphic in maskf and the pixel blocks in the neighborhood surrounding the pixel blocks can be filled with random pixel values ​​to obtain the first image mask.

[0181] The line width of the pattern in maskf after random pixel values ​​are filled can be closer to the line width of the second pattern in the second image mask, and the simulated distortion aliasing of the pattern in maskf after random pixel values ​​are filled can be more consistent with the rendering aliasing in actual scenes. Therefore, maskf after random pixel values ​​are filled can be used as the first image mask.

[0182] Optionally, the maskf filled with random pixel values ​​may be subjected to upsampling and downsampling to obtain a first image mask, such as Figure 4Q Through upsampling and downsampling, the fit between the simulated distortion aliasing of the first graphic and the real rendering aliasing can be further improved.

[0183] In this embodiment, by randomly filling pixel blocks half-surrounded by the zigzag unit structure of the graphic in maskf with pixel values, the simulated distortion jaggies of the first graphic can be made more consistent with the rendering jaggies in actual scenes.

[0184] It should be understood that according to the various methods for generating the first image mask and the second image mask provided above, multiple mask sets can be generated, and each mask set can correspond to a type of simulated distortion aliasing. Further, the lines in the image mask can form new set members by random warping (warping). In addition, a "union operation" can be performed on the image masks in multiple mask sets to form new set members. In some embodiments, the thickness of the white lines in the image mask can be random. Except for the pixels at the line edge and the background transition of the white lines, the other pixels can be 255. In some embodiments, a "union operation" can be performed on multiple maskL in the mask set to obtain a new set-type maskL; similarly, a "union operation" can be performed on maskH paired with multiple maskL to obtain a new set-type maskH. Multiple maskL can be part or all of the maskL in the mask set. In other embodiments, to increase the robustness of the model, in the step of performing a "union operation" on multiple masksL in the mask set, several masksL in the multiple masksL can be replaced with paired maskH, so that the final set maskL can be obtained.

[0185] Experiments have shown that in some cases, the end of the first graphic with simulated aliasing in a single maskL is a portion that is prone to poor network processing results. Through analysis, the inventors of this application believe that the reason for this problem is that the end of the first graphic, unlike other parts of the first graphic, is surrounded by more pixels representing simulated aliasing to provide reference for the network during processing. Therefore, the inventors of this application propose the following solution:

[0186] In some embodiments, when creating data (making an image mask), a line without simulated distortion serrations may be connected to the end of at least a portion of the first graphic in maskL, i.e., an extension line. The extension line may be a straight line or a curve. For example, the extension line may satisfy the following conditions: the extension line and the second graphic without simulated distortion serrations in maskH corresponding to maskL have the same width and curvature at the end port. It should be understood that if an extension line is added to maskL, the same extension line should also be added to maskH corresponding to maskL. In some cases, if maskL and maskH are of different sizes, the extension line in maskH should be as similar as possible to the extension line in maskL after being scaled to the same size as the extension line in maskL. Figure 4Q' Schematic diagrams showing a first image mask without extension lines (a) and a first image mask with extension lines (b) are shown. Figure 4Q" shows a schematic diagram of the second image mask with added extension lines. The introduction of extension lines can help guide the network to generate results without simulated distortion and aliasing in the direction of the extension lines when processing the end, and suppress possible undesirable effects.

[0187] In this embodiment, by adding extended lines without simulated distortion jaggies to maskL and maskH respectively, the robustness of the model can be increased.

[0188] According to an embodiment of the present application, step 310 may include: adding a first graphic and image high-frequency detail information to a base image to obtain a first image. Step 320 may include: adding a second graphic and image high-frequency detail information to the base image to obtain a second image, wherein the image high-frequency detail information is located at the same position in the second image as the image high-frequency detail information is located in the first image, and in the first image and the second image, the grayscale value change gradient of the region after the image high-frequency detail information is added is greater than or equal to a specified threshold.

[0189] High-frequency image detail information refers to information corresponding to regions within an image with large grayscale value variations, such as fine textures within the image. The grayscale value variation gradient of the region after the high-frequency image detail information is added is greater than or equal to a specified threshold. The specified threshold can be set as needed.

[0190] The high-frequency image detail information may include at least one of characters and patterns. For example, the high-frequency image detail information may include random text, random numbers, random English characters, or patterns such as grass, flowers, and sand. The format of the high-frequency image detail information may be configured as needed. For example, if the high-frequency image detail information is random text, the format of the random text may include at least one of the following: font, size, weight, color, and position arrangement.

[0191] For the first image, the process of adding the image high-frequency detail information can be performed before or after the process of adding the first graphic, or both can be performed simultaneously. Similarly, for the second image, the process of adding the image high-frequency detail information can be performed before or after the process of adding the second graphic, or both can be performed simultaneously.

[0192] Similar to the method of adding the first graphic / second graphic to the basic image, high-frequency detail information of the image can be directly added to the basic image, such as adding characters and / or patterns to the basic image, or characters and / or patterns can be added to the basic image by combining with an image mask.

[0193] Specifically, in some embodiments, high-frequency image detail information can be represented using an image mask. For example, if the high-frequency image detail information is random text, a random text mask can be pre-stored. The random text mask contains random text, and the size of the random text mask can be consistent with the size of the base image. By combining the random text mask with the base image, the random text can be embedded in the base image, thereby embedding the high-frequency image detail information.

[0194] For example, the random text mask can be represented as masktxt, the base image can be represented as H0, and the image obtained by adding random text to the base image can be represented as I1. The process of embedding random text in the base image can be achieved by combining the base image H0 with the random text mask masktxt. Specifically, it can be achieved by the following formula:

[0195] I1 = H0 * (E - masktxt / 255.0) + Atxt (10)

[0196] Atxt = colormap * masktxt / 255.0 (11)

[0197] In the above formulas (10) and (11), masktxt can represent the position of the random text, and colormap can represent the color of the random text. That is, Atxt can be the image obtained after adding color information to masktxt, and E is a matrix whose elements are all 1.

[0198] Specifically, the background image corresponding to masktxt can be black, and the pixel value of black can be 0. The color of the random text in masktxt can be white, and the pixel value corresponding to white can be 255. The first term H0*(E-masktxt / 255.0) in the above formula (10) can represent the partial area that needs to be subtracted from the base image, and the second term colormap*masktxt / 255.0 in the formula (10) can represent the random text with color that needs to be added to the subtracted partial area. It should be understood that the above formula is only an example. According to the different colors of the background image, as well as the different content and colors of the high-frequency detail information of the image, the coefficients of the various parameters in the above formula can be adjusted.

[0199] Figure 4R A masktxt is shown with a black background image and white random text. Figure 4S Shows Atxt after adding the set color information to masktxt. Figure 4T The image I1 obtained after random text is embedded in the base image is shown. Figure 4SIn the example, since the background image is black and the color of some random text is also black, compared with Figure 4R , Figure 4S Only some random text can be seen, and the black random text cannot be seen.

[0200] In some embodiments, the random text can be constructed by drawing random text in a target area of ​​a black background image using tools such as opencv or word. The position of the random text can be reflected by masktxt, and the color of the random text can be reflected by Atxt.

[0201] It should be understood that if random text is added to the image after the first graphic is added, H0 in the above formula (10) can be replaced with the image after the first graphic is added. Similarly, depending on the object to which the random text is added, H0 in formula (10) can be replaced with a different image.

[0202] In some embodiments, the font size of the random text can vary with the size of the simulated distortion aliasing of the first graphic. The smaller the size of the simulated distortion aliasing of the first graphic, such as the rendered aliasing, the smaller the lower limit of the random text font size can be, thereby improving the robustness of the model.

[0203] By adding high-frequency image detail information to the same areas of the base image during the creation of the first and second images, the model's negative impact on overall image quality can be reduced during subsequent training using the first and second images. In other words, the high-frequency image detail information in the second image serves as a reference, mitigating any negative impact caused by the model's training on the high-frequency image detail information in the first image. This reduces the damage to other image information caused by the model while removing aliasing artifacts, and reduces the risk of introducing additional distortion.

[0204] In this embodiment, by adding high-frequency image detail information to the basic image, the quality of the training samples can be improved, and then in the process of subsequently training the model using the training samples, the negative impact of the model on the overall quality of the image while removing distortion and jagged edges can be reduced.

[0205] According to one embodiment of the present application, the first image is an image that has undergone first image processing, and the first image processing includes at least one of noise addition processing, blur processing, encoding compression processing, image quality degradation processing, and downsampling processing, and / or the second image is an image that has undergone second image processing, and the second image processing includes sharpening processing.

[0206] By training the model using images that have undergone the first image processing, the model can be endowed with functions corresponding to the first image processing. For example, if the first image processing is noise addition, the model can have a denoising function; if the first image processing is blurring, the model can have a deblurring function; if the first image processing is encoding compression, the model can have a compression distortion removal function; if the first image processing is image degradation, the model can have a detail generation function; and if the first image processing is downsampling, the model can have a super-resolution function.

[0207] In some embodiments, a first image processing may be performed on a base image, and then a first graphic with simulated distortion aliasing may be added to the first-processed image to obtain a first image. That is, H0 in the above formula (1) may be replaced with the image after the first image processing. Alternatively, a first graphic with simulated distortion aliasing may be added to the base image, and then the first image processing may be performed on the image after the first graphic is added to obtain a first image. Alternatively, a portion of the multiple processing included in the first image processing may be performed on the base image, and then the first graphic with simulated distortion aliasing may be added to the partially processed image, and then the remaining processing may be performed on the image after the first graphic is added to obtain a first image. In these embodiments, a second graphic without simulated distortion aliasing may be added to the base image to obtain a second image. By using the first and second images as training samples to train a model, the resulting model may have the function of removing distortion aliasing and the function corresponding to the above-mentioned first image processing.

[0208] In some embodiments, the first image may be an image that has not undergone the first image processing, and the second image may be an image that has undergone the second image processing, where the second image processing includes sharpening. The model is trained using the first and second images as training samples, and the resulting model may have the function of removing distortion and aliasing as well as deblurring. The second image processing may be performed on the base image first, and then the second graphic may be added to the image that has undergone the second image processing. That is, H0 in the above formula (2) may be replaced with the image that has undergone the second image processing. Alternatively, the second graphic may be added to the base image first, and then the second image processing may be performed on the image after the second graphic is added.

[0209] It should be understood that in some embodiments, the first image may be an image that has undergone first image processing, and the second image may be an image that has undergone second image processing. By using the first and second images as training samples to train a model, the resulting model may have the ability to remove distortion and jagged edges, as well as functions corresponding to the first and second image processing. For example, the first image may be a blurred image, and the second image may be a sharpened image; thus, the model may have both the ability to remove distortion and jagged edges and the ability to deblur.

[0210] In some embodiments, the first image processing and the second image processing can be determined based on the functional information of the model. The functional information of the model can represent other functions of the model besides removing distortion and aliasing. The functional information of the model can be determined by the server based on user input or randomly determined by the server.

[0211] In some embodiments, the first image and the second image may include high-frequency image detail information. The process of adding high-frequency image detail information can be found in the description of the above embodiments. Specifically, based on the base image, the execution order of the first image processing, the addition of the first graphic, and the addition of the high-frequency image information can be set according to actual needs. Similarly, based on the base image, the execution order of the second image processing, the addition of the second graphic, and the addition of the high-frequency image information can be set according to actual needs.

[0212] In this embodiment, obtaining the first image through the first image processing and / or obtaining the second image through the second image processing can increase the richness of the content contained in the training sample and enhance the functionality of the target model trained based on the training sample. Furthermore, processing the image using the target model can reduce damage to the image quality while achieving the corresponding functionality of the target model.

[0213] In one embodiment, the function information of the model may include a function of removing dark scattered noise from rendering. That is, in this embodiment, the model may have the functions of removing aliasing and removing dark scattered noise from rendering.

[0214] Specifically, the method for constructing dark scattered noise is as follows: the image I2 to be noised is randomly divided into multiple blocks, resulting in a plurality of block regions; each pixel in each block region is assigned a probability value between 0 and 1, where the probability distribution can be a uniform distribution or a Gaussian distribution. This embodiment does not limit this, but recommends the former; if the probability value of each pixel is greater than a preset threshold value (dense), noise is added to that pixel position, where different densities can be used for different block regions. Here, the image I2 to be noised can be a base image, an image after the first graphic is added to the base image, an image after high-frequency image detail information is added to the base image, or an image after the first graphic and high-frequency image detail information are added to the base image.

[0215] Dense can be a real number between 0.01 and 0.09. Of course, the value of dense can be set according to actual conditions. This embodiment does not limit the value of dense.

[0216] According to the above-mentioned method for constructing dark scattered noise, the location information where noise needs to be added in the image I2 to be added can be determined. This location information can be represented by map_noise, where 0 represents no noise and 1 represents noise. The specific process of adding noise can be implemented by the following formula:

[0217] I3 = I2 * (E – map_noise) + noise * map_noise (12)

[0218] In the above formula (12), I3 can represent the image after adding noise, and E can represent a matrix whose elements are all 1. The process of generating noise can be to take a probability value between 0 and 1 for each pixel position in a pixel space as large as the base image. The pixel space after taking the value is recorded as noise_base, noise = noise_base * alpha_noise * 255.0. The sizes of noise, map_noise and I2 can be the same, and alpha_noise is a real number between 0 and 1. Because it is dark noise, alpha_noise can be set to 0.15. Of course, the value of alpha_noise can be set according to actual conditions, and this embodiment does not impose any restrictions on this. Figure 4U shows an image after adding rendering noise, Figure 4U The black dots are rendering noise.

[0219] Optionally, an embodiment of the present application further provides a method for constructing rendering noise, where the rendering noise may appear as dark pixel blocks specifically distributed near the distortion aliasing, which may also be referred to as rendering holes.

[0220] To construct this rendering noise, we can first mark the locations where rendering holes may appear with mask_hollow. Then, according to the method for constructing dark scattered noise provided above, we can create: mask_hollow*map_noise, recorded as map_noise_hollow. Map_noise_hollow can represent the location information of the image I2 to which rendering hole noise needs to be added.

[0221] According to the specific performance of actual rendering holes, the formula for adding rendering hole noise can be "multiplicative noise form":

[0222] I4 = I2 * (E – map_noise_hollow) + I2 * map_noise_hollow * rate_noise (13)

[0223] In the above formula (13), I4 can represent the image after adding the rendered hole noise, and E can represent a matrix whose elements are all 1. rate_noise can be the intensity of the multiplicative noise, or a probability value within a~b. For each pixel position in the pixel space as large as the base image, a probability value within a~b is taken once. The probability distribution can be a uniform distribution or a Gaussian distribution. The embodiment of the present application does not limit this, and the former is recommended. The pixel space after the value is taken is recorded as noise_base. Here, a and b can both be real numbers in the range of 0 to 1, for example, a is 0.2 and b is 0.4.

[0224] Optionally, the process of adding rendering hole noise can also be achieved by the following formula:

[0225] I4 = I2 * (E – map_noise_hollow) + noise * map_noise_hollow (14)

[0226] The process of determining noise in formula (14) can be found in the description of the above embodiment and will not be repeated here.

[0227] Figure 4V shows a schematic diagram representing the location of rendering holes, namely mask_hollow; Figure 4W A base image is shown; Figure 4X showing a second image; Figure 4Y A first image having rendering aliasing and rendering holes is shown.

[0228] In one embodiment, the functional information of the model may include a compression distortion removal function. That is, in this embodiment, the model may have the functions of removing distortion aliasing and removing compression distortion.

[0229] Specifically, the base image can be a frame from a video. Similar to adding noise, the video can first be coded and compressed to introduce video coding and compression distortion into the base image. The first graphic can then be embedded into the coded and compressed image to obtain the first image. Alternatively, the first graphic and high-frequency image detail information (such as random text) can be embedded into the coded and compressed image to obtain the first image.

[0230] In other embodiments, random text embedding can be performed before encoding and compression. For example, random text can be embedded in the video before encoding and compression. After random text embedding, the random text can change gradually between frames without abrupt changes, which helps the model learn to reduce compression artifacts. Figure 4ZA first image is shown. The first image may be an image obtained by first embedding gradual random text in a video and then encoding and compressing the video.

[0231] The present application also provides a model training method, such as Figure 5 shown. Figure 5 The method can be Figure 1 The training method of the model may include the following contents.

[0232] 510: Construct training samples.

[0233] Specifically, the training samples may be constructed according to the training sample construction method provided in the above embodiment.

[0234] 520: Use the training samples to train the initial model and obtain the target model.

[0235] The same or different initial models can be used for training different training samples. For example, an initial model of appropriate complexity can be selected for training based on the type of simulated distortion aliasing in the training sample and / or the functional information of the model input by the user. For details on the type of simulated distortion aliasing and the functional information of the model, please refer to the description of the above embodiments.

[0236] An embodiment of the present application provides a model training method. By training the model using training samples constructed using the above-mentioned training sample construction method, the model's effect in removing distortion and aliasing can be improved, and the impact of the model on the overall image quality while removing distortion and aliasing can be reduced.

[0237] According to one embodiment of the present application, the training method of the model also includes: obtaining distortion sawtooth type information and functional information of the model; constructing training samples based on the distortion sawtooth type information and functional information of the model, and obtaining an initial model based on the distortion sawtooth type information.

[0238] The aliasing type information and the model's functional information may be user-inputted. The aliasing type information may indicate a specific type of aliasing that can be removed by the target model; the model's functional information may indicate other functionalities of the target model besides aliasing removal. These other functionalities may include at least one of denoising, deblurring, compression distortion removal, detail generation, and super-resolution.

[0239] The specific process of constructing the training samples according to the distortion aliasing type information and the function information of the model can be found in the description of the above-mentioned training sample construction method.

[0240] In some embodiments, a server may pre-store multiple models with different functions. Based on the functional information of the model, a model that matches the functional information of the model may be selected from the multiple models as the initial model for training. The model that matches the functional information of the model may be a model that already has the corresponding function. In this way, by training the model with training samples, the model can have the function of removing distortion and aliasing without losing the original function of the model. Alternatively, the model that matches the functional information of the model may be a model that does not have the corresponding function. In this way, by training the model with training samples, the model can have the function of removing distortion and aliasing and the function of matching the functional information of the model.

[0241] In some embodiments, a server may pre-store multiple models with different structures. Based on the distortion aliasing type information, a model with an appropriate structural complexity may be selected from the multiple models as an initial model for training. Specifically, based on the distortion aliasing type information, the specific type of distortion aliasing may be determined, and further, the complexity of the specific type of distortion aliasing may be determined. Based on the complexity of the specific type of distortion aliasing, a model with a corresponding complexity may be selected as the initial model.

[0242] For example, when large-scale distortion aliasing needs to be removed, a relatively complex network model with a large receptive field is required as the initial model. For example, a network model with a Unet structure can be selected as the initial model to better balance performance and effect. Large-scale distortion aliasing can be distortion aliasing or de-interlacing residual aliasing, and distortion aliasing can be line distortion aliasing or distortion aliasing caused by up and down sampling. When the target model is required to have the function of detail generation, the complexity of the corresponding initial model needs to be greater than the complexity of the aforementioned network model. Therefore, a more complex version based on the above-mentioned Unet structure can be selected as the initial model, or other large network models can be selected as the initial model. When it is necessary to remove rendering aliasing (rendering noise and rendering holes can be added), a lightweight network model can be selected as the initial model.

[0243] In some embodiments, three network models of large, medium and small may be pre-stored on the server as a benchmark. According to the specific task and the performance of the user model, the intermediate versions of these three network models can be found, or the number of channels of these three network models can be adjusted, and the number of layers can be added or deleted as the initial model.

[0244] In some embodiments, a model with appropriate complexity may be selected as the initial model based on the distortion aliasing type information and the model's function information.

[0245] In this embodiment, obtaining an initial model based on the model's functional information enables the trained target model to have both antialiasing and other features. This antialiasing feature is well compatible with the other features. Furthermore, obtaining an initial model of appropriate complexity based on antialiasing type information can reduce model training costs.

[0246] According to an embodiment of the present application, a good effect of removing distortion and aliasing requires attention to the processing of image edges. Therefore, during the training process, the weight of the loss part corresponding to the edge area can be increased. For example, the idea of ​​focal loss can be borrowed to increase the power of the loss part corresponding to the edge area:

[0247] W=max{|Laplace(HR)|,|Laplace(UPSAMPLE(LR))|}

[0248]

[0249] Loss=|HR―netout| W

[0250] Laplace calculates the Laplace gradient of the image, UPSAMPLE upsamples the LR to the same size as the HR, max compares each element of the two and takes the larger one, and netout is the inference output during network model training. The LR can be the first image, and the HR can be the second image. UPSAMPLE is only required when the LR is smaller than the HR, meaning the network needs to perform super-resolution.

[0251] Of course, other methods may also be used to increase the weight of the loss portion corresponding to the edge area, and this embodiment of the present application does not limit this.

[0252] According to one embodiment of the present application, the training method of the model also includes: obtaining an output image obtained by processing an input image by a target model; using the input image to modify the distorted area in the output image to obtain a modified image, and the image quality of the distorted area is lower than the image quality of the area corresponding to the distorted area in the input image; using the modified image and the input image to perform secondary training on the target model to obtain a model after secondary training.

[0253] In actual use, the target model processes the input image to produce an output image. In some cases, removing aliasing artifacts from the input image may damage the quality of the input image itself, with the damaged areas appearing as distorted areas in the output image.

[0254] Based on the input image, the distorted regions in the output image can be modified to improve the image quality of the distorted regions, thereby obtaining a modified image. This modification can be done so that the image quality of the regions corresponding to the distorted regions in the modified image is as close as possible to that of the same regions in the input image. For example, the content of the distorted regions in the output image can be replaced with the content of the corresponding regions in the input image.

[0255] The distorted areas in the output image can be identified by a certain algorithm or model, or marked by manual annotation, so that the server / target terminal can identify the distorted areas based on the annotations.

[0256] The following describes the secondary training process of the target model by taking the server executing the model training method and sending the target model to the target terminal as an example.

[0257] Specifically, the user can choose on the target terminal's interface whether to upload input images that are poorly processed by the target model to the server. If the user chooses to upload input images that are poorly processed by the target model to the server, the user can circle the distorted areas in the image processed by the target model. The image containing the circled distorted areas becomes the output image. The input image can be represented as S_in, and the output image can be represented as S_out. In other words, the target terminal can upload both the input and output images to the server. The server can generate a map_merge with the same size as S_out. In map_merge, pixel positions corresponding to the distorted areas are assigned a value of 1, while all other positions are assigned a value of 0. Furthermore, the server can perform an appropriate Gaussian blur on map_merge. The input image S_in (equivalent to the first image) is used as the input of the target model, and S_out*(E – map_merge) + R(S_in)*map_merge is used as the comparison image (equivalent to the second image). The target model can be updated based on the input image S_in and the comparison image. Here, E is a matrix whose elements are all 1s; R() is a resize function. When S_in and S_out are not equal in size, R() resizes S_in to be equal in size to S_out. The specific rescaling method, i.e., the specific implementation of R(), can be a cubic, Lanczos, or pure super-resolution network, and is not limited in this embodiment.

[0258] In this embodiment, the modified image and the input image can be used as a set of training samples. The target model is trained twice using the modified image and the input image, which can further improve the performance of the target model and reduce the impact of the target model on the overall image quality while removing distortion and jagged edges.

[0259] The present application also provides a method for deploying a model. Figure 6 shown. Figure 6The method can be Figure 1 The target terminal in the deployment method of the model may include the following content.

[0260] 610: Send a model acquisition request to the server. The model acquisition request carries distortion aliasing type information. The distortion aliasing type information is used to characterize a specified type of distortion aliasing. The server is used to obtain a target model based on the distortion aliasing type information. The target model is used to remove the specified type of distortion aliasing.

[0261] The target terminal may send a model acquisition request to the server according to the user's instructions. The specific content of the distortion sawtooth type information may refer to the description in the above embodiment. A trained target model may be stored on the server, and the target model is used to remove the specified type of distortion sawtooth. Therefore, the server may find the target model based on the distortion sawtooth type information, thereby realizing the target model acquisition process. Alternatively, a plurality of training samples may be stored on the server, and each training sample may correspond to a type of distortion sawtooth. Therefore, the server may obtain the corresponding training sample according to the distortion sawtooth type information, and use the training sample to train the initial model to obtain the target model, thereby realizing the target model acquisition process. Alternatively, the server may construct a training sample according to the distortion sawtooth type information, and use the training sample to train the initial model to obtain the target model, thereby realizing the target model acquisition process.

[0262] The specific process of constructing training samples according to the distortion aliasing type information, and the process of training the initial model using the training samples, can be referred to the description in the above embodiment and will not be repeated here.

[0263] 620: Receive the target model sent by the server to deploy the target model on the target terminal.

[0264] The target terminal can receive the target model sent by the server and deploy the target model on the target terminal so that the target terminal can use the target model to provide services to users. This service can include removing jagged edges from images, meaning that the target model has the function of removing jagged edges. Furthermore, the target model can also have other functions, such as at least one of denoising, deblurring, compression distortion removal, detail generation, and super-resolution.

[0265] The present invention provides a model deployment method that sends a model acquisition request to a server and receives a target model from the server. This allows the target model to be deployed to the target terminal when it is needed, improving the user experience and reducing the storage burden on the target terminal. Furthermore, by receiving the target model from the server to deploy the target model, the target terminal can be exempted from the model training process, thereby reducing the performance requirements for the target terminal.

[0266] The present application also provides a method for deploying a model. Figure 7 shown. Figure 7 The method can be Figure 1 The server in the , and Figure 7 Methods and Figure 6 Therefore, the same parts can refer to each other. Figure 7 The deployment method of the illustrated model may include the following.

[0267] 710: Receive a model acquisition request sent by a target terminal, where the model acquisition request carries distortion sawtooth type information, and the distortion sawtooth type information is used to represent a specified type of distortion sawtooth.

[0268] For the specific content of the distortion sawtooth type information, reference may be made to the description in the above embodiment.

[0269] 720: Acquire a target model according to the distortion aliasing type information, wherein the target model is used to remove the distortion aliasing of a specified type.

[0270] For the specific process of the server obtaining the target model, please refer to the above Figure 6 Related description in the embodiments.

[0271] 730: Send the target model to the target terminal to deploy the target model on the target terminal.

[0272] The server sends the target model to the target terminal to implement the deployment of the target model. Specifically, the target terminal can automatically configure itself after receiving the target model and use the configured target model to provide services to users.

[0273] The present application provides a model deployment method that receives a model acquisition request from a target terminal and sends a target model to the target terminal. This allows for quick deployment of the target model to the target terminal when the target terminal needs it, improving the user experience and reducing the storage burden on the target terminal. Furthermore, by deploying the target model by sending the target model to the target terminal, the target model can be deployed on multiple target terminals, thereby enabling resource sharing and conserving training resources.

[0274] According to one embodiment of the present application, step 720 may include: generating a training sample based on the distortion aliasing type information, wherein the training sample includes a first image having a first graphic compared to the base image, and a second image having a second graphic compared to the base image, the first graphic having simulated distortion aliasing, the simulated distortion aliasing of the first graphic being of a specified type, the position of the second graphic in the second image corresponding to the position of the first graphic in the first image, and the difference between the shape of the second graphic and the shape of the first graphic including that the second graphic does not have simulated distortion aliasing; using the training sample to train the initial model to obtain the target model.

[0275] The specific process of generating training samples based on the distortion aliasing type information can be found in the description of the training sample construction method above. The process of training the initial model using the training samples can be found in the description of the model training method above.

[0276] In this embodiment, the server can generate training samples in real time according to the distortion sawtooth type information, and use the training samples to train the initial model to obtain the target model, thereby meeting the diverse needs of users.

[0277] According to an embodiment of the present application, step 720 may include: obtaining a target model from a specified model set according to the distortion aliasing type information, where the specified model set includes multiple models, and the multiple models are used to remove different types of distortion aliasing.

[0278] The server may pre-store a specified model set, which may include multiple trained models. Each model can be used to remove one or more types of distortion artifacts. Different models can remove completely different or partially identical types of distortion artifacts. Based on the distortion artifact type information, the server can retrieve a target model from the specified model set. The target model can be used to remove the specified type of distortion artifacts.

[0279] In this embodiment, the server may store multiple trained models. When receiving a model acquisition request sent by a target terminal, the server may respond quickly, shortening the waiting time of the target terminal, thereby improving the user experience.

[0280] According to one embodiment of the present application, the deployment method of the model also includes: receiving a first feedback image sent by the target terminal and a second feedback image obtained by processing the first feedback image by the target model; using the first feedback image to modify the distorted area in the second feedback image to obtain a modified image, and the image quality of the distorted area is lower than the image quality of the area corresponding to the distorted area in the first feedback image; using the modified image and the first feedback image to perform secondary training on the target model to obtain a model after secondary training.

[0281] The first feedback image in this embodiment is equivalent to the input image mentioned in the above-mentioned model training method; the second feedback image in this embodiment is equivalent to the output image mentioned in the above-mentioned model training method. Therefore, the specific process of secondary training the target model in this embodiment can be found in the description of the above-mentioned model training method. Furthermore, the server can send the secondary trained model to the target terminal for use by the target terminal.

[0282] In this embodiment, the modified image and the first feedback image are used as a set of training samples, which can improve the quality of the training samples. Furthermore, the modified image and the first feedback image are used to perform a second training on the target model, which can increase the total number of training samples and thus improve the performance of the target model.

[0283] Exemplary devices

[0284] Figure 8 FIG. 8 is a schematic diagram of a structure of a training sample construction device 800 provided by an exemplary embodiment of the present application. Figure 8 As shown, the training sample construction device 800 includes: a first acquisition module 810 and a second acquisition module 820.

[0285] The first acquisition module 810 is used to acquire a first image having a first graphic compared to the base image, wherein the first graphic has simulated distortion aliasing; the second acquisition module 820 is used to acquire a second image having a second graphic compared to the base image, wherein the first image and the second image serve as training samples, the position of the second graphic in the second image corresponds to the position of the first graphic in the first image, and the difference between the shape of the second graphic and the shape of the first graphic includes that the second graphic does not have simulated distortion aliasing.

[0286] An embodiment of the present application provides a device for constructing a training sample. By obtaining a first image having a first graphic compared to a base image, and obtaining a second image having a second graphic compared to the base image, the first image and the second image can be used as training samples, wherein the first graphic has simulated distortion aliasing and the second graphic does not have simulated distortion aliasing. Since the first graphic can be located in a specified area in the first image and the second graphic can be located in a specified area in the second image, the damage to the image quality of other areas in the training sample caused by the process of constructing the simulated distortion aliasing can be reduced, and the quality of the training sample can be improved, that is, a targeted training sample is obtained. Using the training sample to train the model can improve the effect of the model in removing distortion aliasing and reduce the impact of the model on the overall image quality while removing distortion aliasing.

[0287] According to one embodiment of the present application, the training samples are used to train a target model, which is used to remove a specified type of distortion aliasing. The first acquisition module 810 is further configured to: select a target graphic set from multiple graphic sets based on information indicating the specified type of distortion aliasing; and determine a first graphic from the target graphic set, wherein the simulated distortion aliasing of the first graphic belongs to the specified type.

[0288] According to one embodiment of the present application, the first acquisition module 810 is configured to add a first graphic and high-frequency image detail information to a base image to obtain a first image; and the second acquisition module 820 is configured to add a second graphic and high-frequency image detail information to the base image to obtain a second image. The high-frequency image detail information is located in the second image at the same position as the high-frequency image detail information was located in the first image, and the grayscale value change gradient of the region after the high-frequency image detail information is added in the first and second images is greater than or equal to a specified threshold.

[0289] According to an embodiment of the present application, the high-frequency detail information of the image includes at least one of characters and patterns.

[0290] According to one embodiment of the present application, the first acquisition module 810 is used to combine the basic image with the first image mask representing the first graphic to embed the first graphic in the basic image to obtain the first image; the second acquisition module 820 is used to combine the basic image with the second image mask representing the second graphic to embed the second graphic in the basic image to obtain the second image, wherein the size of the second image mask is the same as the size of the first image mask.

[0291] According to one embodiment of the present application, the second acquisition module 820 is further configured to add a second graphic to the background image to obtain a second image mask. The first acquisition module 810 is further configured to divide the second graphic in the second image mask into a plurality of segments, and rotate at least one of two adjacent segments in the plurality of segments by a specified angle about the geometric center of the segment to obtain the first image mask.

[0292] According to one embodiment of the present application, the second acquisition module 820 is further configured to: add a second graphic to the background image to obtain a second image mask. The first acquisition module 810 is further configured to: downsample and upsample the second image mask to obtain an intermediate image mask, the intermediate image mask having the same size as the second image mask; and adjust the pixel values ​​of the intermediate image mask according to specified pixel value adjustment requirements to obtain the first image mask.

[0293] According to an embodiment of the present application, the second acquisition module 820 is further configured to: add a second graphic to the background image to obtain a second image mask. The first acquisition module 810 is further configured to: divide the second image mask into a plurality of blocks, each of the plurality of blocks comprising a plurality of adjacent pixel rows, the plurality of pixel rows comprising an upper portion of pixel rows and a lower portion of pixel rows; and interchange the lower portion of pixel rows with the upper portion of pixel rows to obtain the first image mask.

[0294] According to one embodiment of the present application, the first acquisition module 810 is further configured to: add the first graphic to the background image to obtain a first image mask. The second acquisition module 820 is further configured to: add a third graphic to the enlarged background image to obtain a third image mask, wherein the third graphic is an enlarged graphic of the first graphic; and downsample the third image mask to obtain a second image mask.

[0295] According to one embodiment of the present application, the second acquisition module 820 is further configured to: add a second graphic to the background image to obtain a second image mask. The first acquisition module 810 is further configured to: divide the second graphic in the second image mask into a plurality of segments, rotate at least one of two adjacent segments in the plurality of segments by a specified angle about the geometric center of the segments to obtain a fourth image mask; and binarize the pixel values ​​of the pixels in the fourth image mask to obtain the first image mask.

[0296] According to one embodiment of the present application, the first acquisition module 810 is further configured to: add a fourth graphic to the background image to obtain a fifth image mask; divide the fifth image mask into multiple blocks, each of the multiple blocks including at least one adjacent pixel row or at least one adjacent pixel column; and stagger two adjacent blocks by a specified pixel distance to form a sawtooth step between the two adjacent blocks to obtain the first image mask. The second acquisition module 820 is further configured to: perform regular pixel value filling on the pixels involved in the triangular area formed by the sawtooth step in the first image mask to obtain a second image mask, wherein the pixel value of the fourth graphic is represented as P1, the pixel value of the pixel after the regular pixel value filling is represented as P2, and the ratio of the area of ​​the pixel in the triangular area to the area of ​​the pixel is represented as R, where P2 = P1*R.

[0297] According to one embodiment of the present application, the second acquisition module 820 is further configured to: add a fourth graphic to the background image to obtain a fifth image mask; divide the fifth image mask into a plurality of blocks, each of the plurality of blocks comprising at least one adjacent pixel row or at least one adjacent pixel column; stagger two adjacent blocks by a specified pixel distance to form a sawtooth step between the two adjacent blocks to obtain a sixth image mask; and perform regular pixel value filling on the pixels involved in the triangular area formed by the sawtooth step in the sixth image mask to obtain a second image mask, wherein the pixel value of the fourth graphic is represented as P1, the pixel value of the pixel after the regular pixel value filling is represented as P2, and the ratio of the area of ​​the pixel in the triangular area to the area of ​​the pixel is represented as R, where P2 = P1 * R. The first acquisition module 810 is further configured to: perform random pixel value filling on the pixels involved in the triangular area formed by the sawtooth step in the sixth image mask to obtain the first image mask, or perform random pixel value filling on the pixels involved in the triangular area formed by the sawtooth step and the pixel neighborhood surrounding the triangular area in the sixth image mask to obtain the first image mask.

[0298] It should be understood that the operations and functions of the first acquisition module 810 and the second acquisition module 820 in the above embodiment can refer to the above Figure 3 The description of the method for constructing training samples provided in the embodiment will not be repeated here to avoid repetition.

[0299] Figure 9 FIG. 9 is a schematic diagram of a model training device 900 provided by an exemplary embodiment of the present application. Figure 9 As shown, the model training device 900 includes: a construction module 910 and a training module 920.

[0300] The construction module 910 is used to construct a training sample according to the training sample construction method provided in the above embodiment; the training module 920 is used to train the initial model using the training sample to obtain the target model.

[0301] An embodiment of the present application provides a model training device. By training the model using training samples constructed using the above-mentioned training sample construction method, the model's effect in removing distortion and aliasing can be improved, and the impact of the model on the overall image quality while removing distortion and aliasing can be reduced.

[0302] It should be understood that the operations and functions of the construction module 910 and the training module 920 in the above embodiment can refer to the above Figure 5 To avoid repetition, the description of the training method of the model provided in the embodiment will not be repeated here.

[0303] Figure 10 FIG. 1 is a schematic diagram of a model deployment device 1000 provided by an exemplary embodiment of the present application. Figure 10 As shown, the model deployment device 1000 includes: a sending module 1010 and a receiving module 1020.

[0304] The sending module 1010 is used to send a model acquisition request to the server, where the model acquisition request carries distortion sawtooth type information, where the distortion sawtooth type information is used to characterize a specified type of distortion sawtooth. The server is used to obtain a target model based on the distortion sawtooth type information, where the target model is used to remove the specified type of distortion sawtooth. The receiving module 1020 is used to receive the target model sent by the server so as to deploy the target model on the target terminal.

[0305] Embodiments of the present application provide a model deployment device that, by sending a model acquisition request to a server, receives a target model from the server. This allows the target model to be deployed to the target terminal when it is needed, improving the user experience and reducing the storage burden on the target terminal. Furthermore, by receiving the target model from the server to deploy the target model, the target terminal eliminates the need to train the model, thereby reducing the performance requirements for the target terminal.

[0306] It should be understood that the operations and functions of the sending module 1010 and the receiving module 1020 in the above embodiment can refer to the above Figure 6 To avoid repetition, the description of the deployment method of the model provided in the embodiment will not be repeated here.

[0307] Figure 11 FIG. 1 is a schematic diagram of a model deployment device 1100 provided by another exemplary embodiment of the present application. Figure 11 As shown, the model deployment device 1100 includes: a receiving module 1110 , an acquisition module 1120 and a sending module 1130 .

[0308] The receiving module 1110 is used to receive a model acquisition request sent by the target terminal, where the model acquisition request carries distortion sawtooth type information, where the distortion sawtooth type information is used to characterize a specified type of distortion sawtooth; the acquisition module 1120 is used to obtain a target model based on the distortion sawtooth type information, where the target model is used to remove the specified type of distortion sawtooth; the sending module 1130 is used to send the target model to the target terminal to deploy the target model on the target terminal.

[0309] The present invention provides a model deployment device that receives a model acquisition request from a target terminal and sends a target model to the target terminal. This allows the target model to be deployed to the target terminal when the target terminal needs it, improving the user experience and reducing the storage burden on the target terminal. Furthermore, by deploying the target model by sending the target model to the target terminal, the target model can be deployed on multiple target terminals, thereby enabling resource sharing and saving training resources.

[0310] According to one embodiment of the present application, the acquisition module 1120 is used to: generate training samples based on distortion aliasing type information, wherein the training samples include a first image having a first graphic compared to a base image, and a second image having a second graphic compared to the base image, the first graphic having simulated distortion aliasing, the simulated distortion aliasing of the first graphic being of a specified type, the position of the second graphic in the second image corresponding to the position of the first graphic in the first image, and the difference between the shape of the second graphic and the shape of the first graphic including that the second graphic does not have simulated distortion aliasing; and use the training samples to train the initial model to obtain a target model.

[0311] According to an embodiment of the present application, the acquisition module 1120 is used to: acquire a target model from a specified model set according to the distortion aliasing type information, where the specified model set includes multiple models, and the multiple models are used to remove different types of distortion aliasing.

[0312] According to an embodiment of the present application, the receiving module 1110 is further configured to: receive a first feedback image sent by a target terminal and a second feedback image obtained by processing the first feedback image by a target model. The acquiring module 1120 is further configured to: modify a distorted region in the second feedback image using the first feedback image to obtain a modified image, wherein the image quality of the distorted region is lower than the image quality of the region corresponding to the distorted region in the first feedback image; and perform a second training on the target model using the modified image and the first feedback image to obtain a second-trained model.

[0313] It should be understood that the operations and functions of the receiving module 1110, the acquiring module 1120 and the sending module 1130 in the above embodiment can refer to the above Figure 7 To avoid repetition, the description of the deployment method of the model provided in the embodiment will not be repeated here.

[0314] Figure 12 Shown is a block diagram of an electronic device 1200 for a method for constructing training samples, a method for training a model, or a method for deploying a model, provided by an exemplary embodiment of the present application.

[0315] Reference Figure 12The electronic device 1200 includes a processing component 1210, which further includes one or more processors and a memory resource represented by a memory 1220 for storing instructions executable by the processing component 1210, such as an application. The application stored in the memory 1220 may include one or more modules, each corresponding to a set of instructions. In addition, the processing component 1210 is configured to execute the instructions to perform the above-mentioned training sample construction method, model training method, or model deployment method.

[0316] The electronic device 1200 may further include a power supply component configured to perform power management of the electronic device 1200, a wired or wireless network interface configured to connect the electronic device 1200 to a network, and an input / output (I / O) interface. The electronic device 1200 may be operated based on an operating system stored in the memory 1220, such as Windows Server 200. TM , MacOS X TM , Unix TM , Linux TM , FreeBSD TM or similar.

[0317] A non-temporary computer-readable storage medium, when the instructions in the storage medium are executed by the processor of the above-mentioned electronic device 1200, enables the above-mentioned electronic device 1200 to execute a method for constructing training samples, a method for training models, or a method for deploying models.

[0318] All of the above optional technical solutions can be arbitrarily combined to form optional embodiments of the present application, and will not be described in detail here.

[0319] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0320] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0321] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0322] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0323] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0324] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program check codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0325] It should be noted that, in the description of this application, the terms "first," "second," "third," etc. are used for descriptive purposes only and should not be understood as indicating or implying relative importance. In addition, in the description of this application, unless otherwise specified, "plurality" means two or more.

[0326] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0327] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent replacements, etc. made within the spirit and principles of the present application should be included in the scope of protection of the present application.

Claims

1. A method for constructing training samples. It is characterized in that include: Acquire a first image having a first pattern compared to a base image, wherein the first pattern has simulated distortion aliasing; Acquire a second image having a second graphic compared to the base image, wherein the first image and the second image serve as the training samples, a position of the second graphic in the second image corresponds to a position of the first graphic in the first image, and a difference between a shape of the second graphic and a shape of the first graphic includes that the second graphic does not have simulated distortion aliasing.

2. The method for constructing a training sample according to claim 1, It is characterized in that The training samples are used to train a target model, and the target model is used to remove distortion jaggedness of a specified type. The method further includes: Selecting a target graphic set from a plurality of graphic sets according to the distortion sawtooth type information indicating the specified type; The first graphic is determined from the target graphic set, wherein the simulated distortion aliasing of the first graphic belongs to the specified type.

3. The method for constructing a training sample according to claim 1 or 2, It is characterized in that The step of acquiring a first image having a first graphic compared to the base image includes: The first graphic and the image high-frequency detail information are added to the base image to obtain the first image, wherein: The acquiring of a second image having a second graphic compared to the base image comprises: The second graphic and the image high-frequency detail information are added to the basic image to obtain the second image, wherein the image high-frequency detail information is located at the same position in the second image as the image high-frequency detail information is located in the first image, and in the first image and the second image, the grayscale value change gradient of the area after the image high-frequency detail information is added is greater than or equal to a specified threshold.

4. The method for constructing a training sample according to claim 3, It is characterized in that The image high-frequency detail information includes at least one of characters and patterns.

5. The method for constructing a training sample according to claim 1, It is characterized in that The step of acquiring a first image having a first graphic compared to the base image includes: The base image is combined with a first image mask representing the first graphic to embed the first graphic in the base image to obtain the first image, wherein: The acquiring of a second image having a second graphic compared to the base image comprises: The base image is combined with a second image mask representing the second graphic to embed the second graphic in the base image to obtain the second image, wherein a size of the second image mask is the same as a size of the first image mask.

6. The method for constructing a training sample according to claim 5, It is characterized in that The method further comprises: Adding the second graphic to the background image to obtain the second image mask; The second figure in the second image mask is divided into a plurality of segments, and at least one segment of two adjacent segments in the plurality of segments is rotated by a specified angle according to the geometric center of the segment to obtain the first image mask, Alternatively, the method further comprises: Adding the second graphic to the background image to obtain the second image mask; Performing downsampling and upsampling on the second image mask to obtain an intermediate image mask, wherein the size of the intermediate image mask is the same as the size of the second image mask; The pixel values ​​of the intermediate image mask are adjusted according to the specified pixel value adjustment requirement to obtain the first image mask, Alternatively, the method further comprises: Adding the second graphic to the background image to obtain the second image mask; Dividing the second image mask into a plurality of blocks, each of the plurality of blocks comprising a plurality of adjacent pixel rows, the plurality of pixel rows comprising an upper portion of pixel rows and a lower portion of pixel rows; The lower pixel rows are exchanged with the upper pixel rows to obtain the first image mask, Alternatively, the method further comprises: Adding the first graphic to the background image to obtain the first image mask; Adding a third graphic to the enlarged background image to obtain a third image mask, wherein the third graphic is an enlarged graphic of the first graphic; Downsampling the third image mask to obtain the second image mask, Alternatively, the method further comprises: Adding the second graphic to the background image to obtain the second image mask; Dividing the second graphic in the second image mask into a plurality of segments, and rotating at least one segment of two adjacent segments in the plurality of segments by a specified angle according to the geometric center of the segment, to obtain a fourth image mask; Binarizing the pixel values ​​of the pixels in the fourth image mask to obtain the first image mask, Alternatively, the method further comprises: Adding a fourth graphic to the background image to obtain a fifth image mask; Dividing the fifth image mask into a plurality of blocks, each of the plurality of blocks comprising at least one adjacent pixel row or at least one adjacent pixel column; staggering two adjacent blocks by a specified pixel distance to form a sawtooth step between the two adjacent blocks, thereby obtaining the first image mask; Regular pixel value filling is performed on the pixels involved in the triangular area formed by the sawtooth step in the first image mask to obtain the second image mask, wherein the pixel value of the fourth figure is represented by P1, the pixel value of the pixel after the regular pixel value filling is represented by P2, and the ratio of the area of ​​the pixel located in the triangular area to the area of ​​the pixel is represented by R, P2=P1*R, Alternatively, the method further comprises: Adding a fourth graphic to the background image to obtain a fifth image mask; Dividing the fifth image mask into a plurality of blocks, each of the plurality of blocks comprising at least one adjacent pixel row or at least one adjacent pixel column; staggering two adjacent blocks by a specified pixel distance to form a sawtooth step between the two adjacent blocks, thereby obtaining a sixth image mask; Performing regular pixel value filling on pixels involved in the triangular area formed by the sawtooth step in the sixth image mask to obtain the second image mask, wherein the pixel value of the fourth figure is represented by P1, the pixel value of the pixel after the regular pixel value filling is represented by P2, and the ratio of the area of ​​the pixel located in the triangular area to the area of ​​the pixel is represented by R, P2=P1*R; The pixels involved in the triangular area formed by the sawtooth steps in the sixth image mask are randomly filled with pixel values ​​to obtain the first image mask, or the pixels involved in the triangular area formed by the sawtooth steps in the sixth image mask and the pixel neighborhood around the triangular area are randomly filled with pixel values ​​to obtain the first image mask.

7. A model training method, It is characterized in that include: Constructing a training sample according to the method for constructing a training sample according to any one of claims 1 to 6; The training samples are used to train an initial model to obtain a target model.

8. A method for deploying a model, It is characterized in that include: Sending a model acquisition request to a server, the model acquisition request carrying distortion sawtooth type information, the distortion sawtooth type information being used to characterize a specified type of distortion sawtooth, the server being used to acquire a target model according to the distortion sawtooth type information, the target model being used to remove the specified type of distortion sawtooth; The target model sent by the server is received to deploy the target model on a target terminal.

9. A model deployment method, It is characterized in that include: Receiving a model acquisition request sent by a target terminal, wherein the model acquisition request carries distortion sawtooth type information, and the distortion sawtooth type information is used to characterize a specified type of distortion sawtooth; Acquire a target model according to the distortion aliasing type information, wherein the target model is used to remove the distortion aliasing of the specified type; The target model is sent to the target terminal to deploy the target model on the target terminal.

10. The deployment method of the model according to claim 9, It is characterized in that The acquiring the target model according to the distortion sawtooth type information includes: Generate a training sample according to the distortion aliasing type information, wherein the training sample includes a first image having a first graphic compared to a base image, and a second image having a second graphic compared to the base image, the first graphic having simulated distortion aliasing, the simulated distortion aliasing of the first graphic being of the specified type, the position of the second graphic in the second image corresponding to the position of the first graphic in the first image, and the difference between the shape of the second graphic and the shape of the first graphic includes that the second graphic does not have the simulated distortion aliasing; The initial model is trained using the training samples to obtain the target model.

11. The deployment method of the model according to claim 9, It is characterized in that The acquiring the target model according to the distortion sawtooth type information includes: The target model is obtained from a specified model set according to the distortion aliasing type information, where the specified model set includes a plurality of models, and the plurality of models are used to remove different types of distortion aliasing.

12. A method for deploying a model according to any one of claims 9 to 11, It is characterized in that Also includes: receiving a first feedback image sent by the target terminal and a second feedback image obtained by processing the first feedback image by the target model; Using the first feedback image to modify the distorted region in the second feedback image to obtain a modified image, wherein the image quality of the distorted region is lower than the image quality of the region corresponding to the distorted region in the first feedback image; The target model is trained twice using the modified image and the first feedback image to obtain a twice-trained model.

13. An electronic device, It is characterized in that include: processor; a memory for storing instructions executable by the processor, The processor is used to execute the training sample construction method described in any one of claims 1 to 6, the model training method described in claim 7, or the model deployment method described in any one of claims 8 to 12.

14. A computer-readable storage medium, It is characterized in that The storage medium stores a computer program, which is used to execute the method for constructing a training sample as described in any one of claims 1 to 6, the method for training a model as described in claim 7, or the method for deploying a model as described in any one of claims 8 to 12.

Images