Image processing methods, apparatuses, electronic devices, storage media, and software products

Abstract

This application discloses an image processing method, apparatus, electronic device, storage medium, and program product, relating to the field of computer technology. The method includes: acquiring multiple sample images; segmenting the sample images using an image segmentation model to obtain a first image containing a virtual character and / or a second image containing the virtual character and an interactive virtual object, wherein the interactive virtual object interacts with the virtual character; reconstructing the first image and / or the second image to obtain a reconstructed image; segmenting the reconstructed image using the image segmentation model to obtain a third image and / or a fourth image; training the image segmentation model based on the differences between the first and third images and / or the differences between the second and fourth images; acquiring an image to be detected, segmenting it using the trained image segmentation model to obtain a segmentation result, and then inputting the result into a classification model to obtain a clipping classification result. By implementing the technical solution of this application, automated detection of clipping problems in virtual character rendering can be achieved.

Description

Technical Field

[0001] This application relates to the field of computer technology, specifically to image processing methods, apparatus, electronic devices, storage media, and program products. Background Technology

[0002] In the fields of digital content such as games and virtual reality, the visual rendering quality of virtual objects directly affects the realism and immersion of the user experience. With the frequent updates and enrichment of content such as characters, skins, and actions, the interactions between virtual objects and between them and the scene are becoming increasingly complex. The resulting rendering clipping problems—such as unreasonable visual penetration and overlap between characters and props, and between scenes—have become a significant defect affecting visual performance.

[0003] Currently, the detection of clipping issues in rendering mainly relies on manual visual inspection, requiring a large number of people to verify each action and interaction scenario one by one. This method is not only inefficient and costly in terms of manpower, but also prone to missed detections. Summary of the Invention

[0004] In view of this, this application provides an image processing method, apparatus, electronic device, storage medium, and program product to solve the problem of low efficiency in rendering clipping detection.

[0005] In a first aspect, this application provides an image processing method, comprising: acquiring multiple first sample images; segmenting any first sample image using an initial image segmentation model to obtain a first image containing a virtual character, and / or a second image containing a virtual character and an interactive virtual object, wherein there is interaction between the interactive virtual object and the virtual character; performing image reconstruction processing on the first image and / or the second image to obtain a reconstructed image; segmenting the reconstructed image using the initial image segmentation model to obtain a third image containing a virtual character, and / or a fourth image containing a virtual character and an interactive virtual object; training the initial image segmentation model based on a first difference between the first image and the third image, and / or a second difference between the second image and the fourth image, to obtain a target image segmentation model; acquiring an image to be detected, segmenting the image to be detected using the target image segmentation model to obtain a target segmentation result; and inputting the target segmentation result into a target classification model to obtain a clipping classification result for the image to be detected.

[0006] In one optional implementation, the first sample image is segmented using an initial image segmentation model to obtain a first image containing a virtual character, and / or a second image containing a virtual character and an interactive virtual object. This includes: inputting a first prompt word and the first sample image into the initial image segmentation model to obtain the first image, wherein the first prompt word is used to instruct the initial image segmentation model to segment the virtual character; and / or, inputting a second prompt word and the first sample image into the initial image segmentation model to obtain the second image, wherein the second prompt word is used to instruct the initial image segmentation model to segment the virtual character and the interactive virtual object.

[0007] In one optional implementation, image reconstruction processing is performed on the first image and / or the second image to obtain a reconstructed image, including: obtaining a background image corresponding to the first sample image; and scrambling the positions of the first image and / or the second image in the background image to obtain the reconstructed image.

[0008] In one optional implementation, the reconstructed image is segmented using an initial image segmentation model to obtain a third image containing a virtual character, and / or a fourth image containing a virtual character and interactive virtual objects. This includes: inputting a third prompt word and the reconstructed image into the initial image segmentation model to obtain a fifth image corresponding to each virtual object in the reconstructed image, wherein the third prompt word is used to instruct the initial image segmentation model to segment each virtual object in the reconstructed image; performing similarity matching between each fifth image and a first image, and determining a third image that matches the first image among multiple fifth images based on each first similarity matching result; and / or performing similarity matching between each fifth image and a second image, and determining a fourth image that matches the second image among multiple fifth images based on each second similarity matching result.

[0009] In one optional implementation, an initial image segmentation model is trained based on a first difference between the first image and the third image, and / or a second difference between the second image and the fourth image, to obtain a target image segmentation model. This includes: determining a first pixel difference between corresponding pixels in the first image and the third image, and / or determining a second pixel difference between corresponding pixels in the second image and the fourth image; and adjusting the parameters of the initial image segmentation model based on the first pixel difference and / or the second pixel difference to obtain the target image segmentation model.

[0010] In one optional implementation, the first sample image is segmented using an initial image segmentation model to obtain a first image containing a virtual character, and / or a second image containing the virtual character and interactive virtual objects. The method further includes: obtaining the location information of the region of interest in the first sample image, wherein the region of interest is associated with the virtual character; inputting the first sample image and the location information of the region of interest into the initial image segmentation model to obtain the first image and / or the second image, wherein the location information of the region of interest is used to guide the initial image segmentation model to segment the virtual character.

[0011] In one alternative implementation, the target classification model is trained based on multiple sample segmentation results obtained by segmenting multiple second sample images using a target image segmentation model, and the labeled rendered pattern sample.

[0012] Secondly, this application provides an image processing apparatus, comprising: an acquisition module for acquiring a plurality of first sample images; a first segmentation module for segmenting any one of the first sample images using an initial image segmentation model to obtain a first image containing a virtual character, and / or a second image containing a virtual character and an interactive virtual object, wherein there is interaction between the interactive virtual object and the virtual character; a reconstruction module for performing image reconstruction processing on the first image and / or the second image to obtain a reconstructed image; a second segmentation module for segmenting the reconstructed image using the initial image segmentation model to obtain a third image containing a virtual character, and / or a fourth image containing a virtual character and an interactive virtual object; a training module for training the initial image segmentation model based on a first difference between the first image and the third image, and / or a second difference between the second image and the fourth image, to obtain a target image segmentation model; a third segmentation module for acquiring an image to be detected and segmenting the image to be detected using the target image segmentation model to obtain a target segmentation result; and a classification module for inputting the target segmentation result into a target classification model to obtain a clipping classification result for the image to be detected.

[0013] Thirdly, this application provides an electronic device, including: a memory and a processor, which are communicatively connected to each other. The memory stores computer instructions, and the processor executes the computer instructions to perform the image processing method described in the first aspect or any corresponding embodiment.

[0014] Fourthly, this application provides a computer-readable storage medium storing computer instructions for causing a computer to perform the image processing method described in the first aspect or any corresponding embodiment thereof.

[0015] Fifthly, this application provides a computer program product, including computer instructions for causing a computer to execute the image processing method described in the first aspect or any corresponding embodiment thereof.

[0016] The image processing method provided in this application segmentes a first sample image using an initial image segmentation model, extracting a first image containing a virtual character and / or a second image containing the virtual character and an interactive object, thereby achieving dual attention to both character clipping and interaction clipping. By reconstructing these segmentation results and inputting the reconstructed images back into the same segmentation model, corresponding third and / or fourth images are obtained. By determining the difference between the original segmentation results and the reconstructed segmentation results, a self-supervised signal is constructed to train and optimize the image segmentation model, enabling it to gradually learn and master the accurate segmentation ability of characters and scene objects in specific game scenarios without manual annotation. In the actual inference stage, the trained target image segmentation model is used to segment the image to be detected, and the segmentation results are input into a classification model to automatically output a judgment on whether clipping exists. Therefore, this application significantly improves the automation level and model generalization ability of rendering clipping detection by introducing a self-supervised learning mechanism. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the specific embodiments or related technologies of this application, the drawings used in the description of the specific embodiments or related technologies will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0018] Figure 1 This is a schematic diagram illustrating an application scenario according to an embodiment of this application;

[0019] Figure 2 This is a schematic flowchart of a first embodiment of an image processing method according to this application. Figure 3 This is a schematic diagram of a second process of an image processing method according to an embodiment of this application; Figure 4 This is a schematic diagram illustrating the segmentation of a first sample image according to an embodiment of this application; Figure 5 This is a schematic diagram of the marked boundary box according to an embodiment of this application; Figure 6 This is a schematic diagram of a reconstructed image according to an embodiment of this application; Figure 7 This is a schematic diagram illustrating the segmentation of a reconstructed image according to an embodiment of this application; Figure 8 This is a structural block diagram of an image processing apparatus according to an embodiment of this application; Figure 9 This is a schematic diagram of the hardware structure of an electronic device according to an embodiment of this application. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0021] It should be noted that the information (including but not limited to user input information, such as information entered into input boxes), data (including but not limited to data used for analysis, stored data, and displayed data, such as context code, all code of the current project, the service pressure corresponding to operations performed on all code of the current project, and the code development status of the current project), and signals involved in this application are all authorized by the user or fully authorized by all parties, and the collection, use, and processing of related data must comply with relevant laws, regulations, and standards. For example, the context code, operations performed on all code of the current project, the corresponding service pressure, and the code development status involved in this application were all obtained with full authorization.

[0022] The terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this application, "multiple" means two or more, unless otherwise explicitly specified.

[0023] Before providing a detailed description of the embodiments of this application, some of the nouns and terms involved in the embodiments of this application will be explained.

[0024] (1) Virtual character refers to a digital image controlled by program logic or directly manipulated by the user in a virtual environment (such as games, simulations, virtual reality, etc.) generated by electronic devices. It is usually the core subject in the scene, with certain dynamic behavior capabilities (such as moving, performing specific actions, etc.), and its form can be a person, an animal or anthropomorphic object.

[0025] (2) Interactive virtual objects refer to objects in a virtual environment that have spatial contact, occlusion, or logical connection with the aforementioned virtual characters. These objects are usually static or dynamic elements in the scene, such as environmental facilities, props, or other non-player characters. When the position or action of the virtual character does not match these objects, visual penetration in the rendering will occur.

[0026] (3) Clipping (or rendering clipping) refers to a technical anomaly in the generated virtual environment where conflicts arise due to the position, movement, or collision logic of the model geometry, causing two or more virtual objects that should be occluded or separated to inexplicably penetrate, intersect, or overlap in the rendered image, thus destroying visual realism. For example, a character standing in front of a chair may have the chair behind them opaquely displayed (which should normally be obscured by the character's body). Another example is a character sitting in a chair whose belt model passes through the chair model.

[0027] As one optional application scenario in this application embodiment, Figure 1 A schematic diagram illustrating an application scenario of an image processing system is shown. For example... Figure 1 As shown, the system may include at least one terminal device and at least one server. Figure 1 The system is illustrated in the example, which includes a computer 101, a mobile terminal 102, and a server 103, and the terminal devices such as the computer 101 and the mobile terminal 102 are connected to the server 103 through a network 110.

[0028] Specifically, the terminal device can be a smartphone, tablet, laptop, PDA, desktop computer, game console, smart TV, smart wearable device, in-vehicle terminal, VR (Virtual Reality) device, AR (Augmented Reality) device, etc. Server 103 can be a standalone physical server, a server cluster, a distributed system, or a cloud server providing cloud services. Network 110 can be a wired or wireless network, examples of which include, but are not limited to, the Internet, corporate intranet, local area network, wide area network, mobile communication network, and combinations thereof.

[0029] Taking asymmetrical competitive games as an example, the terminal device has the corresponding game application installed. By running the application, a corresponding virtual scene can be rendered and displayed to the player using the terminal device's graphical user interface. For example, the virtual scene can be part or all of a simulated three-dimensional (3D) virtual space; for example, the virtual scene displayed to the player can be a scene that can be observed from the perspective of the virtual character controlled by the player. The virtual scene can include environmental elements such as scene props, buildings, and terrain, and can also include non-player characters, virtual characters controlled by other players, etc.

[0030] In the rendering and presentation of game visuals, the realism and continuity of visual presentation directly impact the player's immersion and gaming experience. However, due to the dynamic interactions and complex spatial relationships between character models and scene objects, unreasonable visual overlap between models may occur, known as clipping. For example, when a character performs actions, uses skills, or interacts with scene props, visual penetration often occurs between the character model and scene elements such as tables, chairs, walls, and equipment. This type of clipping not only detracts from the realism of the visuals but can also interfere with the player's judgment of the game character's status, position, and interaction results, affecting the smoothness and consistency of the overall gaming experience.

[0031] Currently, the following three methods are mainly used to detect clipping in rendering: (1) Manual visual inspection: Testers directly observe the game screen, going through all characters, actions and interactive scenes to find clipping problems. This method relies entirely on manpower, is inefficient, and is prone to omissions. Moreover, the inspection cost increases sharply as the game content increases.

[0032] (2) Automated recording and manual review: The game screen is recorded by automating all actions and interactions, and then the recorded video is reviewed by a human to make a judgment. Although this method reduces some manual operation, the script needs to be rewritten for new characters, new actions or new interactions, which has poor adaptability and still requires a lot of time for manual review of the video.

[0033] (3) Detection based on supervised learning: Automatic detection is performed by training a classification model of a convolutional neural network through a large amount of manually labeled image data. Although this method achieves automation to a certain extent, it requires a large amount of labeled data, has a long training cycle, and the model has weak generalization ability across different game projects, making it difficult to transfer and apply.

[0034] The image processing method provided in this application segmentes sample images using an initial image segmentation model to obtain images of virtual characters and / or their interactive objects. Then, through image reconstruction and re-segmentation, the model is self-supervised trained based on the differences between the segmentation results. This allows for the acquisition of a target image segmentation model with generalization capabilities without relying on extensive manual annotation. Subsequently, the image to be detected is input into the model to obtain segmentation results, and a target classification model automatically determines whether clipping occurs. This process not only significantly reduces the workload of manual detection and annotation, improving detection efficiency, but also enhances the model's adaptability and generalization ability across different projects, overcoming the shortcomings of related technologies such as reliance on manual work, cumbersome script writing, high cost of supervised learning annotation, and weak generalization.

[0035] According to an embodiment of this application, an image processing method embodiment is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0036] This embodiment provides an image processing method that can be used in electronic devices, such as computer 101, mobile terminal 102, etc. Figure 2 This is a flowchart of an image processing method according to an embodiment of this application, such as... Figure 2 As shown, the process includes the following steps: Step S201: Obtain multiple first sample images.

[0037] The first sample images refer to the raw image data used to train the image segmentation model, typically screenshots of scenes in games where clipping may occur. These images contain virtual characters (such as game characters) and objects that may interact with them (such as chairs, walls, etc.). Specifically, the first sample images can be from game scenes, virtual environments, or any visual content containing virtual characters. They can be acquired through automatic rendering by a game engine, cropped from existing game footage, or obtained through other image acquisition methods. Importantly, these images should cover a wide range of possible scenes and character states so that the subsequent model can learn diverse features. During acquisition, it's important to ensure the images are representative, for example, containing different characters, different actions, and different environmental backgrounds, thus providing a rich data foundation for subsequent segmentation and detection tasks.

[0038] Step S202: For any first sample image, the first sample image is segmented using the initial image segmentation model to obtain a first image containing a virtual character, and / or a second image containing a virtual character and an interactive virtual object, wherein there is interaction between the interactive virtual object and the virtual character.

[0039] The initial image segmentation model refers to the untuned SAM (Segment Anything Model) used for preliminary image segmentation, extracting the contours of virtual characters and / or interactive objects. The first image refers to the segmented image containing only the virtual character obtained after segmenting the first sample image using the initial image segmentation model. The second image refers to the segmented image containing both the virtual character and interactive virtual objects obtained after segmenting the first sample image using the initial image segmentation model. Specifically, for each acquired first sample image, an initial image segmentation model is used to process it. This segmentation model can identify different objects or regions in the image and segment them according to a preset target. Specifically, the initial image segmentation model can extract the portion containing only the virtual character from the first sample image to obtain the first image; simultaneously, it can also extract the portion containing both the virtual character and interactive virtual objects to obtain the second image. Interactive virtual objects refer to objects that are visually or logically associated with the virtual character, such as props held by the character or furniture the character is sitting on. In this way, regions with different semantic levels can be separated from the original image, providing a foundation for subsequent reconstruction and comparison.

[0040] Step S203: Perform image reconstruction processing on the first image and / or the second image to obtain a reconstructed image.

[0041] A reconstructed image refers to the result of image reconstruction processing on a first image and / or a second image. The purpose of reconstruction is to generate a new synthetic image based on the segmented image content. Specifically, characters and object elements in the first and second images can be combined with background or other environmental information and recombined into a complete image using a certain reconstruction algorithm. For example, this reconstruction process will adjust the position, order, or viewpoint of these elements, but will not introduce new objects or delete existing ones. The reconstructed image is essentially a variant of the original scene; it retains key elements but alters their layout or relative relationships, thus creating conditions for subsequent segmentation consistency checks.

[0042] Step S204: The reconstructed image is segmented using the initial image segmentation model to obtain a third image containing the virtual character, and / or a fourth image containing the virtual character and interactive virtual objects.

[0043] The third image refers to the segmentation result image containing only the virtual character, obtained after the reconstructed image is input into the initial image segmentation model again. The fourth image refers to the segmentation result image containing both the virtual character and interactive virtual objects, obtained after the reconstructed image is input into the initial image segmentation model again. Specifically, the reconstructed image is input into the same initial image segmentation model for a second segmentation. This time, the segmentation model performs the same segmentation task on the reconstructed image as the first time, namely, attempting to extract the virtual character and related interactive objects. The resulting images are called the third image (containing only the virtual character) and the fourth image (containing both the virtual character and interactive objects). Since the elements in the reconstructed image are essentially the same as those in the original image (only the layout may be different), ideally, the segmentation model should be able to accurately identify these elements again, thus obtaining an image that matches the content of the first segmentation result.

[0044] Step S205: Based on the first difference between the first image and the third image, and / or the second difference between the second image and the fourth image, train the initial image segmentation model to obtain the target image segmentation model.

[0045] The first difference refers to the difference between the first and third images, used to measure the consistency of the model's segmentation of virtual characters before and after reconstruction, belonging to the contrastive learning loss in self-supervised learning. The second difference refers to the difference between the second and fourth images, used to measure the consistency of the model's segmentation of virtual characters and interactive objects before and after reconstruction. The target image segmentation model refers to the fine-tuned SAM model, which can more accurately segment virtual characters and their interactive objects in the game and has the ability to adapt to game-style scenes. Specifically, by comparing the images obtained from the first segmentation (first and / or second images) with the images obtained from the second segmentation (third and / or fourth images), the differences between them can be calculated. This difference can be a pixel-level similarity measure or a higher-level feature consistency assessment. Using these differences as training signals, the initial image segmentation model can be optimized and adjusted. The goal of training is to make the model produce more consistent results in the two segmentations, that is, regardless of the image reconstruction transformation, the model can stably identify the same virtual characters and / or interactive objects. After such self-supervised training, the model can better understand the semantics of key elements in the image, thus obtaining the target image segmentation model, which has stronger adaptability and accuracy for subsequent detection tasks.

[0046] Step S206: Obtain the image to be detected, and use the target image segmentation model to segment the image to be detected to obtain the target segmentation result.

[0047] The image to be detected refers to a game screenshot or rendered image where it is necessary to determine whether there is a clipping issue. The target segmentation result refers to the segmented image (containing the outlines of virtual characters and / or interactive objects) obtained after inputting the image to be detected into the target image segmentation model. Specifically, after the model training is completed, the practical application stage begins. First, the image to be detected, i.e., the image to be detected, is acquired. These images can be, for example, newly rendered game scenes, screenshots of character actions, etc. Then, the image to be detected is input into the trained target image segmentation model, which will segment the image and extract the virtual characters and / or interactive objects related to the characters. The resulting image or mask after segmentation is the target segmentation result. This result clearly identifies which regions in the image belong to virtual characters and which belong to interactive objects, providing key input for subsequent classification judgments.

[0048] Step S207: Input the target segmentation result into the target classification model to obtain the clipping classification result of the image to be detected.

[0049] A target classification model is a binary classification model used to determine whether rendering clipping exists in the target segmentation result. For example, a structure of Convolutional Neural Network (CNN) followed by Multilayer Perceptron (MLP) can be used, directly taking the segmented image as input and outputting the clipping classification result. The clipping classification result refers to the judgment result output by the target classification model, indicating whether rendering clipping has occurred in the image to be detected, that is, unreasonable overlap or penetration between virtual characters and scenes or other objects, for example, 0 (indicating no clipping) or 1 (indicating clipping). Specifically, the obtained target segmentation result is input into a pre-trained target classification model. The task of this classification model is to directly determine whether clipping exists in the original image based on the image features of the segmentation result. The output of the model is a binary classification result, namely clipping or no clipping. This result can automatically indicate whether the image to be detected has a rendering clipping problem.

[0050] The image processing method provided in this application segmentes a first sample image using an initial image segmentation model, extracting a first image containing a virtual character and / or a second image containing the virtual character and an interactive object, thereby achieving dual attention to both character clipping and interaction clipping. By reconstructing these segmentation results and inputting the reconstructed images back into the same segmentation model, corresponding third and / or fourth images are obtained. By determining the difference between the original segmentation results and the reconstructed segmentation results, a self-supervised signal is constructed to train and optimize the image segmentation model, enabling it to gradually learn and master the accurate segmentation ability of characters and scene objects in specific game scenarios without manual annotation. In the actual inference stage, the trained target image segmentation model is used to segment the image to be detected, and the segmentation results are input into a classification model to automatically output a judgment on whether clipping exists. Therefore, this application significantly improves the automation level and model generalization ability of rendering clipping detection by introducing a self-supervised learning mechanism.

[0051] This embodiment provides an image processing method that can be used in electronic devices, such as computer 101, mobile terminal 102, etc. Figure 3 This is a flowchart of an image processing method according to an embodiment of this application, such as... Figure 3 As shown, the process includes the following steps: Step S301: Acquire multiple first sample images. See details below. Figure 2 Step S201 of the illustrated embodiment will not be described again here.

[0052] Step S302: For any first sample image, the first sample image is segmented using the initial image segmentation model to obtain a first image containing a virtual character, and / or a second image containing a virtual character and an interactive virtual object, wherein there is interaction between the interactive virtual object and the virtual character.

[0053] Specifically, the initial image segmentation model is used to segment the first sample image to obtain a first image containing a virtual character, and / or a second image containing a virtual character and an interactive virtual object, including: inputting a first prompt word and the first sample image into the initial image segmentation model to obtain the first image, wherein the first prompt word is used to instruct the initial image segmentation model to segment the virtual character; and / or, inputting a second prompt word and the first sample image into the initial image segmentation model to obtain the second image, wherein the second prompt word is used to instruct the initial image segmentation model to segment the virtual character and the interactive virtual object.

[0054] The first cue word is input information used to guide the initial image segmentation model in segmenting the first sample image, directing the model to focus on and extract the virtual character within the image. It essentially provides the model with a clear instruction to identify the region belonging to the virtual character and generate the corresponding segmentation result. For example, it could be "Help me segment the character in the image." Specifically, during segmentation, the first cue word and the first sample image are used as input to the model. The first cue word is an abstract instruction that explicitly tells the model to focus on and extract the virtual character region in the image. After receiving the cue word and the image, the model uses its built-in semantic understanding capabilities to locate the target matching the cue word description—the virtual character—and generates a segmentation mask or contour for that character. Finally, the model outputs an image containing only the virtual character, i.e., the first image.

[0055] The second cue word is input information used when segmenting the first sample image to instruct the initial image segmentation model to pay attention not only to the virtual character but also to virtual objects that interact with the virtual character. It expands the model's segmentation scope, enabling it to extract regions from the image that contain both the virtual character and the objects interacting with it, thus obtaining the second image. For example, it could be "objects that directly interact with the character also need to be segmented." Similarly, when it's necessary to simultaneously capture the virtual character and its interacting objects, the second cue word is input along with the first sample image into the initial image segmentation model. The second cue word expands the guidance scope, instructing the model to identify not only the virtual character but also the virtual objects that interact with it. Based on this broader semantic instruction, the model simultaneously locates the virtual character and its associated interacting objects in the image and segments them as a single region. The output is an image that simultaneously contains the virtual character and the interacting objects—the second image.

[0056] For example, such as Figure 4 As shown, the left side is the first sample image, and the right side shows the virtual character and interactive virtual object (chair) obtained after the first sample image is input into the initial image segmentation model.

[0057] The image processing method provided in this application further enhances the flexibility and accuracy of the image segmentation process by introducing a prompt word guidance mechanism. When segmenting the first sample image using the initial image segmentation model, a first prompt word is input to instruct the model to segment the virtual character, thereby obtaining a first image containing only the virtual character; and / or, a second prompt word is input to instruct the model to segment the virtual character and its interactive object, obtaining a second image containing both. This prompt word-based segmentation method allows the model to dynamically adjust the segmentation target according to different semantic instructions, accurately extracting the character itself while completely preserving the associated regions of the character and the interactive object, thus providing richer and more accurate semantic input for subsequent self-supervised training. The combined use of the two prompt words effectively enhances the modeling capability for complex interactive scenarios, satisfying both the detection requirements for clipping through the character itself and the recognition of clipping through the character's interaction with props, significantly improving the controllability and adaptability of the segmentation task.

[0058] In some optional implementations, the first sample image is segmented using an initial image segmentation model to obtain a first image containing the virtual character, and / or a second image containing the virtual character and interactive virtual objects, further comprising: Step a1: Obtain the location information of the region of interest in the first sample image. The region of interest is related to the virtual character.

[0059] The region of interest (ROI) refers to a local image area associated with the virtual character, actively marked by the tester (or user) on the first sample image. Location information refers to the specific spatial description of the ROI within the first sample image, such as the region's coordinates, bounding box, or outline. Specifically, obtaining the location information of the ROI can be achieved in various ways. A common method is with manual assistance, where an operator manually marks the area related to the virtual character on the first sample image, for example, using rectangles or outlines to depict the character's position or the area where the character interacts with objects. Figure 5 As shown. Another approach is to use automatic detection algorithms, leveraging existing object detection or segmentation models to initially locate the virtual character, thereby automatically generating its bounding box or mask. Regardless of the method used, the final result will be information describing the specific location of the region in the image, such as coordinate range and geometry. This location information represents key regions associated with the virtual character and can be used to guide the subsequent segmentation process, improving the accuracy and efficiency of segmentation.

[0060] Step a2: Input the first sample image and the location information of the region of interest into the initial image segmentation model to obtain the first image and / or the second image. The location information of the region of interest is used to guide the initial image segmentation model to segment the virtual character.

[0061] After obtaining the location information of the region of interest, this information, along with the first sample image, is input into the initial image segmentation model. The location information, as a spatial prior, provides additional constraints to the model, guiding it to focus its attention on segmenting within the specified image region. By combining image content and location information, the model can more accurately extract virtual characters within that region, thus obtaining the first image; or, when it is necessary to extract both characters and interactive objects simultaneously, the model can further identify relevant objects within the region, outputting the second image. Introducing location information helps reduce background interference and improve segmentation quality, especially suitable for complex scenes or situations where the initial model capabilities are insufficient.

[0062] In the above implementation, when segmenting the first sample image using the initial image segmentation model, the location information of the region of interest associated with the virtual character in the image is first obtained. This location information, along with the first sample image, is then input into the segmentation model. The region location information guides the model to perform more accurate segmentation of the virtual character, thereby obtaining the first image and / or the second image. This mechanism allows the model to focus on specific key regions for segmentation, effectively avoiding segmentation errors caused by semantic understanding biases, especially in complex scenes or situations with ambiguous boundaries. By introducing manually labeled region prior knowledge, not only is the accuracy of the segmentation results improved, but the model's ability to perceive local details of the character is also enhanced, providing higher-quality sample input for subsequent self-supervised training.

[0063] Step S303: Perform image reconstruction processing on the first image and / or the second image to obtain a reconstructed image.

[0064] Specifically, step S303 includes: Step S3031: Obtain the background image corresponding to the first sample image.

[0065] The background image refers to the portion of the image remaining after stripping virtual characters and / or interactive objects from the first sample image. It represents the static environment of the scene that does not contain the main elements of interest. The background image typically retains the structure and texture of the original scene but removes the foreground virtual characters and related objects, providing underlying scene layout information for subsequent image reconstruction. Specifically, the method for obtaining the background image usually depends on the processing of the first sample image. One direct approach is to utilize existing segmentation results to remove the identified virtual character and interactive object regions from the first sample image. However, this leaves blank areas in the image, requiring image completion or restoration techniques to adaptively fill these blanks, allowing them to blend naturally with the surrounding background, thus obtaining a complete background image without foreground elements. Another approach is to directly obtain the original scene image without characters from the game engine or scene library as the background, which requires pre-preparation or real-time rendering. The background image retains the underlying structure and texture of the scene, providing the environmental foundation for subsequent reconstruction.

[0066] Step S3032: The positions of the first image and / or the second image are shuffled in the background image to obtain the reconstructed image.

[0067] After obtaining the background image and the first and / or second images, the foreground elements (i.e., virtual characters and / or interactive objects) are rearranged on the background image. Specifically, the position, orientation, or relative order of the foreground elements in the background can be randomly changed, such as moving them to different coordinate positions or adjusting their stacking order. This shuffling process simulates a variety of possible scene layouts while preserving the visual content of the foreground elements themselves. Finally, the adjusted foreground and background are composited to generate a new image, i.e., the reconstructed image. The characters and objects in this image are the same as in the original image, but their spatial relationships have changed. For example, as shown... Figure 6 As shown in the figure, the reconstructed image is generated by randomly shuffling the positions of virtual characters and interactive virtual objects in the background image. The original virtual objects are preserved, but the spatial layout between the objects is changed.

[0068] In some alternative implementations, image reconstruction processing can be achieved using various scene reconstruction models, such as neural radiation field (NeRF)-based models or multi-view stereo vision (MVS) methods, the core of which is to synthesize a new scene image using background images and foreground elements.

[0069] The image processing method provided in this application, after obtaining a first image and / or a second image, first acquires the background image corresponding to the first sample image, and then shuffles the positions of the segmented first image and / or second image in the background image to generate a reconstructed image. This processing method not only effectively preserves the background information in the original scene, but also constructs diverse scene combinations by randomly adjusting the positions of the segmented objects, providing rich training samples for subsequent self-supervised learning. Through position perturbation operations, reconstructed images with differences can be automatically generated without additional manual annotation, thereby effectively supporting the contrastive learning training of the image segmentation model, enhancing the model's semantic understanding of characters and objects in different spatial positions, and further improving the model's generalization performance and robustness.

[0070] Step S304: The reconstructed image is segmented using the initial image segmentation model to obtain a third image containing the virtual character, and / or a fourth image containing the virtual character and interactive virtual objects.

[0071] Specifically, step S304 includes: Step S3041: Input the third prompt word and the reconstructed image into the initial image segmentation model to obtain the fifth image corresponding to each virtual object in the reconstructed image. The third prompt word is used to instruct the initial image segmentation model to segment each virtual object in the reconstructed image.

[0072] The third cue word is input information used during image segmentation to instruct the initial image segmentation model to identify and extract all possible virtual objects in the image. It does not limit the specific target type but rather requires the model to segment each independent object or region in the reconstructed image as comprehensively as possible, thereby generating a series of intermediate results corresponding to different objects. For example, it could be "Segment all objects in the image." Each virtual object in the reconstructed image refers to multiple independent and distinguishable elements contained in the new image obtained after image reconstruction, such as virtual characters, objects interacting with characters, or other objects that may appear in the background.

[0073] The fifth image refers to the segmentation result image corresponding to each independent virtual object extracted from the reconstructed image after segmentation using the initial image segmentation model. Each fifth image corresponds to a specific object (such as a virtual character or a physical object), and they are individual representations obtained by separating multiple objects in the reconstructed image one by one. Specifically, since the positions of virtual characters and interactive objects in the reconstructed image may change, the original "segment characters and interactive objects" prompt may no longer be applicable. Therefore, a more general segmentation strategy is adopted: the reconstructed image and the third prompt are input together into the initial image segmentation model. The semantics of the third prompt are to guide the model to segment all independent virtual objects in the image one by one, rather than targeting only a specific target. After receiving the instruction, the model analyzes the content in the reconstructed image, identifies each distinguishable virtual object (such as a character, object, etc.), and generates corresponding segmentation results for them. These segmentation results are output in the form of images, each corresponding to an independent virtual object, i.e., the fifth image.

[0074] For example, such as Figure 7 As shown, the left side is the reconstructed image, and the right side is the fifth image corresponding to each virtual object obtained after the reconstructed image is input into the initial image segmentation model, including virtual characters and interactive virtual objects (chairs).

[0075] Step S3042: Perform similarity matching between each fifth image and the first image, and determine the third image that matches the first image among the multiple fifth images based on the first similarity matching results; and / or: Perform similarity matching between each fifth image and the second image, and determine the fourth image that matches the second image among the multiple fifth images based on the second similarity matching results.

[0076] The first similarity matching result refers to the matching metric obtained after calculating the similarity between each of the fifth images (each corresponding to a virtual object) obtained from the segmentation of the reconstructed image and the original segmented first image (containing only the virtual character). Specifically, after obtaining the fifth images corresponding to each virtual object in the reconstructed image, these fifth images need to be compared one by one with the original segmented first image. This process first involves calculating the similarity between each fifth image and the first image, which can be done using various image similarity measurement methods, such as pixel-level structural similarity index, cosine similarity based on feature extraction, or matching after extracting high-level semantic features using a pre-trained visual model. Through calculation, each fifth image will obtain a similarity value to the first image, which is the first similarity matching result. Subsequently, based on these similarity scores, one or more images that best match the first image are selected from all the fifth images. Usually, the fifth image with the highest similarity is selected as the third image corresponding to the first image. This matching process ensures that the part corresponding to the original virtual character is accurately located in the reconstructed image, laying the foundation for subsequent calculation of the difference between the two.

[0077] The second similarity matching result refers to the matching metric obtained by matching the similarity of each fifth image (each corresponding to a virtual object) obtained after segmenting the reconstructed image with the original segmented second image (containing virtual characters and interactive virtual objects). Similarly, when it is necessary to determine the region in the reconstructed image that corresponds to the second image (containing virtual characters and interactive virtual objects), similarity matching is also required. The similarity between each fifth image and the second image is calculated, resulting in the second similarity matching result. Since the second image itself may contain multiple objects, the matching strategy may need to consider combined matching. For example, multiple fifth images can be combined, and the similarity of the combined whole with the second image can be calculated, or the character and object parts can be matched separately and then judged comprehensively. Finally, based on these matching results, one or more images that best match the content of the second image are determined from the fifth images and designated as the fourth image. The fourth image may be a single fifth image (if the characters and objects in the second image are still segmented as a whole after reconstruction) or a collection of multiple fifth images (if the characters and objects are segmented into independent objects). This step ensures that the regions corresponding to the original interaction relationships are correctly identified in the reconstructed image.

[0078] The image processing method provided in this application, when segmenting a reconstructed image using an initial image segmentation model, first uses a third cue word to instruct the model to segment all virtual objects in the reconstructed image, obtaining a fifth image corresponding to each object. These fifth images are then matched for similarity with the previously obtained first and second images. Based on the matching results, the third image corresponding to the first image and the fourth image corresponding to the second image are accurately identified from among the multiple fifth images. This process cleverly combines full object segmentation with similarity matching, ensuring that even if the object position changes or multiple objects exist in the reconstructed image, the target region corresponding to the original segmentation can still be accurately found, thus providing a reliable data alignment basis for subsequent calculation of pixel differences before and after segmentation. This approach not only enhances adaptability to complex reconstruction scenarios but also avoids training errors caused by object misalignment or mismatch, significantly improving the effectiveness of the loss function in self-supervised learning and laying a solid foundation for fine-tuning of the image segmentation model.

[0079] Step S305: Based on the first difference between the first image and the third image, and / or the second difference between the second image and the fourth image, train the initial image segmentation model to obtain the target image segmentation model.

[0080] Specifically, step S305 includes: Step S3051: Determine the first pixel difference between corresponding pixels in the first image and the third image, and / or determine the second pixel difference between corresponding pixels in the second image and the fourth image.

[0081] The first pixel difference refers to the numerical difference between the first and third images at corresponding pixel positions, typically calculated by subtracting the differences in brightness, color, and other values ​​for each pixel. The second pixel difference refers to the numerical difference between the second and fourth images at corresponding pixel positions. Similar to the first pixel difference, it measures the pixel-level deviation between the second image (containing virtual characters and interactive objects) and the reconstructed and re-segmented fourth image. Specifically, after determining the correspondence between the first and third images, the difference between them needs to be quantified. This process is usually achieved through pixel-by-pixel comparison. First, ensure that the first and third images have the same size and spatial alignment. Then, at each corresponding pixel position (i, j), calculate the difference between their pixel values ​​(such as grayscale values ​​or RGB channel values). Summarizing the differences at all pixel positions yields an overall difference metric, the first pixel difference. Similarly, for the second and fourth images, a pixel-by-pixel comparison is performed, calculating the pixel difference at each corresponding position to obtain the second pixel difference. It's important to note that if the fourth image is composed of multiple fifth images, these fifth images must first be combined into a single image according to their original positions in the reconstructed image, and then compared pixel-by-pixel with the second image. These pixel differences reflect the degree of consistency in detail between the model output and the original segmentation result after reconstruction and re-segmentation.

[0082] Step S3052: Based on the first pixel difference and / or the second pixel difference, adjust the parameters of the initial image segmentation model to obtain the target image segmentation model.

[0083] After obtaining the first and / or second pixel differences, these differences can be used as training signals to optimize the initial image segmentation model. Specifically, these pixel differences can be used to construct a loss function, such as calculating the sum or average of the squares of the differences between all pixel positions. The larger the value of the loss function, the more inconsistent the model's segmentation results for the same object are before and after reconstruction. Using the backpropagation algorithm, the gradient of the loss function with respect to the model parameters is calculated, and the model's weight parameters are adjusted according to the gradient direction, causing the value of the loss function to gradually decrease. This optimization process can be iterative, continuously updating the model parameters using multiple first sample images and their corresponding reconstructed images. After multiple rounds of training, the model learns to stably identify and segment virtual characters and related objects even when the image content changes (such as position scrambling and reconstruction), thus obtaining the target image segmentation model. This model exhibits better adaptability and segmentation consistency for the target game scene. This method of optimizing the model using the differences between two segmentation results belongs to the contrastive learning strategy in self-supervised learning.

[0084] The loss function uses pixel-level mean squared error, and the calculation formula is shown below:

[0085] in, These are the corresponding pixel values ​​from the initial segmentation results. denoted as the pixel value of the resegmentation result. By minimizing this loss, the model can progressively improve its robustness to segmentation in a specific domain.

[0086] The image processing method provided in this application calculates the pixel difference between each pixel position in the first and third images, and the second and fourth images, respectively, after obtaining the first and third images, thereby quantifying the degree of change in the segmentation result before and after reconstruction. Subsequently, the parameters of the initial image segmentation model are adjusted based on these pixel differences, ultimately obtaining an optimized target image segmentation model. This process, through pixel-level fine-grained comparison, allows the model to intuitively perceive the consistency of its segmentation results for the same object before and after reconstruction, thus guiding the model to learn more stable and accurate image segmentation capabilities. Through this self-supervised learning method based on pixel differences, model optimization can be driven by the structural information of the image itself without relying on manually labeled real tags, significantly improving training efficiency and automation. Simultaneously, it effectively enhances the model's semantic understanding of game characters and interactive objects, laying a high-quality segmentation foundation for subsequent clipping detection tasks.

[0087] Step S306: Obtain the image to be detected, and segment the image using a target image segmentation model to obtain the target segmentation result. For details, please refer to [link to relevant documentation]. Figure 2 Step S206 of the illustrated embodiment will not be described again here.

[0088] Step S307: Input the target segmentation result into the target classification model to obtain the clipping classification result of the image to be detected.

[0089] In some alternative implementations, the target classification model is trained based on multiple sample segmentation results obtained by segmenting multiple second sample images using a target image segmentation model, and on annotated rendered pattern samples.

[0090] The second sample images refer to another batch of image data used to train the target classification model. They may originate from the same or similar sources as the first sample images, but have different purposes. After being processed by the target image segmentation model, the second sample images yield corresponding segmentation results. These results, combined with manually labeled information, are used to train a classifier capable of identifying clipping phenomena. The sample segmentation result refers to the image or mask output after the second sample images are input into the trained target image segmentation model and processed. The labeled rendered clipping samples refer to the data after the above sample segmentation results have been manually judged and labeled; each sample is labeled as "exists clipping" or "does not exist clipping." These labeled segmentation results constitute the supervision signal for training the target classification model, enabling the classification model to learn to identify clipping phenomena from the features of the segmented images.

[0091] Specifically, the construction of the target classification model relies on a pre-trained target image segmentation model and additional manually labeled data. First, a batch of new image data, i.e., second sample images, is collected. These images should cover various scenarios where clipping might occur. The second sample images are input into the pre-trained target image segmentation model, which segments these images and outputs corresponding segmentation result images, i.e., sample segmentation results. These segmentation results clearly indicate the position and contours of virtual characters and related objects in the images. Next, these sample segmentation results are manually labeled to determine whether there is rendering clipping in the original image corresponding to each segmentation result. The labeling results (e.g., 1 for clipping, 0 for no clipping) are then mapped to the segmentation results to form labeled rendering clipping samples. Finally, the sample segmentation results are used as input features, and the manually labeled results are used as supervision labels to train a classification model. This classification model learns to identify clipping patterns from the image features of the segmentation results. After training, it can automatically judge new segmentation results and output clipping classification results. In this way, an effective clipping detection classifier can be obtained with only a small amount of manual labeling.

[0092] The image processing method provided in this application constructs a complete and efficient automated process for rendering clipping detection by building a collaborative training mechanism between a segmentation model and a classification model. The target classification model is not trained independently, but rather based on the sample segmentation results obtained by segmenting multiple second sample images using an optimized target image segmentation model, and is jointly trained with manually annotated rendered clipping samples. This mechanism allows the classification model to directly use high-quality segmentation results as input to learn the mapping relationship between segmented image features and clipping labels, thereby effectively focusing on the clipping discrimination task itself.

[0093] In the following embodiments, the above image processing method will be illustrated by example in the specific application scenario of automated detection of clipping in asymmetrical competitive games.

[0094] During the model preparation phase, representative scene screenshots from the game were collected as the first sample images. These images were automatically rendered by the game engine, covering different characters, actions, and environmental backgrounds to ensure data diversity. The initial image segmentation model used the SAM model, which can be directly used for segmentation tasks without any pre-training. For each sample image, segmentation was guided by two prompts: one input, "Help me segment the characters in the image," yielded a first image containing only the virtual character; the other input, "Objects that directly interact with the character also need to be segmented," yielded a second image containing both the virtual character and interactive objects (such as chairs, doors, boxes, etc.). When the character's position in some images is complex or occluded, testers could additionally annotate the region of interest (e.g., using a rectangle to outline the character's area) and input the positional information along with the image into the SAM to improve segmentation accuracy.

[0095] Next, we move into the self-supervised fine-tuning stage. For each sample image, we first obtain a background image without foreground using image inpainting techniques. Then, we randomly shuffle the foreground elements from the first and second images on the background image (e.g., changing the coordinates of the character or the relative order of objects) to generate a reconstructed image. Here, a NeRF-based scene reconstruction model can be used to synthesize a new image using the background and the shuffled foreground. This process does not add or remove any virtual objects, only changing the spatial layout. The reconstructed image is then input into the SAM again, and a more general third prompt, "Help me segment all objects in the image," is used to obtain multiple fifth images, each corresponding to an independent virtual object. By calculating the similarity between each fifth image and the first image, the best match is selected as the third image; simultaneously, the combined similarity between the fifth image and the second image is calculated to determine the corresponding fourth image. Subsequently, a pixel-level mean squared error loss function is constructed, and the differences between the first and third images, and between the second and fourth images, are compared. This loss is used for backpropagation to fine-tune the SAM model. After multiple iterations, the model gradually learns to stably identify virtual characters and related objects even after layout changes, ultimately obtaining a target image segmentation model optimized for this game scene.

[0096] Next, a downstream clipping classifier is constructed. A new batch of second sample images is collected, covering various scenarios where clipping might occur. These images are then input into the target segmentation model to obtain the corresponding sample segmentation results. A small number of testers manually annotate these segmentation results, marking whether clipping occurs (e.g., a belt passing through a chair, an arm passing through a wall, etc.), forming labeled training samples. A lightweight CNN-MLP binary classification model is used, with the segmented images as input and the clipping labels as supervision signals, for training until the model converges. This stage requires only a small number of sample annotations, significantly reducing manual labor costs.

[0097] In actual reasoning, for any game screenshot to be detected, the target segmentation model is first input, and either the first or second cue word (or both) is selected according to the scenario requirements to obtain the target segmentation result. This result is then input into a trained classification model, which automatically outputs a "clipping" or "not clipping" judgment. If clipping is detected, the screenshot, timestamp, and relevant scene information are automatically recorded for testers to review, thus efficiently locating the problem.

[0098] The image processing method provided in this application reduces the reliance on large amounts of labeled data by using a self-supervised fine-tuning SAM model. Combining prompts and manually labeled bounding boxes improves the robustness of segmentation for complex interactive regions. The classification model requires only a small number of labels for training, resulting in a highly automated and generalized process that can quickly adapt to the clipping detection needs of new characters and scenes, significantly improving the efficiency and coverage of game content quality inspection.

[0099] This embodiment also provides an image processing apparatus for implementing the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the term "module" can refer to a combination of software and / or hardware that performs a predetermined function. Although the apparatus described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0100] This embodiment provides an image processing device, such as... Figure 8 As shown, it includes: The acquisition module 801 is used to acquire multiple first sample images; The first segmentation module 802 is used to segment any first sample image using an initial image segmentation model to obtain a first image containing a virtual character, and / or a second image containing a virtual character and an interactive virtual object, wherein there is an interaction between the interactive virtual object and the virtual character. The reconstruction module 803 is used to perform image reconstruction processing on the first image and / or the second image to obtain a reconstructed image; The second segmentation module 804 is used to segment the reconstructed image using the initial image segmentation model to obtain a third image containing virtual characters, and / or a fourth image containing virtual characters and interactive virtual objects; Training module 805 is used to train an initial image segmentation model based on a first difference between a first image and a third image, and / or a second difference between a second image and a fourth image, to obtain a target image segmentation model; The third segmentation module 806 is used to acquire the image to be detected, segment the image to be detected using the target image segmentation model, and obtain the target segmentation result; The classification module 807 is used to input the target segmentation result into the target classification model to obtain the clipping classification result of the image to be detected.

[0101] In some alternative implementations, the first segmentation module 802 includes: The first segmentation submodule is used to input the first prompt word and the first sample image into the initial image segmentation model to obtain the first image. The first prompt word is used to instruct the initial image segmentation model to segment the virtual character. And / or, The second segmentation submodule is used to input the second prompt word and the first sample image into the initial image segmentation model to obtain the second image. The second prompt word is used to instruct the initial image segmentation model to segment the virtual character and interactive virtual object.

[0102] In some alternative implementations, the reconstruction module 803 includes: The first acquisition submodule is used to acquire the background image corresponding to the first sample image; The reconstruction submodule is used to shuffle the positions of the first image and / or the second image in the background image to obtain the reconstructed image.

[0103] In some alternative implementations, the second segmentation module 804 includes: The third segmentation submodule is used to input the third prompt word and the reconstructed image into the initial image segmentation model to obtain the fifth image corresponding to each virtual object in the reconstructed image. The third prompt word is used to instruct the initial image segmentation model to segment each virtual object in the reconstructed image. The first matching submodule is used to perform similarity matching between each fifth image and the first image, and to determine the third image that matches the first image among the multiple fifth images based on the first similarity matching results. And / or, The second matching submodule is used to perform similarity matching between each fifth image and the second image, and to determine the fourth image among the multiple fifth images that matches the second image based on the second similarity matching results.

[0104] In some alternative implementations, the training module 805 includes: The determination submodule is used to determine the first pixel difference between corresponding pixels between the first image and the third image, and / or to determine the second pixel difference between corresponding pixels between the second image and the fourth image. The adjustment submodule is used to adjust the parameters of the initial image segmentation model based on the first pixel difference and / or the second pixel difference to obtain the target image segmentation model.

[0105] In some alternative implementations, the first segmentation module 802 includes: The second acquisition submodule is used to acquire the location information of the region of interest in the first sample image, and the region of interest is related to the virtual character. The fourth segmentation submodule is used to input the first sample image and the location information of the region of interest into the initial image segmentation model to obtain the first image and / or the second image. The location information of the region of interest is used to guide the initial image segmentation model to segment the virtual character.

[0106] In some alternative implementations, the target classification model is trained based on multiple sample segmentation results obtained by segmenting multiple second sample images using a target image segmentation model, and on annotated rendered pattern samples.

[0107] The image processing apparatus provided in this application can execute the image processing method provided in any embodiment of this application, and has the corresponding functional modules and beneficial effects for executing the method. Further functional descriptions of the various modules and units described above are the same as those in the corresponding embodiments described above, and will not be repeated here.

[0108] Figure 9 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0109] The following is a detailed reference. Figure 9 This diagram illustrates a suitable structural schematic for implementing the electronic device described in the embodiments of this application. The electronic device may include a processor (e.g., a central processing unit, graphics processor, etc.) 901, which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 902 or a program loaded from memory 908 into random access memory (RAM) 903. The RAM 903 also stores various programs and data required for the operation of the electronic device. The processor 901, ROM 902, and RAM 903 are interconnected via a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0110] Typically, the following devices can be connected to I / O interface 905: input devices 906 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 907 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; memory devices 908 including, for example, magnetic tapes, hard disks, etc.; and communication devices 909. Communication device 909 allows electronic devices to exchange data via wireless or wired communication with other devices. Although Figure 9 Electronic devices with various devices are shown, but it should be understood that it is not required to implement or have all of the devices shown, and more or fewer devices may be implemented or have instead.

[0111] Specifically, according to embodiments of this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this application include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 909, or installed from a memory 908, or installed from a ROM 902. When the computer program is executed by the processor 901, it performs the functions defined in the image processing method of embodiments of this application.

[0112] Figure 9 The electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0113] This application also provides a computer-readable storage medium. The methods described in this application can be implemented in hardware or firmware, or implemented as recordable on a storage medium, or implemented as computer code downloaded over a network and originally stored on a remote storage medium or a non-transitory machine-readable storage medium and subsequently stored on a local storage medium. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium can be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.; further, the storage medium can also include combinations of the above types of memory. It is understood that computers, processors, microprocessor controllers, or programmable hardware include storage components capable of storing or receiving software or computer code. When the software or computer code is accessed and executed by the computer, processor, or hardware, the image processing method shown in the above embodiments is implemented.

[0114] A portion of this application can be applied as a computer program product, such as computer program instructions, which, when executed by a computer, can invoke or provide the methods and / or technical solutions according to this application through the operation of the computer. Those skilled in the art will understand that the forms in which computer program instructions exist in a computer-readable medium include, but are not limited to, source files, executable files, installation package files, etc. Correspondingly, the ways in which computer program instructions are executed by a computer include, but are not limited to: the computer directly executing the instructions, or the computer compiling the instructions and then executing the corresponding compiled program, or the computer reading and executing the instructions, or the computer reading and installing the instructions and then executing the corresponding installed program. Here, the computer-readable medium can be any available computer-readable storage medium or communication medium accessible to a computer.

[0115] Although embodiments of this application have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of this application, and all such modifications and variations fall within the scope defined by the appended claims.

Claims

1. An image processing method, characterized in that, The method includes: Acquire multiple first sample images; For any of the first sample images, the first sample image is segmented using an initial image segmentation model to obtain a first image containing a virtual character, and / or a second image containing the virtual character and an interactive virtual object, wherein there is an interaction between the interactive virtual object and the virtual character; Image reconstruction processing is performed on the first image and / or the second image to obtain a reconstructed image; The reconstructed image is segmented using the initial image segmentation model to obtain a third image containing the virtual character, and / or a fourth image containing the virtual character and the interactive virtual object; Based on the first difference between the first image and the third image, and / or the second difference between the second image and the fourth image, the initial image segmentation model is trained to obtain the target image segmentation model; The image to be detected is acquired, and the image to be detected is segmented using the target image segmentation model to obtain the target segmentation result; The target segmentation result is input into the target classification model to obtain the clipping classification result of the image to be detected.

2. The method according to claim 1, characterized in that, The step of segmenting the first sample image using an initial image segmentation model to obtain a first image containing a virtual character, and / or a second image containing the virtual character and interactive virtual objects, includes: The first prompt word and the first sample image are input into the initial image segmentation model to obtain the first image. The first prompt word is used to instruct the initial image segmentation model to segment the virtual character. And / or, The second prompt word and the first sample image are input into the initial image segmentation model to obtain the second image. The second prompt word is used to instruct the initial image segmentation model to segment the virtual character and the interactive virtual object.

3. The method according to claim 1 or 2, characterized in that, The step of performing image reconstruction processing on the first image and / or the second image to obtain a reconstructed image includes: Obtain the background image corresponding to the first sample image; The reconstructed image is obtained by scrambling the positions of the first image and / or the second image in the background image.

4. The method according to claim 1, characterized in that, The step of segmenting the reconstructed image using the initial image segmentation model to obtain a third image containing the virtual character, and / or a fourth image containing the virtual character and the interactive virtual object, includes: The third prompt word and the reconstructed image are input into the initial image segmentation model to obtain the fifth image corresponding to each virtual object in the reconstructed image. The third prompt word is used to instruct the initial image segmentation model to segment each virtual object in the reconstructed image. Each of the fifth images is matched with the first image based on similarity, and a third image that matches the first image among the multiple fifth images is determined based on the first similarity matching results. And / or, Each of the fifth images is matched with the second image for similarity, and a fourth image that matches the second image is determined from among the multiple fifth images based on the second similarity matching results.

5. The method according to claim 1, characterized in that, The step of training the initial image segmentation model based on a first difference between the first image and the third image, and / or a second difference between the second image and the fourth image, to obtain a target image segmentation model includes: Determine the first pixel difference between corresponding pixels in the first image and the third image, and / or determine the second pixel difference between corresponding pixels in the second image and the fourth image; Based on the first pixel difference and / or the second pixel difference, the parameters of the initial image segmentation model are adjusted to obtain the target image segmentation model.

6. The method according to claim 1, characterized in that, The step of segmenting the first sample image using an initial image segmentation model to obtain a first image containing a virtual character, and / or a second image containing the virtual character and interactive virtual objects, further includes: Obtain the location information of the region of interest in the first sample image, wherein the region of interest is associated with the virtual character; The first sample image and the location information of the region of interest are input into the initial image segmentation model to obtain the first image and / or the second image. The location information of the region of interest is used to guide the initial image segmentation model to segment the virtual character.

7. The method according to claim 1, characterized in that, The target classification model is obtained by training multiple sample segmentation results based on the target image segmentation model to segment multiple second sample images, as well as the labeled rendered pattern sample.

8. An image processing apparatus, characterized in that, The device includes: The acquisition module is used to acquire multiple first sample images; The first segmentation module is used to segment any first sample image using an initial image segmentation model to obtain a first image containing a virtual character, and / or a second image containing the virtual character and an interactive virtual object, wherein there is an interaction between the interactive virtual object and the virtual character. The reconstruction module is used to perform image reconstruction processing on the first image and / or the second image to obtain a reconstructed image; The second segmentation module is used to segment the reconstructed image using the initial image segmentation model to obtain a third image containing the virtual character, and / or a fourth image containing the virtual character and the interactive virtual object; The training module is used to train the initial image segmentation model based on the first difference between the first image and the third image, and / or the second difference between the second image and the fourth image, to obtain the target image segmentation model; The third segmentation module is used to acquire the image to be detected, and to segment the image to be detected using the target image segmentation model to obtain the target segmentation result; The classification module is used to input the target segmentation result into the target classification model to obtain the clipping classification result of the image to be detected.

9. An electronic device, characterized in that, include: A memory and a processor are communicatively connected, the memory storing computer instructions, and the processor executing the computer instructions to perform the image processing method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing the computer to perform the image processing method according to any one of claims 1 to 7.

11. A computer program product, characterized in that, Includes computer instructions for causing a computer to perform the image processing method according to any one of claims 1 to 7.

Images