Image anomaly testing method, apparatus, device, medium, and program product

Abstract

This application discloses an image anomaly testing method, apparatus, device, medium, and program product, which can be applied to various scenarios such as cloud gaming, game testing, image processing, image recognition, and traffic accident image detection. The method includes: acquiring a test image; inputting the test image into a target anomaly detection model to obtain first predicted anomaly information, wherein the first predicted anomaly information includes border information of at least one first predicted anomaly region in the test image and the predicted probability of at least one first predicted anomaly category to which each first predicted anomaly region belongs; performing semantic segmentation on the test image to obtain a semantically segmented image; and determining at least one target anomaly region in the test image from the at least one first predicted anomaly region based on the semantically segmented image and the first predicted anomaly information, thereby saving computational resources.

Description

Technical Field

[0001] This application relates to the field of image recognition technology, specifically to an image anomaly testing method, apparatus, equipment, medium, and program product. Background Technology

[0002] In related technologies, detecting anomalies in game visuals and identifying abnormal regions within the visuals typically relies on anomaly detection methods based on training deep learning models using pure images. These methods collect a large amount of game visual data, including both normal and abnormal samples, to train a deep learning model, which then identifies abnormal regions within the game visuals. Specifically, deep learning techniques such as Convolutional Neural Networks (CNNs) can be used, allowing the model to automatically learn features in images and identify anomalies. However, this approach suffers from several drawbacks: it requires a large number of training samples to ensure model accuracy, resulting in high data collection and labeling costs; the model needs to be retrained for different game scenarios, making direct transfer of data impossible; and the training and inference processes of deep learning models require significant computational resources, making them unsuitable for low-end devices.

[0003] In summary, the methods for detecting abnormal situations in game screens in related technologies have a cumbersome data processing process, limited applicable scenarios, and wasteful computing resources. Summary of the Invention

[0004] This application provides an image anomaly testing method, apparatus, device, medium, and program product that can save computing resources.

[0005] On the one hand, an image anomaly testing method is provided, the method comprising:

[0006] Obtain the image to be tested;

[0007] The image to be tested is input into the target anomaly detection model to obtain first predicted anomaly information. The first predicted anomaly information includes the border information of at least one first predicted anomaly region in the image to be tested and the predicted probability of at least one first predicted anomaly category to which each first predicted anomaly region belongs.

[0008] The image to be tested is semantically segmented to obtain a semantically segmented image;

[0009] Based on the semantic segmentation image and the first predicted anomaly information, at least one target anomaly region in the image to be tested is determined from the at least one first predicted anomaly region.

[0010] On the other hand, an image anomaly testing device is provided, the device comprising:

[0011] The acquisition unit is used to acquire the image to be tested;

[0012] The input unit is used to input the image to be tested into the target anomaly detection model to obtain first predicted anomaly information. The first predicted anomaly information includes the border information of at least one first predicted anomaly region in the image to be tested and the predicted probability of at least one first predicted anomaly category to which each first predicted anomaly region belongs.

[0013] A segmentation unit is used to perform semantic segmentation on the image to be tested to obtain a semantically segmented image;

[0014] The determining unit is configured to determine at least one target anomaly region in the image to be tested from the at least one first predicted anomaly region based on the semantic segmentation image and the first predicted anomaly information.

[0015] On the other hand, a computer-readable storage medium is provided that stores a computer program adapted for loading by a processor to perform the steps in the method described in any of the above embodiments.

[0016] On the other hand, a computer device is provided, the computer device including a processor and a memory, the memory storing a computer program, the processor executing the steps of the method as described in any of the above embodiments by calling the computer program stored in the memory.

[0017] On the other hand, a computer program product is provided, including computer instructions that, when executed by a processor, implement the steps in the method described in any of the above embodiments.

[0018] This application embodiment obtains a test image; inputs the test image into a target anomaly detection model to obtain first predicted anomaly information, the first predicted anomaly information including the border information of at least one first predicted anomaly region in the test image and the predicted probability of at least one first predicted anomaly category to which each first predicted anomaly region belongs; performs semantic segmentation on the test image to obtain a semantic segmentation image; and determines at least one target anomaly region in the test image from the at least one first predicted anomaly region based on the semantic segmentation image and the first predicted anomaly information. This scheme combines the anomaly prediction information of the test image output by the detection model with the semantic segmentation image of the test image to detect the target anomaly region of the test image, using multi-dimensional information. The calculation process is simple, and the target anomaly region in the test image is determined by a concise calculation, which improves the accuracy of anomaly region determination and saves computing resources. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a schematic diagram of the structure of a game system provided in an embodiment of this application.

[0021] Figure 2a This is a flowchart illustrating the image anomaly testing method provided in an embodiment of this application.

[0022] Figure 2b This is a flowchart illustrating the image anomaly testing method provided in an embodiment of this application.

[0023] Figure 3 This is a schematic diagram of the image anomaly testing device provided in an embodiment of this application.

[0024] Figure 4 A schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0025] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0026] The embodiments of this application can be applied to various scenarios such as cloud gaming, game testing, image processing, image recognition, and traffic accident image detection.

[0027] This application provides an image anomaly testing method, apparatus, device, medium, and program product. Specifically, the image anomaly testing method of this application can be executed by a computer device, which can be a terminal or a server. The terminal can be a smartphone, tablet, laptop, smart voice interaction device, etc., and the terminal can also include a client, which can be a video client, browser client, or instant messaging client, etc. The server can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms.

[0028] First, some of the nouns or terms that appear in the description of the embodiments of this application are explained as follows:

[0029] AI, short for Artificial Intelligence, is a new technological science that studies and develops theories, methods, technologies, and application systems to simulate, extend, and expand human intelligence. AI is a branch of computer science that aims to produce intelligent machines that can react in a way similar to human intelligence. Research in this field includes robotics, language and image recognition, natural language processing, and expert systems.

[0030] CNN, short for Convolutional Neural Network, is a deep learning architecture suitable for processing data with a grid-like topology, such as images.

[0031] Scene capture components play a crucial role in game development and graphics rendering, allowing developers to capture snapshots of scenes from specific viewpoints or locations and use them in various applications. Scene capture components are a graphics rendering technique that allows developers to capture 2D or 3D images of scenes from specific viewpoints or locations within games or graphics applications. These captured images can be used as textures, reflections, minimaps, dynamic textures, etc., thereby enhancing the visual effects and interactivity of games.

[0032] A depth map is a special type of image that plays an important role in computer vision, image processing, 3D reconstruction, and virtual reality. A depth map is an image in which each pixel not only carries location information but also represents the distance between the object or surface corresponding to that pixel and the camera or sensor.

[0033] Instance segmentation: Instance segmentation is a key task in computer vision, requiring models to identify different categories of objects in an image and perform pixel-level classification for each individual object. It aims to extract the precise boundaries or masks of specific objects from an image. For each type of object, instance segmentation algorithms not only need to determine their location and type but also need to generate a unique binary mask for each individual object to distinguish different instances within the same category.

[0034] Semantic segmentation is an image segmentation technique that aims to assign a category label to each pixel in an image, thereby enabling fine-grained understanding and analysis of the image. It assigns a semantic label to each pixel in an image, meaning each pixel is categorized into a specific object or category.

[0035] YOLO (You Only Look Once) is a deep learning-based object detection model. YOLO treats object detection as a single regression problem, predicting the location and class of all objects in an image through a single forward propagation. YOLO accepts a fixed-size image as input and extracts features from the input image through a series of convolutional and pooling layers. After feature extraction, fully connected or convolutional layers are used to predict the location and class of the objects. For each predicted object, YOLO outputs the coordinates of its bounding box, its confidence score, and its class probability.

[0036] Prior probability: refers to the initial estimate of the probability of an event occurring based on existing knowledge, experience, or historical data when no new data or evidence is available.

[0037] Robustness refers to the ability of a system, algorithm, or model to maintain its core functions, performance stability, and reliability when faced with uncertainties such as input disturbances, environmental changes, and parameter adjustments.

[0038] A game engine is a software framework specifically designed for developing and creating video games. It contains multiple core subsystems, such as graphics rendering, physics simulation, sound processing, input processing, and artificial intelligence. These subsystems work together to help developers create game scenes, characters, animations, sound effects, and more, thereby achieving various functions and effects in the game.

[0039] The depth buffer (also known as the Z-buffer or W-buffer) is one of the core technologies in computer graphics for handling visibility issues in 3D scenes. It determines the occlusion relationships between objects by storing the depth information of each pixel, thereby optimizing rendering efficiency and accuracy. The depth buffer is a memory area that stores the depth value of each pixel, typically with the same resolution as the frame buffer. Each pixel corresponds to a depth value (Z-value), representing its distance from the camera. The depth value range is generally from 0 to 1 (in OpenGL, 0 represents closest, and 1 represents farthest; in Direct3D, it may be the opposite).

[0040] YOLOv8n is a lightweight version of the YOLOv8 series, designed specifically for resource-constrained scenarios. Through structural optimization, parameter compression, and improved computational efficiency, it achieves fast inference while maintaining high detection accuracy.

[0041] Model pre-training is a crucial concept in deep learning, particularly in fields like Natural Language Processing (NLP) and Computer Vision (CV). It refers to the process of initially training a model using a large amount of unlabeled or labeled data before applying it to a specific task. The main purpose of pre-training is to allow the model to learn some general features or knowledge from the data, enabling it to converge faster and achieve better performance during subsequent fine-tuning for a specific task.

[0042] The inventors discovered through research that existing anomaly detection methods based on training deep models using pure images suffer from the following drawbacks: they require a large number of training samples to ensure model accuracy, resulting in high data collection and annotation costs; they require retraining for different game scenarios, leading to poor versatility; and the training and inference processes of deep learning models require significant computational resources, potentially making them unsuitable for low-end devices. This application proposes a multimodal anomaly detection scheme based on memory and images. By combining memory data and image information, and utilizing depth maps and semantic segmentation maps for multimodal integration, it improves the accuracy and efficiency of anomaly detection, overcoming many limitations of existing technologies.

[0043] Furthermore, existing rule-based anomaly detection methods have some limitations. These methods detect anomalies in game scenes by predefining a series of rules. For example, they calculate the edge features, contours, and depth information of objects in the game scene, and then combine the results of semantic segmentation and instance segmentation to determine whether an anomaly exists based on a threshold. However, these methods are prone to misjudgments due to dynamic objects and changes in lighting; the localization of anomaly regions is not precise enough to accurately pinpoint the cause of the problem; and the detection rules need to be constantly updated and maintained to adapt to different game scenes and anomalies. In contrast, the method in this application extracts depth maps and semantic segmentation maps from the image in memory and combines them with the detection results of the YOLO detection model. This allows for the automatic calculation of the probability of each type of anomaly occurring under different semantic categories, which is then integrated as a prior probability with the final identification result. This approach improves the accuracy of anomaly detection and exhibits good robustness. Moreover, the method in this application incorporates depth maps during the anomaly judgment stage to filter out objects that are too close, reducing false detections. Compared to methods that rely solely on rule detection, this method provides more precise localization and can accurately pinpoint the cause of the problem.

[0044] The following provides a detailed description of the scheme proposed in this application.

[0045] Please refer to Figure 1 , Figure 1 This is a schematic diagram of the structure of a game system provided in an embodiment of this application. The system includes a terminal 10 and a server 20, etc.; the terminal 10 and the server 20 are connected via a network, such as a wired or wireless network.

[0046] In cloud gaming, the game application and the game screen are separate entities. The storage and execution of the application are performed on server 20. The game screen is displayed on the cloud gaming client, such as terminal 10. The cloud gaming client primarily handles the reception and transmission of game data and the presentation of the game screen. For example, the cloud gaming client can be a display device with data transmission capabilities located close to the user, such as a mobile terminal, television, computer, PDA, or personal digital assistant. However, the terminal processing the game data is the cloud gaming server in the cloud. During gameplay, the user operates the cloud gaming client to send commands to the cloud gaming server. The cloud gaming server runs the game according to the commands, encodes and compresses the game screen data, returns it to the cloud gaming client via the network, and finally, the cloud gaming client decodes and outputs the game screen.

[0047] The solution proposed in this application can be applied to the testing of game graphics in cloud gaming.

[0048] This application provides an image anomaly testing method, which can be executed by a terminal or a server, or by both a terminal and a server. This application uses the execution of the image anomaly testing method by a server as an example for illustration.

[0049] Figure 2a This is a flowchart illustrating the image anomaly testing method provided in an embodiment of this application. The method includes the following steps S201-S204:

[0050] S201. Obtain the image to be tested;

[0051] Optionally, the solution in this application is applicable to the testing phase of game graphics testing. This testing phase involves testing the game graphics to check for any anomalies, and typically falls under the "usage testing" or "functional testing" stage of game testing. During this phase, testers closely monitor various aspects of the game, including but not limited to the game graphics.

[0052] The aforementioned image to be tested is one of the images tested during the testing phase.

[0053] Optionally, the content of the aforementioned test image is related to the applicable scenario of this solution. For example, when this solution is applied to the scenario of traffic accident image detection, the test image can be a real-time image or a historical image during the traffic accident scene recognition process.

[0054] S202. Input the image to be tested into the target anomaly detection model to obtain first predicted anomaly information. The first predicted anomaly information includes the border information of at least one first predicted anomaly region in the image to be tested and the predicted probability of at least one first predicted anomaly category to which each first predicted anomaly region belongs.

[0055] The target anomaly detection model is used to output the bounding box of the region with anomalies in the input image and the probability of each anomaly category.

[0056] Optionally, the aforementioned target anomaly detection model can be a YOLO model, specifically a lightweight YOLOv8n model. The YOLO model can quickly and accurately identify objects in an image and label their location and category.

[0057] Optionally, the aforementioned target anomaly detection model is trained based on multiple second sample anomaly images, which can be obtained through the interface provided by the game engine, specifically through the scene capture component.

[0058] Furthermore, regarding the training process of the target anomaly detection model, the above method also includes the following steps S001-S005:

[0059] S001. Obtain multiple second sample abnormal images and second annotation information corresponding to each second sample abnormal image. The second annotation information includes: border annotation information of at least one second sample abnormal region and probability annotation of at least one second sample abnormal category to which each second sample abnormal region belongs.

[0060] The second annotation information corresponding to the abnormal image of the second sample can be annotated by relevant personnel.

[0061] Among them, the border annotation information is the coordinates of the border of the annotation, and the probability annotation is the probability value of the annotation.

[0062] S002. Obtain a portion of the second sample abnormal images from the plurality of second sample abnormal images;

[0063] Optionally, a certain number of previously unobserved partial second sample abnormal images may be obtained from the plurality of second sample abnormal images.

[0064] S003. Input the partial second sample abnormal image into the initial abnormal detection model to obtain the second predicted abnormal information of each second sample abnormal image in the partial second sample abnormal image. The second predicted abnormal information of each second sample abnormal image includes: the bounding box prediction information of at least one second predicted abnormal region in the second sample abnormal image and the prediction probability of at least one second predicted abnormal category to which each second predicted abnormal region belongs.

[0065] Among them, the bounding box prediction information is the predicted bounding box coordinates, and the prediction probability is the predicted probability value.

[0066] S004. Determine the corresponding loss information based on the second predicted anomaly information of each second sample anomaly image in the partial second sample anomaly images and the second annotation information of each second sample anomaly image in the partial second sample anomaly images;

[0067] S005. When the loss information is less than a preset threshold, the most recently adjusted initial anomaly detection model is used as the target anomaly detection model. When the loss information is not less than the preset threshold, the parameters of the initial anomaly detection model are adjusted based on the loss information to obtain a new initial anomaly detection model. Then, the process returns to execute the aforementioned S002 until the target anomaly detection model is determined.

[0068] Specifically, when adjusting the parameters of the initial anomaly detection model based on the loss information, the parameters of the initial anomaly detection model are adjusted according to the principle of minimizing the loss information.

[0069] Optionally, the size of all the aforementioned second-sample abnormal images is uniformly scaled to a resolution of 960x540. This resolution, while preserving image detail, effectively reduces computational complexity and memory consumption, thereby improving the training speed and running efficiency of the model. Additionally, the data augmentation module built into YOLOv8n can be enabled. Data augmentation techniques include various methods such as rotation, scaling, cropping, color transformation, and random noise. These techniques can increase the diversity of training data, improve the model's generalization ability, and enable it to better identify various anomalies.

[0070] Optionally, the aforementioned initial anomaly detection model can be a pre-trained model, which enables the target anomaly detection model to converge quickly on a small dataset when trained, thereby improving the training efficiency of the model.

[0071] Optionally, the first predicted abnormal region in the image to be tested refers to the region predicted by the target abnormality detection model that includes abnormal image information. Optionally, the first predicted abnormal region may belong to at least one of the following abnormality categories: lighting abnormality, texture abnormality, clipping abnormality, floating abnormality, ground crack abnormality, and connection abnormality.

[0072] Among these, lighting anomalies refer to phenomena in games where lighting effects deviate from expectations or realistic physical laws due to technical, setting, or hardware issues. These manifest as image distortion, color deviation, and incorrect shadows. Texture anomalies in game images refer to phenomena where the surface textures (textures) of objects are displayed incorrectly due to technical, setting, or resource issues during gameplay. These manifest as missing textures, misalignment, color distortion, or dynamic anomalies. Clipping anomalies in game images refer to visual phenomena where objects, characters, or scene elements overlap or display unreasonably due to collision detection failure or rendering logic errors. These issues detract from the realism and immersion of the game visuals and are common in complex interactive scenes or high-precision modeling. Floating anomalies in game images refer to unreasonable floating phenomena of objects, characters, or scene elements in the game due to technical or logical errors. These manifest as models detaching from physical supports (such as the ground or platform) or incorrect position calculations during dynamic interactions. Screen tearing in game graphics refers to a visual misalignment caused by the asynchronous output frame rate of the graphics card and the refresh rate of the monitor. It manifests as horizontal or vertical breaks, misalignments, or layering in the image, like cracks in the ground. Frame stitching in game graphics refers to image anomalies with visible stitching lines. This typically refers to discontinuities, breaks, or misalignments that occur when different parts or elements of the game screen connect or transition during gameplay. Such anomalies can affect the overall coherence and visual effect of the game.

[0073] The bounding box information for the first predicted anomaly region is the coordinate information of the rectangular region occupied by the first predicted anomaly region in the test image. Specifically, the YOLO model divides the input image into multiple grids, and each grid is responsible for predicting targets falling within that grid. For each target, the model outputs one or more bounding boxes to locate the target. Each bounding box contains the following coordinate information:

[0074] x and y: Represent the coordinates of the center point of the border. The coordinates are usually normalized, that is, relative to the width and height of the image being tested. However, they can also be transformed as needed to obtain absolute coordinate values ​​(such as the coordinates of the top left or bottom right corner).

[0075] w and h: represent the width and height of the border. These values ​​can also be normalized or transformed to obtain absolute dimensions.

[0076] For each first predicted anomaly region, the first predicted anomaly region may belong to one or more first predicted anomaly categories. The output of the target anomaly detection model includes the predicted probability that the first predicted anomaly region belongs to each first predicted anomaly category.

[0077] In some optional embodiments of this application, if there are no abnormalities in the image to be tested, i.e., no abnormal regions are included, the first predicted abnormality information may be a prompt message indicating that there are no abnormalities, or the prediction probability of the border information of at least one first predicted abnormal region in the image to be tested and the prediction probability of at least one first predicted abnormal category to which each first predicted abnormal region belongs is 0.

[0078] S203. Perform semantic segmentation on the image to be tested to obtain a semantically segmented image;

[0079] Optionally, a semantic segmentation algorithm can be used to process the image to be tested to obtain a semantically segmented image.

[0080] Image semantic segmentation is an important task in computer vision. Its goal is to segment an input image into multiple regions and assign a semantic category label to each pixel, thereby indicating which object or region in the image the pixel belongs to.

[0081] Semantic segmentation maps segment various objects in the game screen (such as characters, buildings, props, etc.) and label the category to which each pixel belongs.

[0082] In this application, the aforementioned S203, which involves semantic segmentation of the image to be tested to obtain a semantically segmented image, may include the following steps S2031-S2033:

[0083] S2031. Assign a unique color to each type of element in at least one type of element in the image to be tested, and obtain the color assignment information of each type of element;

[0084] Optionally, in this process, each type of element in the game screen (such as grass, trees, rocks, buildings, etc.) is first assigned a unique color. This "unique color" is a marker used to identify and distinguish different instances of game elements. This color assignment is typically done in the game engine or graphics processing software, allowing relevant personnel to easily identify each element in subsequent processing.

[0085] S2032. For the regions occupied by each type of element in the image to be tested, a shader is used to color the regions according to the color allocation information to obtain the colored image of the image to be tested.

[0086] A shader is a key component in the graphics rendering pipeline. It is responsible for processing vertex and pixel data to generate the final rendered image.

[0087] The unique color assigned to a game element will be used in the shader to distinguish different object instances. This means that the shader will determine how to render the element based on its color attributes. The purpose of the shader is to convert the color attributes of the game element into the corresponding pixel color, i.e., the assigned color information.

[0088] The aforementioned shader can be a custom shader, which allows relevant personnel to customize the rendering process according to specific needs, such as changing the appearance of elements or applying special effects.

[0089] S2033. Obtain a semantic segmentation image based on the colored image.

[0090] Specifically, the colored image can be used as a semantic segmentation image.

[0091] The color of each pixel in the semantically segmented image corresponds to a specific element in the game.

[0092] In this way, relevant personnel can easily obtain semantic information about the game scene, thereby realizing various advanced functions, such as object recognition and interaction logic.

[0093] S204. Based on the semantic segmentation image and the first predicted anomaly information, determine at least one target anomaly region in the image to be tested from the at least one first predicted anomaly region.

[0094] In some optional embodiments of this application, in the aforementioned S204, determining at least one target anomaly region in the image to be tested from at least one first predicted anomaly region based on the semantic segmentation image and the first predicted anomaly information includes the following steps S2041-S2043:

[0095] S2041. Based on the bounding box information of at least one first predicted abnormal region, match the image to be tested with the semantic segmentation image to obtain the target semantic category corresponding to each first predicted abnormal region.

[0096] Specifically, when matching the image to be tested with the semantic segmentation image, the bounding box of each first predicted abnormal region will be matched with the corresponding region bounding box in the semantic segmentation image. For each first predicted abnormal region, the semantic category in the corresponding region of the semantic segmentation image to which its bounding box is matched can be used as the target semantic category corresponding to the first predicted abnormal region.

[0097] Specifically, the region in the semantic segmentation image to which the bounding box of the first predicted anomaly region is matched refers to the image region within the bounding box of the region corresponding to the bounding box of the first predicted anomaly region in the semantic segmentation image. The semantic category in this image region can refer to the semantic category to which the most pixels in the image region belong. Specifically, each pixel in the image region has a semantic category, and the semantic category to which the most pixels in the image region belong can be used as the target semantic category corresponding to the first predicted anomaly region.

[0098] S2042. For each of the first predicted abnormal regions, the target score information corresponding to the first predicted abnormal region is determined based on the prediction probability of at least one first predicted abnormal category to which the first predicted abnormal region belongs and the target semantic category corresponding to the first predicted abnormal region; thereby obtaining at least one target score information corresponding to the at least one first predicted abnormal region.

[0099] In some optional embodiments of this application, in the aforementioned S2042, determining the target score information corresponding to the first predicted anomaly region based on the prediction probability of at least one first predicted anomaly category to which the first predicted anomaly region belongs and the target semantic category corresponding to the first predicted anomaly region includes the following S421-S422:

[0100] S421. For each of the first predicted anomaly categories in at least one first predicted anomaly category to which the first predicted anomaly region belongs, obtain the target probability of the semantic category corresponding to the first predicted anomaly category being the target semantic category corresponding to the first predicted anomaly region. Based on the predicted probability of the first predicted anomaly region belonging to the first predicted anomaly category and the target probability, determine the anomaly score corresponding to the first predicted anomaly category, and then obtain at least one anomaly score corresponding to the at least one first predicted anomaly category.

[0101] Optionally, determining the anomaly score corresponding to the first predicted anomaly category based on the predicted probability that the first predicted anomaly region belongs to the first predicted anomaly category and the target probability may include: taking the product of the predicted probability that the first predicted anomaly region belongs to the first predicted anomaly category and the target probability as the anomaly score corresponding to the first predicted anomaly category.

[0102] The aforementioned target probability is a prior probability. Integrating the prior probability with the results identified by the YOLO model can improve the accuracy of anomaly detection. The introduction of prior probability allows for more accurate identification of target anomaly regions in subsequent anomaly detection.

[0103] The target probability of the semantic category corresponding to the first predicted anomaly category and the target semantic category corresponding to the first predicted anomaly region can be calculated in advance and stored locally.

[0104] Regarding the calculation method for the target probability that the semantic category corresponding to the first predicted anomaly category is the target semantic category corresponding to the first predicted anomaly region, in some optional embodiments of this application, the method further includes the following steps S1-S3:

[0105] S1. For each of the multiple first sample abnormal images, determine the first annotation information corresponding to the first sample abnormal image. The first annotation information includes: border information of at least one first sample abnormal region, the first sample abnormal category to which each first sample abnormal region belongs, and the sample semantic category corresponding to each first sample abnormal region. The first sample abnormal category to which the same first sample abnormal region belongs corresponds to the corresponding sample semantic category.

[0106] The first sample anomaly category may include at least one of the following anomaly categories: lighting anomaly, texture anomaly, clipping anomaly, floating anomaly, ground crack anomaly, and connection anomaly.

[0107] The first annotation information corresponding to each first sample abnormal image is regarded as a set of first annotation information.

[0108] S2. Determine at least one sample semantic category involved in the at least one first sample abnormal region and the at least one first sample abnormal category involved in the at least one first sample abnormal region based on multiple sets of first annotation information corresponding to the multiple first sample abnormal images.

[0109] The at least one sample semantic category involved in the at least one first sample anomaly region refers to at least one type of sample semantic category involved in the at least one first sample anomaly region. The number of first sample anomaly regions in the at least one first sample anomaly region is not greater than the number of sample semantic categories in the at least one sample semantic category. For example, two different first sample anomaly regions may involve the same sample semantic category. The first sample anomaly region involving a sample semantic category means that the semantic category of the first sample anomaly region is that sample semantic category.

[0110] The at least one first sample anomaly region involving at least one first sample anomaly category refers to at least one type of first sample anomaly category involved in the at least one first sample anomaly region. The number of first sample anomaly regions in at least one first sample anomaly region is not greater than the number of first sample anomaly categories in at least one first sample anomaly category. For example, two different first sample anomaly regions may involve the same first sample anomaly category. A first sample anomaly region involving a first sample anomaly category means that the anomaly category to which the first sample anomaly region belongs is that first sample anomaly category.

[0111] S3. For each of the at least one first sample anomaly categories, based on the multiple sets of first annotation information, determine the semantic category probability information corresponding to the first sample anomaly category. The semantic category probability information includes the probability that the semantic category corresponding to the first sample anomaly category is each of the at least one sample semantic categories.

[0112] In some optional embodiments of this application, obtaining the target probability that the semantic category corresponding to the first predicted anomaly category is the target semantic category corresponding to the first predicted anomaly region includes: obtaining a target sample anomaly category that is the same as the first predicted anomaly category from the at least one first sample anomaly category; and determining the target probability that the semantic category corresponding to the first predicted anomaly category is the target semantic category corresponding to the first predicted anomaly region based on the semantic category probability information corresponding to the target sample anomaly category.

[0113] In some optional embodiments of this application, determining the target probability of the semantic category corresponding to the first predicted anomaly category as the target semantic category corresponding to the first predicted anomaly region based on the semantic category probability information corresponding to the target sample anomaly category includes the following steps S21-S22:

[0114] S21. Obtain the target sample semantic category that is the same as the target semantic category corresponding to the first predicted anomaly region from the at least one sample semantic category;

[0115] S22. The probability that the semantic category corresponding to the anomaly category of the target sample is the target sample anomaly category is taken from the semantic category probability information corresponding to the anomaly category of the target sample as the target probability that the semantic category corresponding to the first predicted anomaly category is the target semantic category corresponding to the first predicted anomaly region.

[0116] In some optional embodiments of this application, in the aforementioned S3, determining the semantic category probability information corresponding to the anomaly category of the first sample based on the plurality of sets of first annotation information includes the following steps S31-S32:

[0117] S31. For each sample semantic category in the at least one sample semantic category, determine the number of first sample anomaly categories in the multiple sets of first annotation information whose corresponding semantic category is the sample semantic category, and obtain at least one number corresponding to the at least one sample semantic category for the first sample anomaly category;

[0118] S32. Determine the semantic category probability information corresponding to the first sample anomaly category based on the at least one quantity.

[0119] In some optional embodiments of this application, step S32, determining the semantic category probability information corresponding to the first sample anomaly category based on the at least one quantity, includes the following steps S321-S323:

[0120] S321. Summing the at least one quantity to obtain the total quantity;

[0121] S322. For each sample semantic category, determine the ratio of the number corresponding to the sample semantic category to the total number; use the ratio as the probability that the semantic category corresponding to the first sample anomaly category is the sample semantic category, and then obtain at least one probability that the semantic category corresponding to the first sample anomaly category is the at least one sample semantic category.

[0122] S323. The at least one probability is used as the semantic category probability information corresponding to the anomaly category of the first sample.

[0123] In some optional embodiments of this application, in order to avoid probability calculation deviations for certain sample semantic categories due to the small number of first samples, a preset value can be added to the number of sample semantic categories corresponding to each first sample abnormal category. Optionally, this value can be 10.

[0124] S422. Determine the target score information corresponding to the first predicted anomaly region based on at least one anomaly score corresponding to at least one first predicted anomaly category.

[0125] Optionally, in S422, determining the target score information corresponding to the first predicted anomaly region based on at least one anomaly score corresponding to at least one first predicted anomaly category includes: taking the highest score among the at least one anomaly score corresponding to at least one first predicted anomaly category as the target score information corresponding to the first predicted anomaly region.

[0126] S2043. Determine at least one target anomaly region in the image to be tested from the at least one first predicted anomaly region based on the at least one target score information.

[0127] In some optional embodiments of this application, in the aforementioned S2043, determining at least one target anomaly region in the image to be tested from the at least one first predicted anomaly region based on the at least one target score information includes the following steps S431-S432:

[0128] S431. Delete the first prediction anomaly region in the at least one first prediction anomaly region whose target score information is less than a preset score, and obtain the remaining first prediction anomaly region.

[0129] S432. The remaining first predicted abnormal region is taken as at least one target abnormal region in the image to be tested.

[0130] Optionally, the aforementioned preset score can be set by relevant personnel, for example, it can be 0.05. Removing detection boxes with target scores lower than this preset score can effectively filter out possible false detections, thereby improving the overall detection accuracy.

[0131] In some optional embodiments of this application, before determining at least one target anomaly region in the test image from the at least one first predicted anomaly region based on the semantic segmentation image and the first predicted anomaly information in the aforementioned S204, the method further includes preprocessing the at least one first predicted anomaly region, wherein the preprocessing of the at least one first predicted anomaly region includes the following S01-S02:

[0132] S01. Obtain the depth map information of the image to be tested;

[0133] S02. Based on the depth map information, at least one first predicted abnormal region is filtered to obtain at least one filtered first predicted abnormal region.

[0134] The semantic segmentation image and depth map information in this application can be obtained from memory, and the depth map information can be obtained from the depth buffer. The semantic segmentation image and depth map information refer to the multimodal data of the image to be tested. The depth map information can be used to filter out objects that are too close, so as to reduce false detections and improve detection efficiency.

[0135] Specifically, the aforementioned semantic segmentation map and depth map information can be obtained using the interface provided by the game engine.

[0136] This application, by comprehensively utilizing the YOLO detection model, depth map information, and semantic segmentation map, can accurately identify and locate abnormal regions, thereby improving game stability and user experience.

[0137] Optionally, the aforementioned depth map information can refer to a depth map. During the process of acquiring the depth map of the image to be tested, a virtual camera needs to be created. The position and viewpoint of this virtual camera will be consistent with the current game camera. Then, the UE (Unreal Engine) interface is called to use this virtual camera to capture the depth information of the game scene. During this process, the UE interface reads the depth buffer to obtain the depth value of each pixel in the camera space.

[0138] Optionally, in the aforementioned S02, filtering the at least one first predicted anomaly region based on the depth map information to obtain at least one filtered first predicted anomaly region includes:

[0139] For each of the at least one first predicted abnormal region, the depth value of the center of the first predicted abnormal region is determined based on the depth map information. If the depth value is less than a preset depth value, the first predicted abnormal region is filtered out from the at least one first predicted abnormal region, thus obtaining at least one filtered first predicted abnormal region. Optionally, the aforementioned preset depth value can be 3 meters.

[0140] This method avoids false detections caused by being too close to the camera, thus improving detection accuracy. For example, being too close can easily result in scenes that are too dark or have stretched textures, which are actually normal.

[0141] Furthermore, the flowchart for image anomaly testing methods can also be found in [reference needed]. Figure 2b The steps S301-S304 are shown in the previous content. For details, please refer to the above content. They will not be repeated here.

[0142] The main benefits of this solution lie in improving the efficiency and quality of game testing. By automatically identifying and locating abnormal areas in game scenes, the workload of manual testing can be greatly reduced. Furthermore, by combining depth information, semantic information, and model prediction information, abnormal areas can be identified and located more accurately, thereby improving the quality and efficiency of game testing, enhancing game stability, and improving user experience.

[0143] This application embodiment obtains a test image; inputs the test image into a target anomaly detection model to obtain first predicted anomaly information, the first predicted anomaly information including the border information of at least one first predicted anomaly region in the test image and the predicted probability of at least one first predicted anomaly category to which each first predicted anomaly region belongs; performs semantic segmentation on the test image to obtain a semantic segmentation image; and determines at least one target anomaly region in the test image from the at least one first predicted anomaly region based on the semantic segmentation image and the first predicted anomaly information. This scheme combines the anomaly prediction information of the test image output by the detection model with the semantic segmentation image of the test image to detect the target anomaly region of the test image, using multi-dimensional information. The calculation process is simple, and the target anomaly region in the test image is determined by a concise calculation, which improves the accuracy of anomaly region determination and saves computing resources.

[0144] All of the above technical solutions can be combined in any way to form optional embodiments of this application, and will not be described in detail here.

[0145] To facilitate better implementation of the image anomaly testing method of this application, this application also provides an image anomaly testing apparatus. Please refer to [link to relevant documentation]. Figure 3 , Figure 3 This is a schematic diagram of the image anomaly testing device provided in an embodiment of this application. The device includes:

[0146] Acquisition unit 31 is used to acquire the image to be tested;

[0147] Input unit 32 is used to input the image to be tested into the target anomaly detection model to obtain first predicted anomaly information. The first predicted anomaly information includes the border information of at least one first predicted anomaly region in the image to be tested and the predicted probability of at least one first predicted anomaly category to which each first predicted anomaly region belongs.

[0148] Segmentation unit 33 is used to perform semantic segmentation on the image to be tested to obtain a semantically segmented image;

[0149] The determining unit 34 is used to determine at least one target anomaly region in the image to be tested from the at least one first predicted anomaly region based on the semantic segmentation image and the first predicted anomaly information.

[0150] In some optional embodiments of this application, before determining at least one target anomaly region in the image to be tested from the at least one first predicted anomaly region based on the semantic segmentation image and the first predicted anomaly information, the apparatus is further configured to preprocess the at least one first predicted anomaly region, wherein the preprocessing of the at least one first predicted anomaly region includes:

[0151] Obtain the depth map information of the image to be tested;

[0152] The at least one first predicted anomaly region is filtered based on the depth map information to obtain at least one filtered first predicted anomaly region.

[0153] In some optional embodiments of this application, when the aforementioned apparatus is used to determine at least one target anomaly region in the image to be tested from at least one first predicted anomaly region based on the semantic segmentation image and the first predicted anomaly information, it is specifically used for:

[0154] Based on the bounding box information of at least one first predicted anomaly region, the image to be tested is matched with the semantic segmentation image to obtain the target semantic category corresponding to each first predicted anomaly region;

[0155] For each of the first predicted abnormal regions, the target score information corresponding to the first predicted abnormal region is determined based on the prediction probability of at least one first predicted abnormal category to which the first predicted abnormal region belongs and the target semantic category corresponding to the first predicted abnormal region; thereby obtaining at least one target score information corresponding to the at least one first predicted abnormal region.

[0156] At least one target anomaly region in the image to be tested is determined from at least one first predicted anomaly region based on the at least one target score information.

[0157] In some optional embodiments of this application, when the aforementioned apparatus is used to determine the target score information corresponding to the first predicted anomaly region based on the prediction probability of at least one first predicted anomaly category to which the first predicted anomaly region belongs and the target semantic category corresponding to the first predicted anomaly region, it is specifically used for:

[0158] For each of the first predicted anomaly categories in at least one first predicted anomaly category to which the first predicted anomaly region belongs, the target probability of the semantic category corresponding to the first predicted anomaly category as the target semantic category corresponding to the first predicted anomaly region is obtained. Based on the predicted probability of the first predicted anomaly region belonging to the first predicted anomaly category and the target probability, the anomaly score corresponding to the first predicted anomaly category is determined, thereby obtaining at least one anomaly score corresponding to the at least one first predicted anomaly category.

[0159] The target score information corresponding to the first predicted anomaly region is determined based on at least one anomaly score corresponding to at least one first predicted anomaly category.

[0160] In some optional embodiments of this application, when the aforementioned device is used to determine the target score information corresponding to the first predicted anomaly region based on at least one anomaly score corresponding to at least one first predicted anomaly category, it is specifically used to: take the highest score among the at least one anomaly score corresponding to at least one first predicted anomaly category as the target score information corresponding to the first predicted anomaly region.

[0161] In some optional embodiments of this application, when the aforementioned apparatus is used to determine at least one target anomaly region in the image to be tested from the at least one first predicted anomaly region based on the at least one target score information, it is specifically used for:

[0162] Delete the first prediction anomaly regions in the at least one first prediction anomaly region whose target score information is less than a preset score, and obtain the remaining first prediction anomaly regions.

[0163] The remaining first predicted abnormal region is taken as at least one target abnormal region in the image to be tested.

[0164] In some optional embodiments of this application, the aforementioned apparatus is further used for:

[0165] For each of the multiple first sample abnormal images, determine the first annotation information corresponding to the first sample abnormal image. The first annotation information includes: at least one first sample abnormal region, the first sample abnormal category to which each first sample abnormal region belongs, and the sample semantic category corresponding to each first sample abnormal region. The first sample abnormal category to which the same first sample abnormal region belongs corresponds to the corresponding sample semantic category.

[0166] Based on multiple sets of first annotation information corresponding to the multiple first sample abnormal images, determine at least one sample semantic category involved in the at least one first sample abnormal region, and at least one first sample abnormal category involved in the at least one first sample abnormal region;

[0167] For each of the at least one first sample anomaly categories, based on the multiple sets of first annotation information, determine the semantic category probability information corresponding to the first sample anomaly category. The semantic category probability information includes the probability that the semantic category corresponding to the first sample anomaly category is each of the at least one sample semantic categories.

[0168] Obtaining the target probability that the semantic category corresponding to the first predicted anomaly category is the target semantic category corresponding to the first predicted anomaly region includes: obtaining a target sample anomaly category that is the same as the first predicted anomaly category from the at least one first sample anomaly category; and determining the target probability that the semantic category corresponding to the first predicted anomaly category is the target semantic category corresponding to the first predicted anomaly region based on the semantic category probability information corresponding to the target sample anomaly category.

[0169] In some optional embodiments of this application, when the aforementioned apparatus is used to determine the target probability that the semantic category corresponding to the first predicted anomaly category is the target semantic category corresponding to the first predicted anomaly region based on the semantic category probability information corresponding to the target sample anomaly category, it is specifically used for:

[0170] Obtain the target sample semantic category that is the same as the target semantic category corresponding to the first predicted anomaly region from the at least one sample semantic category;

[0171] The probability that the semantic category corresponding to the anomaly category of the target sample is the target anomaly category is taken from the semantic category probability information corresponding to the anomaly category of the target sample as the target probability that the semantic category corresponding to the first predicted anomaly category is the target semantic category corresponding to the first predicted anomaly region.

[0172] In some optional embodiments of this application, when the aforementioned device is used to determine the semantic category probability information corresponding to the anomaly category of the first sample based on the plurality of sets of first annotation information, it is specifically used for:

[0173] For each sample semantic category in the at least one sample semantic category, determine the number of first sample anomaly categories in the multiple sets of first annotation information whose corresponding semantic category is the sample semantic category, and obtain at least one number corresponding to the at least one sample semantic category for the first sample anomaly category;

[0174] The semantic category probability information corresponding to the first sample anomaly category is determined based on the at least one quantity.

[0175] In some optional embodiments of this application, when the aforementioned apparatus is used to determine the semantic category probability information corresponding to the first sample anomaly category based on the at least one quantity, it is specifically used for:

[0176] Summing the at least one quantity yields the total quantity;

[0177] For each sample semantic category, determine the ratio of the number corresponding to the sample semantic category to the total number; use the ratio as the probability that the semantic category corresponding to the first sample anomaly category is the sample semantic category, and then obtain at least one probability that the semantic category corresponding to the first sample anomaly category is the at least one sample semantic category.

[0178] The at least one probability is used as the semantic category probability information corresponding to the anomaly category of the first sample.

[0179] Each unit of the aforementioned image anomaly testing device can be implemented entirely or partially through software, hardware, or a combination thereof. Each unit can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the corresponding operations of each unit.

[0180] In this application embodiment, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0181] The aforementioned image anomaly testing device can be integrated into a terminal or server that has storage and a processor and thus computing power, or the image anomaly testing device can be the terminal or server itself.

[0182] Optionally, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0183] Figure 4 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. The computer device may be... Figure 1 The terminal or server shown. For example... Figure 4As shown, the computer device 400 may include: a communication interface 401, a memory 402, a processor 403, and a communication bus 404. The communication interface 401, memory 402, and processor 403 communicate with each other via the communication bus 404. The communication interface 401 is used for data communication between the computer device 400 and external devices. The memory 402 can be used to store software programs and modules, and the processor 403 runs the software programs and modules stored in the memory 402, such as the software programs for the corresponding operations in the foregoing method embodiments.

[0184] Optionally, the processor 403 can invoke software programs and modules stored in the memory 402 to perform the following operations:

[0185] Obtain the image to be tested;

[0186] The image to be tested is input into the target anomaly detection model to obtain first predicted anomaly information. The first predicted anomaly information includes the border information of at least one first predicted anomaly region in the image to be tested and the predicted probability of at least one first predicted anomaly category to which each first predicted anomaly region belongs.

[0187] The image to be tested is semantically segmented to obtain a semantically segmented image;

[0188] Based on the semantic segmentation image and the first predicted anomaly information, at least one target anomaly region in the image to be tested is determined from the at least one first predicted anomaly region.

[0189] This application also provides a computer-readable storage medium for storing a computer program. This computer-readable storage medium can be applied to a computer device, and the computer program causes the computer device to execute the corresponding processes in the methods described in the embodiments of this application; for brevity, these will not be elaborated further here.

[0190] This application also provides a computer program product including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the corresponding processes in the methods described in the embodiments of this application. For brevity, these details will not be elaborated further here.

[0191] This application also provides a computer program that includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the corresponding processes in the methods described in the embodiments of this application. For the sake of brevity, these details will not be elaborated further here.

[0192] It should be understood that the processor in the embodiments of this application may be an integrated circuit chip with signal processing capabilities. In implementation, the steps of the above method embodiments can be completed by integrated logic circuits in the processor's hardware or by instructions in software form. The processor described above can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules can be located in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The storage medium is located in memory, and the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method.

[0193] It is understood that the memory in the embodiments of this application can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDR SDRAM), Enhanced Synchronous DRAM (ESDRAM), Synchlink DRAM (SLDRAM), and Direct Rambus RAM (DR RAM). It should be noted that the memory used in the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0194] It should be understood that the above-described memory is exemplary and not a limiting description. For example, the memory in the embodiments of this application may also be static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct memory bus RAM (DR RAM), etc. That is to say, the memory in the embodiments of this application is intended to include, but is not limited to, these and any other suitable types of memory.

[0195] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0196] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0197] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0198] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0199] In addition, the functional units in the embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0200] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer or a server) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, ROM, RAM, magnetic disks, or optical disks.

[0201] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for testing image anomalies, characterized in that, The method includes: Obtain the image to be tested; The image to be tested is input into the target anomaly detection model to obtain first predicted anomaly information. The first predicted anomaly information includes the border information of at least one first predicted anomaly region in the image to be tested and the predicted probability of at least one first predicted anomaly category to which each first predicted anomaly region belongs. The image to be tested is semantically segmented to obtain a semantically segmented image; Based on the semantic segmentation image and the first predicted anomaly information, at least one target anomaly region in the image to be tested is determined from the at least one first predicted anomaly region.

2. The method according to claim 1, characterized in that, Before determining at least one target anomaly region in the test image from the at least one first predicted anomaly region based on the semantic segmentation image and the first predicted anomaly information, the method further includes preprocessing the at least one first predicted anomaly region, the preprocessing of the at least one first predicted anomaly region including: Obtain the depth map information of the image to be tested; The at least one first predicted anomaly region is filtered based on the depth map information to obtain at least one filtered first predicted anomaly region.

3. The method according to claim 1, characterized in that, Determining at least one target anomaly region in the image to be tested from at least one first predicted anomaly region based on the semantic segmentation image and the first predicted anomaly information includes: Based on the bounding box information of at least one first predicted anomaly region, the image to be tested is matched with the semantic segmentation image to obtain the target semantic category corresponding to each first predicted anomaly region; For each of the first predicted abnormal regions, the target score information corresponding to the first predicted abnormal region is determined based on the prediction probability of at least one first predicted abnormal category to which the first predicted abnormal region belongs and the target semantic category corresponding to the first predicted abnormal region; thereby obtaining at least one target score information corresponding to the at least one first predicted abnormal region. At least one target anomaly region in the image to be tested is determined from at least one first predicted anomaly region based on the at least one target score information.

4. The method according to claim 3, characterized in that, The step of determining the target score information corresponding to the first predicted anomaly region based on the prediction probability of at least one first predicted anomaly category to which the first predicted anomaly region belongs and the target semantic category corresponding to the first predicted anomaly region includes: For each of the first predicted anomaly categories in at least one first predicted anomaly category to which the first predicted anomaly region belongs, the target probability of the semantic category corresponding to the first predicted anomaly category as the target semantic category corresponding to the first predicted anomaly region is obtained. Based on the predicted probability of the first predicted anomaly region belonging to the first predicted anomaly category and the target probability, the anomaly score corresponding to the first predicted anomaly category is determined, thereby obtaining at least one anomaly score corresponding to the at least one first predicted anomaly category. The target score information corresponding to the first predicted anomaly region is determined based on at least one anomaly score corresponding to at least one first predicted anomaly category.

5. The method according to claim 4, characterized in that, Determining the target score information corresponding to the first predicted anomaly region based on at least one anomaly score corresponding to at least one first predicted anomaly category includes: taking the highest score among the at least one anomaly scores corresponding to at least one first predicted anomaly category as the target score information corresponding to the first predicted anomaly region.

6. The method according to claim 3, characterized in that, Determining at least one target anomaly region in the image to be tested from at least one first predicted anomaly region based on the at least one target score information includes: Delete the first prediction anomaly regions in the at least one first prediction anomaly region whose target score information is less than a preset score, and obtain the remaining first prediction anomaly regions. The remaining first predicted abnormal region is taken as at least one target abnormal region in the image to be tested.

7. The method according to claim 4, characterized in that, The method further includes: For each of the multiple first sample abnormal images, determine the first annotation information corresponding to the first sample abnormal image. The first annotation information includes: at least one first sample abnormal region, the first sample abnormal category to which each first sample abnormal region belongs, and the sample semantic category corresponding to each first sample abnormal region. The first sample abnormal category to which the same first sample abnormal region belongs corresponds to the corresponding sample semantic category. Based on multiple sets of first annotation information corresponding to the multiple first sample abnormal images, determine at least one sample semantic category involved in the at least one first sample abnormal region, and at least one first sample abnormal category involved in the at least one first sample abnormal region; For each of the at least one first sample anomaly categories, based on the multiple sets of first annotation information, determine the semantic category probability information corresponding to the first sample anomaly category. The semantic category probability information includes the probability that the semantic category corresponding to the first sample anomaly category is each of the at least one sample semantic categories. Obtaining the target probability that the semantic category corresponding to the first predicted anomaly category is the target semantic category corresponding to the first predicted anomaly region includes: obtaining a target sample anomaly category that is the same as the first predicted anomaly category from the at least one first sample anomaly category; and determining the target probability that the semantic category corresponding to the first predicted anomaly category is the target semantic category corresponding to the first predicted anomaly region based on the semantic category probability information corresponding to the target sample anomaly category.

8. The method according to claim 7, characterized in that, The step of determining the target probability of the semantic category corresponding to the first predicted anomaly category as the target semantic category corresponding to the first predicted anomaly region based on the semantic category probability information corresponding to the anomaly category of the target sample includes: Obtain the target sample semantic category that is the same as the target semantic category corresponding to the first predicted anomaly region from the at least one sample semantic category; The probability that the semantic category corresponding to the anomaly category of the target sample is the target anomaly category is taken from the semantic category probability information corresponding to the anomaly category of the target sample as the target probability that the semantic category corresponding to the first predicted anomaly category is the target semantic category corresponding to the first predicted anomaly region.

9. The method according to claim 7, characterized in that, The step of determining the semantic category probability information corresponding to the anomaly category of the first sample based on the multiple sets of first annotation information includes: For each sample semantic category in the at least one sample semantic category, determine the number of first sample anomaly categories in the multiple sets of first annotation information whose corresponding semantic category is the sample semantic category, and obtain at least one number corresponding to the at least one sample semantic category for the first sample anomaly category; The semantic category probability information corresponding to the first sample anomaly category is determined based on the at least one quantity.

10. The method according to claim 9, characterized in that, Determining the semantic category probability information corresponding to the first sample anomaly category based on the at least one quantity includes: Summing the at least one quantity yields the total quantity; For each sample semantic category, determine the ratio of the number corresponding to the sample semantic category to the total number; use the ratio as the probability that the semantic category corresponding to the first sample anomaly category is the sample semantic category, and then obtain at least one probability that the semantic category corresponding to the first sample anomaly category is the at least one sample semantic category. The at least one probability is used as the semantic category probability information corresponding to the anomaly category of the first sample.

11. An image anomaly testing device, characterized in that, The device includes: The acquisition unit is used to acquire the image to be tested; The input unit is used to input the image to be tested into the target anomaly detection model to obtain first predicted anomaly information. The first predicted anomaly information includes the border information of at least one first predicted anomaly region in the image to be tested and the predicted probability of at least one first predicted anomaly category to which each first predicted anomaly region belongs. A segmentation unit is used to perform semantic segmentation on the image to be tested to obtain a semantically segmented image; The determining unit is configured to determine at least one target anomaly region in the image to be tested from the at least one first predicted anomaly region based on the semantic segmentation image and the first predicted anomaly information.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program adapted for loading by a processor to perform the steps of the image anomaly testing method as described in any one of claims 1-10.

13. A computer device, characterized in that, The computer device includes a processor and a memory, the memory storing a computer program, and the processor executes the steps of the image anomaly testing method according to any one of claims 1-10 by calling the computer program stored in the memory.

14. A computer program product comprising computer instructions, characterized in that, When the computer instructions are executed by the processor, they implement the steps in the image anomaly testing method according to any one of claims 1-10.

Images