Abnormal image detection method, device, electronic device and storage medium
By adding problem code to the application package to generate abnormal image samples, and combining normal image samples to train the detection model, the problem of insufficient number and diversity of abnormal image samples is solved, and the accuracy of abnormal image detection and model accuracy are improved.
Patent Information
- Application Number
- CN202210514120.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-05-11
- Publication Date
- 2024-11-08
- Estimated Expiration
- 2042-05-11
AI Technical Summary
In the prior art, the number of abnormal image samples is small and the data diversity is insufficient, resulting in classifier classification errors, affecting the detection accuracy of abnormal images.
By adding problem code to the application package, the abnormal image samples are obtained, and the detection model is trained in combination with normal image samples to improve the data volume and diversity of abnormal image samples, thereby improving the accuracy of the object detection model.
It improves the accuracy of abnormal image detection, enhances the accuracy of the object detection model, and can more effectively identify and label abnormal image problems.
Smart Images

Figure CN114821240B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of computer technology, and in particular to a method, device, electronic device and storage medium for detecting abnormal images. Background Art
[0002] Before a new version of an application (Application, APP) is launched, testers usually need to test the graphical user interface of the application to determine whether the application has display anomalies. Taking a game application as an example, before a new version of a game application is launched, testers need to test whether the game screen in the game application is abnormal, for example, whether there is a black screen or a distorted screen, abnormal display of virtual elements, abnormal display of skill special effects, etc.
[0003] In related technologies, more and more mobile application manufacturers are introducing artificial intelligence algorithms (AI) into their daily workflows. Most manufacturers will collect certain image data, use neural networks to learn the differences between these normal / abnormal image samples, and analyze the implicit rules to obtain a classifier that determines whether the interface display is normal. However, the number of existing abnormal image samples is small, and the existing process of constructing abnormal image samples is too empirical, resulting in insufficient data diversity of abnormal image samples. As a result, due to the small number of abnormal image samples and insufficient data diversity, it is easy to cause classifier classification errors, thereby affecting the accuracy of abnormal images. Summary of the invention
[0004] In view of this, the purpose of the present application is to provide a method, device, electronic device and storage medium for detecting abnormal images, so as to improve the accuracy of detecting abnormal images by a target detection model by increasing the data volume and data diversity of abnormal image samples.
[0005] In a first aspect, an embodiment of the present application provides a method for detecting abnormal images, the method comprising:
[0006] Acquire an abnormal image sample, where the abnormal image sample is an image obtained by rendering a second application package, where the second application package is a package obtained by adding a problem code to the first application package;
[0007] Acquire a normal image sample, where the normal image sample is an image obtained by rendering the first application package;
[0008] Based on the abnormal image samples and the normal image samples, an initial detection model is trained to obtain a trained target detection model, wherein the target detection model is used to output the probability that the input image is a normal image or an abnormal image.
[0009] In an optional embodiment of the present application, the problem code is a code for generating a target image problem, and the target image problem is a problem related to a virtual camera, wherein the virtual camera is used to capture an image of a virtual scene.
[0010] In an optional embodiment of the present application, the target image problem includes at least one of the following:
[0011] Issues with setting the cache flag of the virtual camera, shutting down the virtual camera, and screen post-processing.
[0012] In an optional embodiment of the present application, the obtaining of abnormal image samples includes: obtaining images of different virtual scenes of the second application package;
[0013] The obtaining of normal image samples includes: obtaining images of different virtual scenes of the first application package.
[0014] In an optional embodiment of the present application, the obtaining of abnormal image samples includes:
[0015] Taking multiple screenshots of the image obtained by rendering the second application package to obtain multiple abnormal image frames;
[0016] The multiple abnormal image frames are screened to obtain abnormal image samples.
[0017] In an optional embodiment of the present application, screening the plurality of abnormal image frames to obtain abnormal image samples includes:
[0018] For any target abnormal image frame among the multiple abnormal image frames, the target abnormal image frame is compared with an adjacent previous abnormal image frame for similarity to obtain a similarity;
[0019] If the similarity is greater than a preset threshold, the target abnormal image frame is deleted from the multiple abnormal image frames, and the abnormal image sample is composed of the remaining abnormal image frames.
[0020] In an optional embodiment of the present application, screening the plurality of abnormal image frames to obtain abnormal image samples includes:
[0021] Based on a preset interval frame number, starting from any abnormal image frame of the plurality of abnormal image frames, a plurality of target abnormal image frames are screened to obtain the abnormal image samples, and the plurality of target abnormal image frames constitute the abnormal image samples.
[0022] In an optional embodiment of the present application, the method further includes:
[0023] When a first image in the abnormal image sample is input into the target detection model to obtain a second image, a derivative of each pixel of the second image with respect to the first image is calculated;
[0024] determining abnormal pixels in the first image based on the derivative of each pixel;
[0025] Abnormal pixels in the first image are marked, and the marked first image is obtained and displayed.
[0026] In a second aspect, an embodiment of the present application provides a device for detecting abnormal images, the device comprising:
[0027] An abnormal image acquisition module, used to acquire an abnormal image sample, wherein the abnormal image sample is an image obtained by rendering a second application package, wherein the second application package is a package obtained by adding a problem code to the first application package;
[0028] A normal image acquisition module, used to acquire a normal image sample, where the normal image sample is an image obtained by rendering the first application package;
[0029] The model training module is used to train the initial detection model based on the abnormal image samples and the normal image samples to obtain a trained target detection model, wherein the initial detection model is used to output the probability that the input image is a normal image or an abnormal image.
[0030] In a third aspect, an embodiment of the present application provides an electronic device, comprising: a processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor communicates with the memory through the bus, and the processor executes the machine-readable instructions to perform the abnormal image detection method as described above.
[0031] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the method for detecting abnormal images as described above is executed.
[0032] The embodiment of the present application provides a method, device, electronic device and storage medium for detecting abnormal images, the method comprising: obtaining abnormal image samples, the abnormal image samples are images obtained by rendering a second application package, the second application package is a package obtained after adding a problem code to the first application package; obtaining normal image samples, the normal image samples are images obtained by rendering the first application package; based on the abnormal image samples and the normal image samples, an initial detection model is trained to obtain a trained target detection model, wherein the target detection model is used to output the probability that the input image is a normal image or an abnormal image. The embodiment of the present application obtains a second application package after adding a problem code to the first application package, and renders the second application package to obtain an abnormal image sample, so as to increase the data volume and data diversity of the abnormal image sample, so that the target detection model obtained by training the abnormal image sample and the normal image sample has higher accuracy, thereby improving the accuracy of the target detection model in detecting abnormal images.
[0033] Other features and advantages of the present application will be described in the following description, and partly become apparent from the description, or understood by practicing the present application. The purpose and other advantages of the present application are realized and obtained by the structures specifically pointed out in the description, claims and drawings.
[0034] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, preferred embodiments are specifically cited below and described in detail with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS
[0035] In order to more clearly illustrate the specific implementation methods of the present application or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without paying any creative work.
[0036] Figure 1 A flowchart of a method for detecting abnormal images provided in an embodiment of the present application;
[0037] Figure 2 A schematic diagram of an abnormal image interface provided in an embodiment of the present application;
[0038] Figure 3 A schematic diagram of another abnormal image interface provided in an embodiment of the present application;
[0039] Figure 4 A schematic diagram of another abnormal image interface provided in an embodiment of the present application;
[0040] Figure 5A schematic diagram of the structure of an abnormal image detection device provided in an embodiment of the present application;
[0041] Figure 6 A schematic diagram of the structure of another abnormal image detection device provided in an embodiment of the present application;
[0042] Figure 7 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0043] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution of the present application will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present application.
[0044] With the rapid development of computer technology, mobile devices on the market are changing with each passing day. The operating systems of different models of mobile devices are not the same. Especially for Android-based mobile devices, device manufacturers will optimize the native system, which has led to extremely serious fragmentation of mobile devices. How to automatically perform software compatibility testing on a large number of mobile devices has become a major difficulty for current software manufacturers. Furthermore, before a new version of an application (Application, APP) goes online, testers usually need to test the graphical user interface of the application to determine whether the application has display abnormalities. Taking a game application as an example, before a new version of a game application goes online, testers need to detect whether there are abnormalities in the game screen of the game application, for example, whether there is a black screen or a flowery screen, abnormal display of virtual elements, abnormal display of skill special effects, etc. In other words, unlike traditional Internet applications such as e-commerce and video, game applications will design a large number of animation scenes in order to attract more users, and 3D rendering will be frequently triggered during the game running process, which leads to the problem of abnormal image display of game applications being more complicated than traditional Internet applications, and the representation form is not fixed. After statistical analysis, image display abnormality accounts for about 40% of the compatibility test problems of game applications.
[0045] However, unlike compatibility issues such as flashbacks and installation failures, application display abnormalities are generally not reflected in the mobile device operation log. That is to say, even if an abnormal image appears during the application operation, the software will not report an error, but it requires people with certain testing experience to judge, so it is impossible to use the operation log to judge such problems. In addition, due to the wide variety of image abnormalities (such as flower screen, black edges, abnormal color blocks, etc.) and the huge differences in different game styles, directly performing pixel statistics at the image level cannot accurately judge all abnormal problems. More and more mobile application manufacturers have introduced artificial intelligence algorithms (AI) in their daily workflows. Most manufacturers will collect certain image data and use neural networks to learn the differences between these normal / abnormal image samples. By analyzing the implicit rules, a classifier that determines whether the interface display is normal can be obtained. With the help of this classifier, display abnormalities that occur during the application operation can be identified. However, since machine learning algorithms require a large amount of data support, relying solely on a small number of abnormal image samples accumulated in history cannot allow the classifier to learn the difference between normal image samples and abnormal image samples. Therefore, it is necessary to use certain data augmentation techniques to generate a large number of abnormal image samples for the classifier to learn.
[0046] In the related art, most existing compatibility testing solutions have two methods: the first is to conduct manual verification, that is, professional testers manually traverse application scenarios (such as game scenarios) and judge whether the graphical user interface displays normally based on experience. The second is to conduct it in an automated way, such as writing a specified test script, and then using certain rules (such as judging whether the image contains a large area of rectangular color blocks) to detect whether the graphical user interface displays normally during the script running process.
[0047] At present, more and more application manufacturers are introducing artificial intelligence algorithms (AI) in their daily work processes to reduce labor costs. Since judging whether the graphical user interface display is normal during compatibility testing is a job that requires a lot of testing experience, the introduction of AI algorithms can greatly reduce the manpower input in the testing process and improve work efficiency. Therefore, most application manufacturers will collect certain image sample data, use neural networks to learn the differences between these normal / abnormal image samples, analyze the implicit rules, and then obtain a classifier to determine whether the interface display is normal.
[0048] However, manual testing is time-consuming and laborious due to the large number of application scenarios and the large number of mobile devices to be tested. Taking game applications as an example, for a game, it takes at least one week to conduct compatibility testing after each game content update. Therefore, more and more application manufacturers choose to use automated methods to conduct compatibility testing, that is, to write rules to count image pixels to determine whether the graphical user interface display is normal. However, due to the excessive number of abnormalities in the graphical user interface display, a certain rule script needs to be written for each category. Due to the different game styles and different adaptation solutions for different models, once the generalization of the rules is not strong enough, the calculation method of the corresponding script needs to be adjusted, which requires a certain cost for maintenance. In addition, due to the excessive number of abnormalities in the graphical user interface display, it is difficult for these rules to cover all abnormal images. Even for the same abnormal problem, it is difficult to use rules to reproduce the corresponding abnormal image samples 100%.
[0049] From the above content, we can know that most manufacturers will collect a certain amount of image data, use neural networks to learn the differences between these normal / abnormal image samples, analyze the implicit rules, and then get a classifier that can judge whether the interface is normal or not. However, the number of existing abnormal image samples is very small, and the existing process of constructing abnormal image samples is too empirical, resulting in insufficient data diversity of abnormal image samples. As a result, due to the small number of abnormal image samples and insufficient data diversity, it is easy to cause classifier classification errors, thereby affecting the accuracy of abnormal images.
[0050] Based on this, the embodiment of the present application provides a method for detecting abnormal images, and provides the following inventive concept: a normal image sample is obtained by rendering a first application package, a second application package is obtained after adding a problem code to the first application package, an abnormal image sample is obtained by rendering the second application package, and an initial detection model is trained based on the obtained normal image sample and abnormal image sample to obtain a trained target detection model, wherein the target detection model is used to output the probability that the input image is a normal image or an abnormal image. In the above method, the second application package is obtained after adding a problem code to the first application package, and the abnormal image sample is obtained by rendering the second application package to increase the data volume and data diversity of the abnormal image sample, so that the accuracy of the target detection model obtained by training the abnormal image sample and the normal image sample is higher, thereby improving the accuracy of the target detection model in detecting abnormal images.
[0051] See also Figure 1 , Figure 1 The following is a flow chart of a method for detecting abnormal images provided in an embodiment of the present application. Figure 1 As shown, the method includes:
[0052] S101, obtaining an abnormal image sample, where the abnormal image sample is an image obtained by rendering a second application package, where the second application package is a package obtained by adding a problem code to the first application package;
[0053] S102, obtaining a normal image sample, where the normal image sample is an image obtained by rendering the first application package;
[0054] S103. Based on the abnormal image samples and the normal image samples, an initial detection model is trained to obtain a trained target detection model, wherein the target detection model is used to output the probability that the input image is a normal image or an abnormal image.
[0055] The embodiment of the present application obtains a second application package by adding a problem code to a first application package, and renders the second application package to obtain abnormal image samples, so as to increase the data volume and data diversity of the abnormal image samples, so that the target detection model trained by the abnormal image samples and normal image samples has higher accuracy, thereby improving the accuracy of the target detection model in detecting abnormal images.
[0056] In step S101, abnormality refers to non-normality. An image is composed of many pixels. An abnormal image refers to an abnormal image, that is, some pixels on the image are abnormal. The manifestations of abnormal images include: flower screen, green screen, blue screen, water ripples, black screen, no image; red light, blue light, stray light, light leakage and other phenomena appear in the middle of the image; there are often bright spots or black spots, damaged spots in the middle of the image, etc. In the embodiment of the present application, an abnormal image refers to an image in which the graphical user interface of the application program is abnormal. The abnormal image samples include a large number of abnormal images. The manifestations of abnormal graphical user interfaces include but are not limited to: black screen or flower screen, abnormal display of virtual elements, and abnormal display of skill special effects.
[0057] The application package is an installation package for installing the application, including all files installed by the application, including the application code. In the embodiment of the present application, the engine is used to render the program code in the application package to obtain the corresponding image in the running application. For example, the resource files corresponding to the virtual scene in the game are packaged into a resource package, and the resource package is rendered by the game engine to obtain the image of the virtual scene in the game.
[0058] Since the application package includes the application code, the embodiment of the present application can obtain the second application package after adding the problem code to the first application package. Specifically, the method of adding the problem code to the first application package can include: first, directly adding the problem code to the original program code of the first application package, and then compiling the code to generate a new application, but each time the code is modified, it is necessary to recompile and package the project, which takes a long time, and once the effect of the new code is not the desired effect, it is necessary to repeat the process of adding code, compiling, and packaging. Second, using hotfix technology, that is, using the code patch method to let the first application package replace the original program code with the problem code, generally first specifying the code fragment in the application code to bury the injection point (because all codes may have problems, generally speaking, the application will bury the code injection point for all code fragments to prevent accidents), and the server can send instructions and corresponding patch codes to the application to replace the original program code with the newly written problem code. Hotfix technology does not require compilation engineering, plug-and-play, and even if the presentation effect is not ideal, it can be withdrawn and re-patched. The embodiment of the present application preferably chooses to use hotfix technology to add the problem code to the first application package.
[0059] The problem code is a code for generating a target image problem, where the target image problem is a problem related to a virtual camera, wherein the virtual camera is used to capture an image of a virtual scene.
[0060] Here, multiple virtual cameras can be set in the virtual scene of the online game, and the virtual camera can be used to virtually shoot the virtual scene from different perspectives to obtain the game screen. Specifically, all resources required for the operation of the virtual camera tool are packaged into a resource package compatible with the game engine, and the game engine can be the Unity engine. The ExportPackage function provided by the Unity engine is used to package all resources required for the operation of the camera tool into a Unity resource package. When the virtual camera tool needs to be used, the resource package can be imported in the game engine using Import Package, and the virtual camera tool can be dragged into the virtual scene to capture the image of the virtual scene.
[0061] Taking a game application written based on the Unity engine as an example, the embodiments of the present application have obtained three common reasons, all of which are related to the virtual camera in the Unity engine. The virtual camera is a device in the Unity game that captures and displays virtual scenes for users. A virtual scene can have multiple virtual cameras, which can be placed anywhere on the screen. By setting the position and properties of the virtual camera, different game screens can be presented to users.
[0062] Since the target image problems that lead to the generation of abnormal images may be repeated, here, only the non-repeating target image problems that lead to the generation of these abnormal images are counted from multiple abnormal images.
[0063] Based on the above description, in the embodiment of the present application, the target image problem used to represent the problem related to the virtual camera includes at least one of the following: the problem of setting the cache flag of the virtual camera, the problem of shutting down the virtual camera, and the problem of screen post-processing. The following describes these three target image problems in detail:
[0064] The first case: the target image problem includes the cache flag setting problem of the virtual camera. When there is an abnormality in the graphical user interface, the target image problem is used to indicate that there is a cache flag of the camera in the software engine that does not meet the setting requirements. Here, the cache flag of the camera is used to clear the specified area of the screen (cache frame), which is usually used when multiple cameras render different objects. If the current setting of this cache flag does not meet the setting requirements, historical object tracks may remain in the scene, that is, a pattern similar to frame overlay appears, such as Figure 2 shown.
[0065] The second case: target image problems include virtual camera shutdown problems. When there is an abnormality in the graphical user interface, the target image problem is used to indicate that some cameras in the application scene are turned off. Here, the camera is a tool used to display objects in the scene to the user. If a camera is turned off, the objects captured by this camera will not be displayed to the user. At this time, the content of this part of the screen will be replaced by other cameras, and some noise pixels or spots may appear. Figure 3 shown.
[0066] In this way, turning off the camera will cause the object it is responsible for capturing to not be displayed. At this time, the content of this part on the screen will be replaced by other cameras. At this time, some cameras in the application scene are turned off, which will cause abnormal screen display.
[0067] The third case: the target image problem includes the problem of screen post-processing. When there is an abnormality in the graphical user interface, the target image problem is used to characterize that the historical screen post-processing special effects have not been cleared. Here, screen post-processing refers to rendering the complete scene to obtain the screen image, and then processing the screen image to achieve screen special effects. Using this technology, you can add more artistic effects to the game screen, such as adding filters, blurs, mirror flips and other effects to the image. If the screen post-processing special effects are not cleared in time after leaving a specific scene (such as a battle death scene) or the code to clear the special effects fails for some reason, it will cause some gray, blurry, snowflake and other special effects to appear in the current scene. For example Figure 4 shown.
[0068] The embodiment of the present application can determine the problem code corresponding to each target image problem when there is an abnormality in the graphical user interface. The following is a specific description:
[0069] For the first case, when the graphical user interface is abnormal, the target image problem is used to indicate that there is a cache flag of the camera that does not meet the setting requirements in the software engine. The steps to determine the problem code corresponding to the target image problem can be as follows:
[0070] Step 201: According to the cache flag of the camera that meets the setting requirements in the software engine, determine the original clear mark attribute and the corresponding original cache code corresponding to the cache flag of the camera that meets the setting requirements.
[0071] In step 201, the cache flag of the camera that meets the setting requirements in the software engine under the current scene is obtained, that is, the cache flag of the camera that meets the setting requirements will make the game scene appear normal.
[0072] Since the cache flag of the camera has a clear mark attribute, in this way, the original clear mark attribute corresponding to the cache flag of the camera that meets the setting requirements and the original cache code corresponding to the cache flag of the camera that meets the setting requirements can be determined. Here, the original clear mark attribute refers to the clear mark attribute that makes the game scene appear normally, and the original cache code refers to the code of the cache flag of the camera that makes the game scene appear normally.
[0073] Specifically, the clear flag attribute Clearflags refers to the attribute used to set the camera cache in the Unity game. It is responsible for managing the color cache value and depth cache value of the camera. When there are multiple cameras, since the cameras have a rendering order, by setting this clear flag attribute value, you can decide whether to clear the color cache value ColorBuffer and depth cache value DepthBuffer of all the previous cameras.
[0074] Specifically, the clear flag attribute Clearflags includes 4, namely: solid color Solid Color, skybox Skybox, depth only DepthOnly, don't clear Don't Care. Among them, solid color Solid Color means clearing the depth cache value of all the previous cameras and the color cache value of all the previous cameras, and using the background color instead of the color cache value of the previous camera; skybox means clearing the depth cache value of all the previous cameras and the color cache value of all the previous cameras, and using the skybox color value instead of the color cache value of the previous camera; depth only DepthOnly means clearing the depth cache value of all the previous cameras and retaining the color cache value of all the previous cameras; don't clear Don't Care means retaining the depth cache value and color cache value of all the previous cameras. This embodiment can make the game screen present different display abnormalities by setting different clear flag attributes Clearflags.
[0075] Step 202, in response to the modification operation of the original cache code, the original clear mark attribute is modified to other clear mark attributes to obtain a problem code for characterizing the cache flag of the camera that does not meet the setting requirements in the software engine; wherein the other clear mark attribute is different from the original clear mark attribute.
[0076] In step 202, the cache flag of the camera represented by the original cache code is changed by modifying the code for the original clear mark attribute in the original cache code. In one embodiment, the original clear mark attribute can be modified to other clear mark attributes, wherein the original clear mark attribute can include any one of the following items: pure color, sky box, depth only, no clear; further, the other clear mark attributes can also include any one of the following items: pure color, sky box, depth only, no clear, but the other clear mark attributes are different from the original clear mark attributes.
[0077] It should be noted here that in the current game scene, the original clear mark attribute corresponding to the camera's cache flag is one of the following: solid color, skybox, depth only, and no clear. At this time, the game scene can be guaranteed to be presented normally. However, if the game scene is presented abnormally, you only need to modify the original clear mark attribute corresponding to the camera's cache flag to other clear mark attributes. The other clear mark attribute is any one of the four items of solid color, skybox, depth only, and no clear that is different from the original clear mark attribute. You can get the problem code used to characterize the existence of a camera's cache flag that does not meet the setting requirements in the software engine. Among them, the problem code is used to characterize the camera's cache flag corresponding to other clear mark attributes.
[0078] In addition, when the original clear mark attribute corresponding to the cache flag of the camera is modified to other clear mark attributes, since the other clear mark attributes are not unique and the attributes of each other clear mark attribute are different, the types of abnormal images obtained are also different, thereby enriching the diversity of abnormal image samples.
[0079] For example, the original clear mark attribute corresponding to the cache flag of the camera that meets the setting requirements in the software engine is the skybox. At this time, the original cache code of the cache flag of the camera corresponding to the original clear mark attribute of the skybox is obtained. When the skybox is changed to depth only, the code segment representing the original clear mark attribute as the skybox in the original cache code is modified accordingly, that is, the code segment with the original clear mark attribute as the skybox is modified to other clear mark attributes as depth only, and the code segments in the original cache code except the code segments corresponding to the original clear mark attribute remain unchanged.
[0080] For example, a sample code is as follows:
[0081] local cameraobjs=CS.UnityEngine.Object.FindObjectsOfType(typeof(CS.UnityEngine.Camera))
[0082] for i=0,cameraobjs.Length-1
[0083] do
[0084] cameraobjs[i].clearFlags=UnityEngine.CameraClearFlags.DepthOnly
[0085] end
[0086] For the second case, when there is an abnormality in the graphical user interface, the target image problem is used to indicate that some cameras in the application scenario are turned off. The steps for determining the problem code corresponding to the target image problem can be as follows:
[0087] Step 301: Determine an activation code for activating some cameras in an application scenario.
[0088] In step 301, the activation codes for activating some cameras in the application scene in the current scene can be obtained from the installation package. For example, if there are 10 cameras in the current scene, the activation codes of these 10 cameras can be obtained to determine the activation codes of some cameras to be closed; or the activation codes of only some cameras can be directly obtained and then these cameras will be closed.
[0089] Step 302: In response to the modification operation on the enable code, a problem code is obtained for indicating that some cameras in the application scenario are disabled.
[0090] In step 302, in order to close some cameras in the application scene, the opening code of these cameras can be modified to a closing code, where the closing code is the problem code. In this way, these cameras can be closed, so that the objects captured by the closed cameras cannot be displayed, resulting in the content of this part on the screen being replaced by other cameras, thereby causing abnormal screen display.
[0091] For example, a sample code is as follows:
[0092] local cameraobjs=CS.UnityEngine.Object.FindObjectsOfType(typeof(CS.UnityEngine.Camera))
[0093] for i=0,cameraobjs.Length-1
[0094] do
[0095] cameraobjs[i].enabled=false
[0096] end
[0097] For the third case, when the graphical user interface is abnormal, the target image problem is used to characterize the historical screen post-processing effects that are not cleared. The steps to determine the problem code corresponding to the target image problem can be as follows:
[0098] Step 401: Determine a clearing code for clearing historical screen post-processing special effects.
[0099] In step 401, a clearing code for clearing the historical screen post-processing special effects in the current scene may be obtained from the installation package.
[0100] Step 402, in response to the code writing operation, the clearing code is deleted or the clearing code is invalidated to obtain the problem code for representing the screen post-processing special effects that are not cleared in the history.
[0101] In step 402, in order to delete or invalidate the clearing code, the clearing code may be directly modified to invalidate it, or a new code may be added to delete or invalidate the clearing code. The newly added code or the modified clearing code is used as the problem code representing the historical screen post-processing special effect that has not been cleared.
[0102] In this way, since the screen post-processing special effects of certain specific scenes are not cleaned up in time, abnormal pictures (such as mirror flipping of controls) may appear in the current scene, thereby obtaining some abnormal images.
[0103] Since it is more convenient to directly add some impossible post-processing effects to the current scene than not clearing the screen post-processing effects of the previous scene, this embodiment randomly adds some impossible post-processing effects to the current scene, thereby generating some abnormal images.
[0104] Specifically, the steps for determining the problem code corresponding to the target problem factor may be as follows:
[0105] Determines the clear code for screen post-processing effects that clear history;
[0106] In response to the code writing operation, a code for representing other screen post-processing special effects is generated to obtain a problem code; wherein the code for the other screen post-processing special effects is different from the code for the historical screen post-processing special effects corresponding to the clearing code.
[0107] For example, a sample code is as follows:
[0108] local detectCamera=GameObject.Find("UICamera")
[0109] if (detectCamera~=nil)
[0110] then
[0111] detectCamera.gameObject:AddComponent(typeof(CameraFilterPack_3D_Mirror))
[0112] end
[0113] In an optional embodiment, the step of obtaining the first application package body after adding the problem code to the first application package body in step S101 includes:
[0114] Step 1011: Set an injection point for each of the multiple code fragments of the first application package to obtain multiple injection points corresponding to the first application package.
[0115] Here, an injection point is a place where injection can be performed, usually a connection to access a database. Through this access point, code can be injected into the first application package body through this injection point at a position corresponding to this injection point.
[0116] Step 1012: According to the problem code corresponding to the target image problem, determine a target injection point corresponding to the problem code from multiple injection points.
[0117] Here, through the problem code corresponding to the target image problem, the code fragment corresponding to the problem code is found from the first application package, and then the injection point of the code fragment is found, and the injection point of the code fragment is determined as the target injection point corresponding to the problem code.
[0118] Step 1013: insert the problem code into the front of the original code corresponding to the problem code in the first application package through the target injection point to obtain a second application package, so that the second application package runs the problem code.
[0119] Here, the problem code is inserted in front of the original code corresponding to the problem code in the first application package through the target injection point to obtain the second application package. When the application runs to the problem code in the second application package, it will no longer load the original code, and thus can successfully generate an abnormal image caused by the problem code.
[0120] In an optional embodiment of the present application, step S101 includes: acquiring images of different virtual scenes of the second application package;
[0121] Step S102 includes: acquiring images of different virtual scenes of the first application package.
[0122] Here, by respectively acquiring abnormal images and normal images in different virtual scenes through the first application package and the second application package, the data diversity of abnormal images and normal images can be improved, so that the target detection model trained by the abnormal image samples and normal image samples has higher accuracy, thereby improving the accuracy of the target detection model in detecting abnormal images.
[0123] In the embodiment of the present application, by generating an abnormal image in step S101, a problem code for generating a target image problem can be added to a first application package to generate a second application package, so that the second application package generates an abnormal image corresponding to the target image problem during the running of the problem code, thereby increasing the data volume of abnormal image samples. At the same time, since the type of problem code can affect the diversity of abnormal image samples, a more comprehensive target image problem can increase the type of generated abnormal image display, thereby increasing the data diversity of abnormal image samples, so that the data used to train the detection model is more accurate.
[0124] Further, with respect to the method of obtaining abnormal image samples, step S101 specifically includes:
[0125] Step 1014: Take multiple screenshots of the image obtained by rendering the second application package to obtain multiple abnormal image frames;
[0126] Step 1015: Screen multiple abnormal image frames to obtain abnormal image samples.
[0127] Specifically, during the process of the second application package running the problematic code, abnormal images can be collected by recording the screen and taking multiple screenshots to obtain abnormal image samples.
[0128] Exemplarily, after inserting the problem code into the first application package, a second application package can be obtained. During the process of running the problem code in the second application package, if a display abnormality occurs in the graphical user interface, this solution will start recording the screen and take multiple screenshots to obtain several abnormal image samples.
[0129] In step 1014, since it is necessary to traverse the abnormal images of each scene, this embodiment will use an automated clicking tool: Monkey tool to perform pixel-level random clicking on the graphical user interface. Since only the interface display is abnormal after the problem code is injected, the game operation logic will not be affected. Therefore, as long as the Monkey tool clicks on the game control, the interface can jump, so that the abnormal image frames in each scene can be collected. If the game interface does not jump for a long time after using the Monkey tool, it may be that the game is trapped in a scene that is very difficult to jump out (for example, certain specific characters must be entered in the text box before the scene jump occurs). At this time, this embodiment has two methods to continue the data collection process: first, clear the patch file first to restore the scene interface to normal, then manually jump out of the scene, and then repeat the initial patching step to continue data collection. Second, restart the application and restart the patching step to continue data collection.
[0130] Since the training of the detection model for detecting abnormal images requires both abnormal image samples and normal image samples, the embodiment of the present application also needs to collect a series of normal image samples. As shown in step S102, normal image samples are obtained, and the normal image samples are images obtained by rendering the first application package.
[0131] Specifically, during the running of the first application package, a plurality of normal images can be obtained by taking screenshots of the graphical user interface, and each screenshot corresponds to a frame of image.
[0132] For example, you only need to open the application and use the Monkey tool to randomly click and record the screen. The method is relatively simple. After collecting normal image sample data, you can manually confirm whether the interface display in the image is normal.
[0133] Since the embodiment of the present application collects abnormal image samples and normal image samples in the form of screen captures, directly extracting images by frame will obtain a large number of images with high similarity, so further screening by image similarity can prevent excessive sample repetition, thereby not affecting the final training effect of the detection model used to detect abnormal images. Since the video is continuous, the similarity of adjacent frames in the video is much higher than that of non-adjacent frames. In order to improve the screening efficiency, the embodiment of the present application only compares the similarity of the target abnormal image with the adjacent previous abnormal image.
[0134] In an optional embodiment, in step 1015, screening multiple abnormal image frames to obtain abnormal image samples may include the following scheme: based on a preset interval frame number, starting from any abnormal image frame of the multiple abnormal image frames, screening to obtain multiple target abnormal image frames, and the abnormal image sample is composed of the multiple target abnormal image frames.
[0135] Here, starting from any abnormal image frame of the plurality of abnormal image frames, a plurality of target abnormal image frames are selected at a preset interval frame number to form an abnormal image sample.
[0136] For example, assuming that a 10-second video has 200 abnormal image frames, the preset interval frame number can be 20 frames, so that the target abnormal image frame selected from these 200 abnormal image frames can be the 1st abnormal image frame, the 21st abnormal image frame, the 41st abnormal image frame... the 181st abnormal image frame, and these selected abnormal image frames are combined into an abnormal image sample.
[0137] However, in the above method, in order to avoid the similarity between adjacent target abnormal image frames in the selected target abnormal image frames being greater than the preset threshold, the preset interval frame number needs to be set larger, but the larger preset interval frame number may lose some abnormal image frames caused by different target image problems. Based on this, in step 1015, screening multiple abnormal image frames to obtain abnormal image samples may include the following scheme:
[0138] Step 10151: for any target abnormal image frame among the multiple abnormal image frames, compare the target abnormal image frame with the adjacent previous abnormal image frame to obtain a similarity;
[0139] Step 10152: If the similarity is greater than a preset threshold, the target abnormal image frame is deleted from the multiple abnormal image frames, and the remaining abnormal image frames constitute an abnormal image sample.
[0140] Step 10151 also includes: selecting at least part of the abnormal image frames from the multiple abnormal image frames in sequence, and sorting the selected at least part of the abnormal image frames in chronological order to obtain an abnormal image frame sequence, so as to select a target abnormal image frame from the abnormal image frame sequence; any two adjacent abnormal image frames in the abnormal image frame sequence are separated by a preset number of frames; wherein the preset number of frames can be n frames, n≥1.
[0141] Here, it is necessary to screen the target abnormal image frames in the abnormal image frame sequence according to the similarity between abnormal image frames separated by a preset number of frames, and filter out some target abnormal image frames with relatively high similarity. The preset number of frames may be n frames, where n≥1.
[0142] As an optional embodiment, the step of determining an abnormal image frame that meets the screening requirement from a plurality of abnormal image frames includes:
[0143] Selecting at least part of the abnormal image frames from the plurality of abnormal image frames in sequence, and sorting the selected at least part of the abnormal image frames in chronological order to obtain an abnormal image frame sequence; any two adjacent abnormal image frames in the abnormal image frame sequence are separated by a preset number of frames;
[0144] The first abnormal image frame in the abnormal image frame sequence is retained, and the target abnormal image frame is selected starting from the second abnormal image frame in the abnormal image frame sequence in turn, and a loop process is performed until a preset condition is met, and all abnormal image frames retained in the abnormal image frame sequence are determined as abnormal image frames that meet the screening requirements; wherein the preset condition is that the target abnormal image frame selected from the abnormal image frame sequence is the last abnormal image frame in the abnormal image frame sequence;
[0145] The cycle process includes:
[0146] Determine the similarity between the target abnormal image frame and an abnormal image frame located one position above the target abnormal image frame in the abnormal image frame sequence;
[0147] If the similarity is greater than a preset threshold, the target abnormal image frame is deleted from the abnormal image frame sequence;
[0148] If the similarity is not greater than the preset threshold, the target abnormal image frame is retained.
[0149] It should be noted that the target abnormal image frame and the abnormal image frame immediately above the target abnormal image frame are separated by a preset number of frames n, where n≥1.
[0150] For example, the target abnormal image frame and the abnormal image frame above the target abnormal image frame are separated by one frame. Assuming that a 10-second video has 200 abnormal image frames, the 200 abnormal image frames are processed as follows:
[0151] The 200 abnormal image frames are sequentially sorted in time order to obtain an abnormal image frame sequence; any two adjacent abnormal image frames in the abnormal image frame sequence are separated by one frame, wherein the time is the recording time displayed by each abnormal image frame when recording the video, and the 200 abnormal image frames can be sorted in order from the front to the back of the recording time, or can be sorted in order from the back to the front of the recording time;
[0152] The first abnormal image frame in the abnormal image frame sequence is retained; the target abnormal image frame is selected starting from the second abnormal image frame in the abnormal image frame sequence in turn, and a loop process is performed until a preset condition is met, and all abnormal image frames retained in the abnormal image frame sequence are determined as abnormal image frames that meet the screening requirements; wherein the preset condition is that the target abnormal image frame selected from the abnormal image frame sequence is the last abnormal image frame in the abnormal image frame sequence; the loop process includes: determining the similarity between the target abnormal image frame and the abnormal image frame in the abnormal image frame sequence that is located one frame above the target abnormal image frame; if the similarity is greater than a preset threshold, deleting the target abnormal image frame from the abnormal image frame sequence; if the similarity is not greater than the preset threshold, retaining the target abnormal image frame.
[0153] That is, first retain the first abnormal image frame in the abnormal image frame sequence, start from the second abnormal image frame, calculate the similarity between the second abnormal image frame and the first abnormal image frame, if the similarity is greater than a preset threshold, delete the second abnormal image frame from the abnormal image frame sequence; continue to calculate the similarity between the third abnormal image frame and the first abnormal image frame, if the similarity is not greater than the preset threshold, retain the third abnormal image frame, then calculate the similarity between the fourth abnormal image frame and the third abnormal image frame, if the similarity is greater than the preset threshold, delete the fourth abnormal image frame from the abnormal image frame sequence; if the similarity is not greater than the preset threshold, retain the fourth abnormal image frame, and so on, all abnormal image frames retained in the abnormal image frame sequence are determined as abnormal image frames that meet the screening requirements, that is, all remaining abnormal image frames in the abnormal image frame sequence can be composed of abnormal image samples.
[0154] Specifically, starting from the second frame of the video, the similarity between the current abnormal image frame x and the previous abnormal image frame y is calculated using the formula:
[0155]
[0156] Among them, u x is the mean of x, u y is the mean of y, is the variance of x, is the variance of y, σxy is the covariance of x and y, c1=(k1L) 2 ,c2=(k2L) 2 , k1 is a constant, generally taken as 0.01, k2 is a constant, generally taken as 0.03, and L is the dynamic range of grayscale. The range of similarity is -1 to 1. When two abnormal image frames are exactly the same, the value of SSIM is equal to 1. If the similarity between the abnormal image frame of the current frame and the abnormal image frame of the previous frame is greater than a preset threshold (set to 0.8 in this solution), then it can be considered that the similarity between the two abnormal image frames is too high, and the abnormal image frame of the current frame may not be retained. In addition, since the number of frames may be high during video recording (a mobile phone with better performance may be able to reach 60 frames per second), in order to increase the running speed, an optional embodiment of this solution is that x and y can be separated by n frames, where n is a fixed parameter and the value of n can be adjusted according to demand.
[0157] In step S103, the initial detection model is trained based on the abnormal image samples and the normal image samples to obtain a trained target detection model, wherein the target detection model is used to output the probability that the input image is a normal image or an abnormal image.
[0158] Exemplarily, after obtaining all normal image samples and abnormal image samples that can be used for training, an image sample set including multiple abnormal image samples and multiple normal image samples is constructed; the collected image sample set is transferred to a pre-constructed neural network model (initial detection model) for training. The structure of the neural network model is not specifically limited and can be designed by itself. The neural network model used in this embodiment is a 4-layer convolution followed by a 3-layer fully connected layer, and the dimension of the final output detection result is 2.
[0159] Specifically, a softmax operation is performed on the output vector of the neural network model to obtain the probability that the image is a normal / abnormal image:
[0160]
[0161] Then, the cross entropy loss of the classification result is calculated according to the label L of the sample (normal is 0, abnormal is 1):
[0162] loss = -L*log(p(abnormal))-(1-L)*log(p(normal));
[0163] Finally, the neural network model calculates the gradient of each neuron according to the cross entropy and updates the neural network model according to the chain rule.
[0164] In this way, after obtaining the trained target detection model, the target detection model can be used to perform anomaly detection on the image to be detected. The specific embodiments include:
[0165] Step 1031: Acquire an image to be detected by taking a screenshot of the graphical user interface.
[0166] Here, a plurality of images to be detected can be obtained by taking screenshots of the graphical user interface, and each screenshot corresponds to a frame of image.
[0167] Step 1032: input the image to be detected into a pre-trained target detection model, and output the probability that the image to be detected is a normal image or an abnormal image.
[0168] Through step 1031 and step 1032, a plurality of images to be detected are sequentially input into the pre-trained target detection model, thereby obtaining the detection result of the target detection model. The detection result is used to characterize the probability that the image to be detected is a normal image or an abnormal image.
[0169] Here, if the detection result for the image to be detected is: the probability that the image to be detected is an abnormal image, and the probability exceeds the preset probability threshold, it is determined that the video to be detected corresponding to the image to be detected is abnormal. In the case that the video to be detected is an abnormal video, a reminder message is sent so that the technician can find the fault and find the abnormal problem in time. Among them, the preset probability threshold is used to characterize the critical value of the probability that the image to be detected is an abnormal image.
[0170] Exemplarily, after the game is started, random click operations can be performed. After each click operation, a screenshot of the current graphical user interface will be taken. After each screenshot, a picture to be detected will be obtained, and the picture to be detected will be transmitted to the target detection model. If the target detection model determines that the picture to be detected is abnormal, the abnormal picture to be detected is saved, and the abnormal area therein is analyzed. Finally, the abnormal picture to be detected and the corresponding abnormal area are packaged and transmitted to the game developer to find the specific cause of the problem, and correct the cause, thereby achieving the effect of assisting the developer in reviewing.
[0171] Based on the above description, the method provided in the embodiment of the present application also includes:
[0172] Step 501: when a first image in an abnormal image sample is input into an initial detection model to obtain a second image, a derivative of each pixel of the second image with respect to the first image is calculated;
[0173] Step 502: determining abnormal pixels in the first image according to the derivative of each pixel;
[0174] Here, if the derivative of a certain pixel is greater than a preset derivative threshold, the pixel is determined to be an abnormal pixel in the first image.
[0175] Step 503: Mark abnormal pixels in the first image, and obtain and display the marked first image.
[0176] Here, the abnormal pixels in the first image are marked, so as to obtain the first image with abnormal pixel points marked.
[0177] Furthermore, after the target detection model training is completed, the target detection model obtained in this embodiment can be introduced into daily testing to detect whether the graphical user interface display is normal.
[0178] For example, the Monkey tool is first used to randomly click in the application, and screenshots of the mobile phone interface are taken at regular intervals. After the screenshots are obtained, the images are transferred to the target detection model, and the label of the image is obtained through forward calculation of the target detection model. If the output result of the target detection model shows that the image is an abnormal image, further abnormality analysis will be performed.
[0179] Specifically, the anomaly analysis solution can be to directly calculate the derivative of the result y of the objective function corresponding to the target detection model for each pixel in the original image I:
[0180]
[0181] The larger the derivative, the more likely the pixel is an abnormal pixel. By marking the positions of all abnormal pixels, you can get a result analysis diagram corresponding to the abnormal image, and record the problem image and result analysis diagram in the test report. In this way, when the game developers review, they will observe the abnormal image more clearly and intuitively, which will assist the developers in reviewing and making modifications.
[0182] The abnormal image detection method provided in the embodiment of the present application adds a problem code to the first application package to obtain a second application package, and renders the second application package to obtain an abnormal image sample, so as to increase the data volume and data diversity of the abnormal image sample, so that the target detection model obtained by training the abnormal image sample and the normal image sample has a higher accuracy, thereby improving the accuracy of the target detection model in detecting abnormal images. And when using the target detection model to detect abnormal images, an image for marking abnormal pixels can also be directly generated, which is helpful for developers to review, so as to analyze, track and modify the target image problems caused by abnormal pixels.
[0183] Based on the same inventive concept, an abnormal image detection device corresponding to the abnormal image detection method is also provided in the embodiment of the present application. Since the principle of solving the problem by the device in the embodiment of the present application is similar to the above-mentioned method in the embodiment of the present application, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be repeated.
[0184] See also Figure 5 and Figure 6 , Figure 5A schematic diagram of the structure of an abnormal image detection device provided in an embodiment of the present application. Figure 6 Schematic diagram of another abnormal image detection device provided in an embodiment of the present application. The present application embodiment provides an abnormal image detection device, such as Figure 5 As shown, the device 500 includes:
[0185] The abnormal image acquisition module 501 is used to acquire an abnormal image sample, where the abnormal image sample is an image obtained by rendering the second application package, where the second application package is a package obtained by adding a problem code to the first application package;
[0186] A normal image acquisition module 502 is used to acquire a normal image sample, where the normal image sample is an image obtained by rendering the first application package;
[0187] The model training module 503 is used to train the initial detection model based on abnormal image samples and normal image samples to obtain a trained target detection model, wherein the initial detection model is used to output the probability that the input image is a normal image or an abnormal image.
[0188] In an optional embodiment of the present application, the problem code is a code for generating a target image problem, and the target image problem is a problem related to a virtual camera, wherein the virtual camera is used to capture an image of a virtual scene.
[0189] In an optional embodiment of the present application, the target image problem includes at least one of the following:
[0190] Issues with setting the cache flag of the virtual camera, shutting down the virtual camera, and screen post-processing.
[0191] In an optional embodiment of the present application, the abnormal image acquisition module 501 is specifically used to: acquire images of different virtual scenes of the second application package;
[0192] The normal image acquisition module 502 is specifically used to acquire images of different virtual scenes of the first application package.
[0193] In an optional embodiment of the present application, the abnormal image acquisition module 501 is specifically used for:
[0194] Taking multiple screenshots of the image obtained by rendering the second application package to obtain multiple abnormal image frames;
[0195] Multiple abnormal image frames are screened to obtain abnormal image samples.
[0196] In an optional embodiment of the present application, the abnormal image acquisition module 501 is further used to:
[0197] For any target abnormal image frame among the multiple abnormal image frames, the target abnormal image frame is compared with an adjacent previous abnormal image frame for similarity to obtain a similarity;
[0198] If the similarity is greater than a preset threshold, the target abnormal image frame is deleted from the multiple abnormal image frames, and the remaining abnormal image frames constitute an abnormal image sample.
[0199] In an optional embodiment of the present application, the abnormal image acquisition module 501 is further used to:
[0200] Based on a preset interval frame number, starting from any abnormal image frame of the multiple abnormal image frames, multiple target abnormal image frames are screened to obtain an abnormal image sample composed of the multiple target abnormal image frames.
[0201] Furthermore, if Figure 6 As shown, the device 500 further includes an abnormal pixel marking module 504, and the abnormal pixel marking module 504 is used to:
[0202] When a first image in the abnormal image sample is input into the target detection model to obtain a second image, a derivative of each pixel of the second image with respect to the first image is calculated;
[0203] determining abnormal pixels in the first image based on the derivative of each pixel;
[0204] Abnormal pixels in the first image are marked, and the marked first image is obtained and displayed.
[0205] The abnormal image detection device provided in the embodiment of the present application obtains a second application package after adding a problem code to the first application package, and renders the second application package to obtain abnormal image samples, so as to increase the data volume and data diversity of the abnormal image samples, so that the target detection model obtained by training the abnormal image samples and normal image samples has higher accuracy, thereby improving the accuracy of the target detection model in detecting abnormal images.
[0206] See also Figure 7 , Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 7 As shown in FIG. 7 , the electronic device 700 includes a processor 701 , a memory 702 , and a bus 703 .
[0207] The memory 702 stores machine-readable instructions executable by the processor 701. When the electronic device 700 is running, the processor 701 communicates with the memory 702 via the bus 703. The processor 701 executes the machine-readable instructions to perform the following steps:
[0208] Acquire an abnormal image sample, where the abnormal image sample is an image obtained by rendering the second application package, where the second application package is a package obtained by adding a problem code to the first application package;
[0209] Obtain a normal image sample, where the normal image sample is an image obtained by rendering the first application package;
[0210] Based on the abnormal image samples and the normal image samples, the initial detection model is trained to obtain a trained target detection model, wherein the target detection model is used to output the probability that the input image is a normal image or an abnormal image.
[0211] In an optional embodiment of the present application, the problem code is a code for generating a target image problem, and the target image problem is a problem related to a virtual camera, wherein the virtual camera is used to capture an image of a virtual scene.
[0212] In an optional embodiment of the present application, the target image problem includes at least one of the following:
[0213] Issues with setting the cache flag of the virtual camera, shutting down the virtual camera, and screen post-processing.
[0214] In an optional embodiment of the present application, when the processor 701 executes the step of acquiring abnormal image samples, the following steps are specifically performed: acquiring images of different virtual scenes of the second application package;
[0215] When the processor executes the step of acquiring normal image samples, the following steps are specifically performed: images of different virtual scenes of the first application package are acquired.
[0216] In an optional embodiment of the present application, when the processor 701 executes acquiring abnormal image samples, the processor 701 specifically performs the following steps:
[0217] Taking multiple screenshots of the image obtained by rendering the second application package to obtain multiple abnormal image frames;
[0218] Multiple abnormal image frames are screened to obtain abnormal image samples.
[0219] In an optional embodiment of the present application, when the processor 701 performs the step of screening multiple abnormal image frames to obtain abnormal image samples, the following steps are specifically performed:
[0220] For any target abnormal image frame among the multiple abnormal image frames, the target abnormal image frame is compared with an adjacent previous abnormal image frame for similarity to obtain a similarity;
[0221] If the similarity is greater than a preset threshold, the target abnormal image frame is deleted from the multiple abnormal image frames, and the remaining abnormal image frames constitute an abnormal image sample.
[0222] In an optional embodiment of the present application, when the processor 701 performs the step of screening multiple abnormal image frames to obtain abnormal image samples, the following steps are specifically performed:
[0223] Based on a preset interval frame number, starting from any abnormal image frame of the multiple abnormal image frames, multiple target abnormal image frames are screened to obtain an abnormal image sample composed of the multiple target abnormal image frames.
[0224] In an optional embodiment of the present application, the processor 701 further performs the following steps:
[0225] When a first image in the abnormal image sample is input into the target detection model to obtain a second image, a derivative of each pixel of the second image with respect to the first image is calculated;
[0226] determining abnormal pixels in the first image based on the derivative of each pixel;
[0227] Abnormal pixels in the first image are marked, and the marked first image is obtained and displayed.
[0228] The embodiment of the present application obtains a second application package by adding a problem code to a first application package, and renders the second application package to obtain abnormal image samples, so as to increase the data volume and data diversity of the abnormal image samples, so that the target detection model trained by the abnormal image samples and normal image samples has higher accuracy, thereby improving the accuracy of the target detection model in detecting abnormal images.
[0229] The present application also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the following steps are performed:
[0230] Acquire an abnormal image sample, where the abnormal image sample is an image obtained by rendering the second application package, where the second application package is a package obtained by adding a problem code to the first application package;
[0231] Obtain a normal image sample, where the normal image sample is an image obtained by rendering the first application package;
[0232] Based on the abnormal image samples and the normal image samples, the initial detection model is trained to obtain a trained target detection model, wherein the target detection model is used to output the probability that the input image is a normal image or an abnormal image.
[0233] In an optional embodiment of the present application, the problem code is a code for generating a target image problem, and the target image problem is a problem related to a virtual camera, wherein the virtual camera is used to capture an image of a virtual scene.
[0234] In an optional embodiment of the present application, the target image problem includes at least one of the following:
[0235] Issues with setting the cache flag of the virtual camera, shutting down the virtual camera, and screen post-processing.
[0236] In an optional embodiment of the present application, when the computer-readable storage medium is running to obtain abnormal image samples, the following steps are specifically performed: obtaining images of different virtual scenes of the second application package;
[0237] When the computer-readable storage medium is running to obtain normal image samples, the following steps are specifically performed: images of different virtual scenes of the first application package are obtained.
[0238] In an optional embodiment of the present application, when the computer-readable storage medium is running to obtain abnormal image samples, the following steps are specifically performed:
[0239] Taking multiple screenshots of the image obtained by rendering the second application package to obtain multiple abnormal image frames;
[0240] Multiple abnormal image frames are screened to obtain abnormal image samples.
[0241] In an optional embodiment of the present application, when the computer-readable storage medium is running to screen multiple abnormal image frames and obtain abnormal image samples, the following steps are specifically performed:
[0242] For any target abnormal image frame among the multiple abnormal image frames, the target abnormal image frame is compared with an adjacent previous abnormal image frame for similarity to obtain a similarity;
[0243] If the similarity is greater than a preset threshold, the target abnormal image frame is deleted from the multiple abnormal image frames, and the remaining abnormal image frames constitute an abnormal image sample.
[0244] In an optional embodiment of the present application, when the computer-readable storage medium is running to screen multiple abnormal image frames and obtain abnormal image samples, the following steps are specifically performed:
[0245] Based on a preset interval frame number, starting from any abnormal image frame of the multiple abnormal image frames, multiple target abnormal image frames are screened to obtain an abnormal image sample composed of the multiple target abnormal image frames.
[0246] In an optional embodiment of the present application, the computer-readable storage medium further performs the following steps:
[0247] When a first image in the abnormal image sample is input into the target detection model to obtain a second image, a derivative of each pixel of the second image with respect to the first image is calculated;
[0248] determining abnormal pixels in the first image based on the derivative of each pixel;
[0249] Abnormal pixels in the first image are marked, and the marked first image is obtained and displayed.
[0250] The embodiment of the present application obtains a second application package by adding a problem code to a first application package, and renders the second application package to obtain abnormal image samples, so as to increase the data volume and data diversity of the abnormal image samples, so that the target detection model trained by the abnormal image samples and normal image samples has higher accuracy, thereby improving the accuracy of the target detection model in detecting abnormal images.
[0251] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.
[0252] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical or other forms.
[0253] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0254] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.
[0255] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present application can essentially be embodied in the form of a software product, or in other words, the part that contributes to the prior art or the part of the technical solution. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.
[0256] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present application, which are used to illustrate the technical solutions of the present application, rather than to limit them. The protection scope of the present application is not limited thereto. Although the present application is described in detail with reference to the above-mentioned embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-mentioned embodiments within the technical scope disclosed in the present application, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.
Claims
1. A method for detecting abnormal images, characterized in that: The method comprises: Acquire an abnormal image sample, where the abnormal image sample is an image obtained by rendering a second application package, where the second application package is a package obtained by adding a problem code to the first application package; Acquire a normal image sample, where the normal image sample is an image obtained by rendering the first application package; Based on the abnormal image samples and the normal image samples, an initial detection model is trained to obtain a trained target detection model, wherein the target detection model is used to output the probability that the input image is a normal image or an abnormal image; The problem code is a code for generating a target image problem, and the target image problem is a problem related to a virtual camera, wherein the virtual camera is used to capture an image of a virtual scene; The target image problem includes at least one of the following: Issues with setting the cache flag of the virtual camera, shutting down the virtual camera, and screen post-processing.
2. The method according to claim 1, characterized in that: The obtaining of abnormal image samples includes: obtaining images of different virtual scenes of the second application package; The obtaining of normal image samples includes: obtaining images of different virtual scenes of the first application package.
3. The method according to claim 1, characterized in that The obtaining of abnormal image samples comprises: Taking multiple screenshots of the image obtained by rendering the second application package to obtain multiple abnormal image frames; The multiple abnormal image frames are screened to obtain abnormal image samples.
4. The method according to claim 3, characterized in that The screening of the plurality of abnormal image frames to obtain abnormal image samples includes: For any target abnormal image frame among the multiple abnormal image frames, the target abnormal image frame is compared with an adjacent previous abnormal image frame for similarity to obtain a similarity; If the similarity is greater than a preset threshold, the target abnormal image frame is deleted from the multiple abnormal image frames, and the abnormal image sample is composed of the remaining abnormal image frames.
5. The method according to claim 3, characterized in that: The screening of the plurality of abnormal image frames to obtain abnormal image samples includes: Based on a preset interval frame number, starting from any abnormal image frame of the plurality of abnormal image frames, a plurality of target abnormal image frames are screened to obtain the abnormal image samples, and the plurality of target abnormal image frames constitute the abnormal image samples.
6. The method according to claim 1, characterized in that The method further comprises: When a first image in the abnormal image sample is input into the target detection model to obtain a second image, a derivative of each pixel of the second image with respect to the first image is calculated; determining abnormal pixels in the first image based on the derivative of each pixel; Abnormal pixels in the first image are marked, and the marked first image is obtained and displayed.
7. A device for detecting abnormal images, characterized in that: The device comprises: An abnormal image acquisition module, used to acquire an abnormal image sample, wherein the abnormal image sample is an image obtained by rendering a second application package, wherein the second application package is a package obtained by adding a problem code to the first application package; A normal image acquisition module, used to acquire a normal image sample, where the normal image sample is an image obtained by rendering the first application package; A model training module, used to train an initial detection model based on the abnormal image samples and the normal image samples to obtain a trained target detection model, wherein the initial detection model is used to output the probability that the input image is a normal image or an abnormal image; The problem code is a code for generating a target image problem, and the target image problem is a problem related to a virtual camera, wherein the virtual camera is used to capture an image of a virtual scene; The target image problem includes at least one of the following: Issues with setting the cache flag of the virtual camera, shutting down the virtual camera, and screen post-processing.
8. An electronic device, characterized in that: include: A processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor communicates with the memory via the bus, and the processor executes the machine-readable instructions to perform the abnormal image detection method as described in any one of claims 1 to 6.
9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method for detecting abnormal images according to any one of claims 1 to 6 is executed.
