Test method and device, electronic equipment and computer readable medium

Abstract

The invention discloses a test method and device, electronic equipment and a computer readable medium, and relates to the technical field of computers, the method comprises the following steps: determining each multimedia file format supported by a target operating system for testing; randomly determining at least one multimedia file format from the multimedia file formats supported to be tested, and taking the at least one multimedia file format as a target multimedia file format; obtaining a to-be-tested sample of the target multimedia file format; based on the to-be-tested sample, obtaining at least one piece of test data of the target multimedia file format; and based on the at least one test data of the target multimedia file format, performing a fuzzy test on the target multimedia file format through a fuzzy test tool to obtain a test result of the target multimedia file format. Therefore, the multi-media file formats do not need to be written manually in advance, and the corresponding test data can be automatically generated for each supported multi-media file format and the test can be completed.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and more specifically, to a testing method, device, electronic device and computer-readable medium. Background Art

[0002] At present, in the field of testing technology, fuzz testing is usually used to generate fuzzy test cases and expand the test cases. Among them, the test objects of file format fuzz testing include web browsers, mail servers, office components and media players. The goal of file format fuzz testing is to discover vulnerabilities that occur when the application parses a specific file format. Summary of the invention

[0003] The present application proposes a testing method, device, electronic device and computer-readable medium to improve the above-mentioned defects.

[0004] In a first aspect, an embodiment of the present application provides a testing method applied to fuzzy testing of multimedia file formats, the method comprising: determining various multimedia file formats supported for testing by a target operating system; arbitrarily determining at least one multimedia file format from the various multimedia file formats supported for testing as a target multimedia file format; obtaining a sample to be tested of the target multimedia file format; based on the sample to be tested, obtaining at least one test data of the target multimedia file format; based on the at least one test data of the target multimedia file format, performing a fuzzy test on the target multimedia file format through a fuzzy testing tool to obtain a test result of the target multimedia file format.

[0005] In a second aspect, an embodiment of the present application further provides a testing device for fuzz testing of multimedia file formats, the device comprising: a first determination unit, a second determination unit, an acquisition unit, a mutation unit and a testing unit. The first determination unit is used to determine the various multimedia file formats supported for testing by the target operating system. The second determination unit is used to arbitrarily determine at least one multimedia file format from the various multimedia file formats supported for testing as the target multimedia file format. The acquisition unit is used to obtain a sample to be tested of the target multimedia file format. The mutation unit is used to obtain at least one test data of the target multimedia file format based on the sample to be tested. The testing unit is used to perform a fuzzy test on the target multimedia file format through a fuzzy testing tool based on the at least one test data of the target multimedia file format to obtain a test result of the target multimedia file format.

[0006] In a third aspect, an embodiment of the present application also provides an electronic device, comprising: one or more processors; a memory; one or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the one or more processors, and the one or more applications are configured to execute the above method.

[0007] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, wherein the readable storage medium stores a program code executable by a processor, and when the program code is executed by the processor, the processor executes the above method.

[0008] The test method, device, electronic device and computer-readable medium provided by the present application determine the various multimedia file formats supported by the target operating system for testing; arbitrarily determine at least one multimedia file format from the various multimedia file formats supported for testing as the target multimedia file format; obtain a sample to be tested of the target multimedia file format; based on the sample to be tested, obtain at least one test data of the target multimedia file format; based on the at least one test data of the target multimedia file format, perform a fuzzy test on the target multimedia file format through a fuzzy testing tool to obtain the test result of the target multimedia file format. Therefore, it is possible to automatically generate the corresponding test data for all multimedia file formats supported by the target operating system, and test the multimedia file format for any multimedia file format determined in the supported multimedia file formats, so there is no need to manually write the multimedia file format in advance, and the corresponding test data can be automatically generated for each supported multimedia file format and the test can be completed.

[0009] Other features and advantages of the embodiments of the present application will be described in the subsequent description, and partly become apparent from the description, or can be understood by practicing the embodiments of the present application. The purposes and other advantages of the embodiments of the present application can be realized and obtained by the structures specifically pointed out in the written description, claims, and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0011] Figure 1 A method flow chart of a testing method provided by an embodiment of the present application is shown;

[0012] Figure 2A method flow chart of a testing method provided by another embodiment of the present application is shown;

[0013] Figure 3 A schematic diagram of a test system provided by an embodiment of the present application is shown;

[0014] Figure 4 An application schematic diagram of a test system provided by an embodiment of the present application is shown;

[0015] Figure 5 A schematic diagram of verification provided by an embodiment of the present application is shown;

[0016] Figure 6 A variation schematic diagram provided in an embodiment of the present application is shown;

[0017] Figure 7 A variation flow chart provided by an embodiment of the present application is shown;

[0018] Figure 8 A monitoring schematic diagram provided by an embodiment of the present application is shown;

[0019] Fig. 9 The overall flow chart of the testing method provided by an embodiment of the present application is shown;

[0020] Fig.10 A module block diagram of a testing device provided by an embodiment of the present application is shown;

[0021] Fig.11 A schematic diagram of an electronic device provided by an embodiment of the present application is shown;

[0022] Fig.12 A schematic diagram of a computer-readable medium provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0023] In order to make those skilled in the art better understand the present application scheme, the technical scheme in the present application embodiment will be clearly and completely described below in conjunction with the drawings in the present application embodiment. Obviously, the described embodiment is only a part of the present application embodiment, rather than all the embodiments. The components of the present application embodiment usually described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the present application for protection, but merely represents the selected embodiment of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present application.

[0024] It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings. At the same time, in the description of this application, the terms "first", "second", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance.

[0025] At present, in the field of testing technology, fuzz testing is usually used to generate fuzzy test cases and expand the test cases. Among them, the test objects of multimedia file format fuzz testing include web browsers, mail servers, office components and media players. The goal of file format fuzz testing is to discover vulnerabilities that occur when the application parses a specific file format.

[0026] The existing file format fuzz tests mainly include: fuzz tests based on Monkey file format parsing; file format fuzz tests based on Peach; and fuzz tests based on AFL.

[0027] The monkey test is a program that runs on a simulator or device. It generates a stream of pseudo-random events (such as clicks, touches, gestures, and other system-level events). It is mainly used for app stress testing to observe the stability and reliability of the application under test. The monkey runs on a device or simulator and can be run independently of a PC (the common practice is to use the monkey as a test tool that sends random keystroke messages to the application under test. Verify whether the application under test will flash back or crash in the face of these random inputs).

[0028] Peach is a smart fuzz tester (SmartFuzzer) with two fuzz testing capabilities: growth-based fuzz testing and mutation-based fuzz testing. Peach needs to create some PeachPits files, which define the structure, type information and relationships in the fuzz tested data. Peach calls the file used for data definition Peachpit file. When using Peach, define such an XML file to instruct the Peach test platform to do the test. This file needs to be written manually. For different file formats, corresponding file format pits files need to be written.

[0029] AFL (American Fuzzy Lop) is a coverage-guided fuzz testing tool that records the code coverage of input samples, thereby adjusting input samples to improve coverage and increase the probability of discovering vulnerabilities. Specifically, when compiling the program from the source code, it inserts stubs to record code coverage; selects some input files and adds them to the input queue as the initial test set; "mutates" the files in the queue according to a certain strategy; if the coverage of the mutated file is updated, it is retained and added to the queue; the above process will continue in a loop, and the files that trigger the crash will be recorded during the process.

[0030] However, the inventors found in their research that the current fuzz testing scheme for multimedia file format parsing has the following disadvantages for the fuzz testing methods for files in conventional and well-known formats: the Monkey-based fuzz testing framework and its scheme are too dependent on the quality and quantity of samples. Once the sample data structure is not very different, the effect will not be very ideal; the peach-based fuzz testing scheme requires the user to be very familiar with the specific data structure of the multimedia file format of the fuzzer, and to write the pits file based on the familiarity with the file structure. Fuzzers cannot be used for some closed-source file structures. AFL is widely used in fuzz testing, but it requires manual writing of test cases for each codec of the multimedia file format, which cannot fully achieve the effect of automation and cannot guarantee efficiency.

[0031] Therefore, in order to overcome the above-mentioned fuzzy testing of multimedia file formats, it is necessary to manually write test cases in advance, which leads to defects such as low testing efficiency or high difficulty. The embodiment of the present application provides a testing method that can target all multimedia file formats supported by the target operating system, and can automatically generate corresponding test data for any determined multimedia file format among the supported multimedia file formats, and test its multimedia file format. Therefore, there is no need to manually write the multimedia file format in advance, and can automatically generate corresponding test data for each supported multimedia file format and complete the test.

[0032] Specifically, see Figure 1 , Figure 1 A testing method provided by an embodiment of the present application is shown, and the method is applied to fuzzy testing of multimedia file formats, wherein the multimedia file formats include file formats that can be processed by Web browsers, mail servers, office components, and media players, such as text, audio files, video files, and images. Specifically, the method includes: S101 to S105.

[0033] S101: Determine various multimedia file formats supported by the target operating system for testing.

[0034] It should be noted that the fuzz testing tools mentioned above support a limited number of multimedia file formats for testing. Usually, the fuzz testing is developed based on a certain operating system, which can be named as the native operating system here. The target operating system can be an operating system developed on the basis of the native operating system. For example, other instruction sets are added to the instruction set of the native operating system, or new functional modules are added to the functional modules of the native operating system. Of course, it also includes adding new multimedia file formats based on the multimedia file formats supported by the native operating system. In other words, adding new multimedia file formats that can be encoded and decoded based on the multimedia file formats supported by the native operating system. For example, a new multimedia file format is customized in the target operating system and a codec program for the newly added multimedia file format is added.

[0035] Since the fuzz testing tool is developed based on the native operating system, that is, the native operating system is the operating environment of the fuzz testing tool, the fuzz testing tool can support the testing of multimedia file formats supported by the native operating system. Therefore, before fuzz testing the multimedia file format of the target operating system, it is necessary to first determine the various multimedia file formats supported by the target operating system to avoid the test failure caused by the target operating system being unable to successfully parse the multimedia file format when testing a certain multimedia file format.

[0036] Exemplarily, determining each multimedia file format supported for testing by the target operating system may be performed by comparing the multimedia file formats supported for processing by the target operating system with the multimedia file formats supported for testing by the fuzz testing tool. Specifically, the implementation method for determining each multimedia file format supported for testing by the target operating system is to obtain a first multimedia file list of the target operating system, the first multimedia file list including identifiers of all multimedia file formats supported for decoding by the target operating system; obtain a second multimedia file list corresponding to the fuzz testing tool, the second multimedia file list including identifiers of all multimedia file formats supported for testing by the fuzz testing tool; and obtain each multimedia file format supported for testing by the target operating system by comparing the first multimedia file list with the second multimedia file list. The identifier of the multimedia file format may be the name of the multimedia file format.

[0037] Specifically, the target operating system corresponds to a first multimedia file list, which may be a codec list of multimedia files of the target operating system, including the formats of multimedia files that can be coded and decoded by the target operating system. For example, the first multimedia file list is shown in Table 1 below, and the second multimedia file list is shown in Table 2 below.

[0038] Table 1

[0039] Supported audio formats Supported video formats Audio Format 1 Video Format 1 Audio Format 2 Video Format 2 Audio Format 3 Video Format 3 Audio Format 4 Video Format 4 Audio Format 5 Video Format 5 Audio Format 6 Video Format 6

[0040] It should be noted that the audio format may include MP3, WMA, WAV, APE, FLAC, OGG, AAC, etc., and the video format may include MPG, AVI, MOV, MP4, RM, OGG, TTA, etc. It can be understood that the audio formats 1 to 6 and video formats 1 to 6 shown in Table 1 are references to multiple current audio or video formats, and are used to distinguish different audio formats and video formats. Specifically, they may be any of the aforementioned formats or currently existing audio and video formats, and are not limited to this.

[0041] Table 2

[0042]

[0043]

[0044] Exemplarily, the above-mentioned Table 1 is an example of a first multimedia file list, which shows all multimedia file formats that the target operating system supports decoding. The above-mentioned Table 2 is an example of a second multimedia file list, which shows the identifiers of all multimedia file formats that the fuzz testing tool supports testing. By comparing Table 1 and Table 2, it can be seen that audio format 1, audio format 2 and audio format 4 in Table 1 also exist in Table 2, and video format 1, video format 2, video format 5 and video format 6 in Table 1 also exist in Table 2. Therefore, the various multimedia file formats supported for testing by the target operating system include audio format 1, audio format 2, audio format 4, video format 1, video format 2, video format 5 and video format 6.

[0045] As an implementation method, considering that the fuzz testing tool is developed based on the native operating system, the multimedia file formats that the fuzz testing tool can support for testing are usually multimedia file formats that the native operating system can decode. Therefore, the implementation method of obtaining the second multimedia file list corresponding to the fuzz testing tool can be to obtain the third multimedia file list of the native operating system as the second multimedia file list, and the third multimedia file list includes the identifiers of all multimedia file formats that the native operating system supports decoding, wherein the fuzz testing tool is developed based on the native operating system. Among them, the target operating system can also be a derivative system of the native operating system, that is, a target operating system developed on the basis of the native operating system, for example, the target operating system is the Color OS operating system, and the native operating system is the Android operating system.

[0046] S102: arbitrarily determine at least one multimedia file format from various multimedia file formats supported for testing as a target multimedia file format.

[0047] It should be noted that the target multimedia file format can be one or more, and can be any one, that is, each multimedia file format supported by the target operating system determined above can be tested by the test method mentioned in the embodiment of the present application. Therefore, the embodiment of the present application can select one multimedia file format from each multimedia file format supported by the target operating system for testing in turn as the target multimedia file format for testing, or select some multimedia file formats as the target multimedia file format for testing, which is not limited here, that is, no matter which multimedia file format or formats are selected as the target multimedia file format, the test method provided in the embodiment of the present application is applicable.

[0048] S103: Obtain a sample to be tested in the target multimedia file format.

[0049] The sample to be tested may be a file in a target multimedia file format. For example, if the target multimedia file format is MP4, the sample to be tested is a video file in MP4 format.

[0050] S104: Based on the sample to be tested, obtain at least one test data in the target multimedia file format.

[0051] As an implementation method, the test data may be obtained through the sample to be tested, for example, the sample to be tested may be used as a test data, or other samples may be generated based on the sample to be tested, and the sample to be tested and other generated samples may be used as test data. Specifically, generating other samples based on the sample to be tested may be obtained through a mutation algorithm, specifically, changing at least part of the sample data in the sample to be tested to obtain a new sample.

[0052] S105: Based on at least one test data of the target multimedia file format, perform a fuzzy test on the target multimedia file format by using a fuzzy testing tool to obtain a test result of the target multimedia file format.

[0053] The test data is used as the input of the fuzz testing tool, and the object to be tested is tested based on the test data. For example, the object to be tested is a target application, and the fuzz testing tool needs to test the running data of the target application when the target application runs the test data, for example, whether a freeze or flashback occurs. The function of the fuzz testing tool is to simulate the motion environment of the target application when the target application runs the test data, for example, operations such as being clicked, downloading data, etc., so as to test the running data of the target application, and the running data of the target application is the test result of the target multimedia file format. For example, if the running data is a freeze, the test result can characterize that the target application will freeze when running the file in the target multimedia file format. Specifically, the test result can also be used to analyze the specific code that generates the exception, which is specifically described in the subsequent embodiments.

[0054] It can be seen that in the embodiment of the present application, at least one test data of the target multimedia file format can be automatically generated based on the sample to be tested of the target multimedia file format, and the test data can be automatically generated for each multimedia file format supported by the target operating system to avoid manually writing the test data for each multimedia file format. Therefore, the embodiment of the present application can automatically generate the corresponding test data for all multimedia file formats supported by the target operating system, and test the multimedia file format for any multimedia file format determined in the supported multimedia file formats, so there is no need to manually write the multimedia file format in advance, and the corresponding test data can be automatically generated for each supported multimedia file format and the test can be completed.

[0055] Specifically, see Figure 2 , Figure 2A testing method provided by an embodiment of the present application is shown, and the method is applied to fuzzy testing of multimedia file formats, wherein the multimedia file formats include file formats that can be processed by Web browsers, mail servers, office components, and media players, such as text, audio files, video files, and images. Specifically, the method includes: S201 to S209.

[0056] Before introducing the method used in the embodiment of the present application, the test system used in the method embodiment of the present application is first introduced. Specifically, Figure 3 As shown, the test system may include an input module 301, a test module 302 and a monitoring and analysis module 303, wherein the test module 302 may be a fuzzer engine. Specifically, the input module 301 is used as the input of the test module 302 to mutate the sample through the file mutation strategy to obtain the mutated sample data, wherein: Figure 3 The seed in the above-mentioned sample data is mutated by the mutation algorithm selected by the mutation strategy to obtain the mutated sample data, i.e., the test data, which is then used as the input parameter of the test module 302. The test module 302 is used to perform fuzzy testing on the test object based on the test data, that is, to test whether an exception occurs when the test object runs the test data, thereby obtaining the log file corresponding to each test data, and the monitoring and analysis module 303 analyzes the abnormal log and locates the specific abnormal code, that is, to find the bug. The general process of testing through the test system can be as follows: Figure 4 As shown, a sample set (i.e., loading seeds) is obtained through a crawler, such as an existing poc file, a public media library, and a carefully constructed multimedia file as input. Through file verification, sample preprocessing is performed, a mutation mode is selected for mutation, and the mutation file is used as a parameter of the fuzz testing tool (fuzz engine). The log information is monitored, and the correspondence between the sample and the log is marked. Once an exception (crash) occurs, the crash log is parsed, the sample file is saved, and the test (fuzz) is continued. The test data (poc file) corresponding to the generated crash is obtained, and the sample file is used as a sample file for mutation in a loop, and the vulnerability location point and test data (poc file) are output. Specifically, in the subsequent embodiments, the role of each module of the test system in each step is introduced.

[0057] S201: Determine various multimedia file formats supported by the target operating system for testing.

[0058] S202: Determine at least one multimedia file format from among the multimedia file formats supported for testing as a target multimedia file format.

[0059] The implementation of S201 and S202 may refer to the aforementioned embodiments and will not be described in detail here.

[0060] S203: Acquire a sample to be selected in the target multimedia file format.

[0061] It should be noted that the selected samples of the target point multimedia file format can be the initial sample set of the target point multimedia file format, that is, the collected file format is the target multimedia file format. Specifically, the initial sample set can be automatically obtained through the following ways: obtaining from some online sites that provide sample sets; crawling a large number of sample files through the syntax of search engines; some open source projects will bring some test cases to test the program; sample files generated when fuzzing other similar software; bug submission pages of target programs or similar programs; generated with format conversion tools (unconventional file formats). Among them, common URLs for obtaining sample files can include those shown in Table 3 below:

[0062] Table 3

[0063] Website 1 https: / / files.fuzzing-project.org / Website 2 http: / / samples.ffmpeg.org / Website 3 http: / / 1camtuf.coredump.cx / af1 / demo / Website 4 https: / / github.com / Cisco-Talos / clamav-fuzz-corpus Website 5 https: / / github.com / mozilla / pdf.js / tree / master / test / pdfs Website 6 https: / / github.com / MozillaSecurity / fuzzdata Website 7 https: / / github.com / strongcourage / fuzzing-corpus Website 8 https:1lgithub.com / codelibs / fess-testdata

[0064] S204: If the sample to be selected passes the verification, the sample to be selected is used as a sample to be tested in the target multimedia file format.

[0065] Considering that the obtained samples may not be qualified samples, that is, the quality of the samples cannot meet normal usage requirements. Specifically, the format of the file is changed by arbitrarily changing the file suffix, but the file structure does not meet the file structure corresponding to the format of the changed suffix. For example, the suffix of other file types is changed to .mp4 or .mp3, but it actually cannot meet the container and parsing of files in a specific format. And such low-quality samples, for example, samples of the target multimedia file format obtained by arbitrarily changing the file suffix, that is, seed files, are deformed data. Most of the deformed data cannot break through the checksum, fixed field or length in the file format, and are forced to exit early during the program running process. Therefore, it is necessary to ensure that the variant sample can pass the verification of the code itself (to ensure the validity of the generated sample). In order to ensure the validity of the sample generation, this situation can be prevented by preliminary filtering and screening of the Seed file.

[0066] Exemplarily, the sample to be selected is verified, and if the sample to be selected passes the verification, the sample to be selected is used as a sample to be tested in the target multimedia file format. Figure 5As shown, "loading seeds" is to obtain the candidate samples of the target multimedia file format, and then verify them through filtering / preprocessing. The obtained verified samples are the candidate samples that have passed the verification, that is, the test samples of the target multimedia file format.

[0067] As an implementation method, the selected sample can be verified by the instruction tool. If the selected sample passes the verification, the selected sample is used as the sample to be tested in the target multimedia file format. The implementation method is that the selected sample is parsed by the instruction tool to obtain a parsing result; the verification result of the selected sample is obtained based on the parsing result; if the verification result is that the verification passes, the selected sample is used as the sample to be tested in the target multimedia file format. Exemplarily, the instruction tool can parse the selected sample. The instruction tool can be a playback tool of the target multimedia file format, which can parse the file corresponding to the target multimedia file format during the playback process and present the parsing result. Specifically, the parsing result can reflect the parameters related to the selected sample obtained after the instruction tool parses the selected sample, such as storage address, number of frames, etc. If the instruction tool cannot successfully parse the selected sample, the parsing failure will be indicated in the parsing result. Therefore, by analyzing the parsing result, the verification result of the selected sample can be obtained.

[0068] Exemplarily, when the command tool cannot successfully parse the sample to be selected, the parsing result will contain specified content, which is used to indicate that the current sample to be selected cannot be successfully parsed. Therefore, the implementation method of obtaining the verification result of the sample to be selected based on the parsing result is to determine whether the specified content exists in the parsing result; if the specified content does not exist, the verification result of the sample to be selected is determined to be passed; if the specified content exists, the verification result of the sample to be selected is determined to be failed. For example, the command tool is the stagefright command line, which is a small tool for parsing audio and video under Android. The parsing result is the echo and output result of the command line. For example, the parsing result is:

[0069] $ / system / bin / stagefright Disco.240p.mp4 thumbnailTime:0us(0.00secs)

[0070] AVC video profile 66and level 13format changed.

[0071] avg.180.73fps

[0072] avg.time to decode one buffer 5485.86uses decoded a total of 304frame(s)

[0073] It should be noted that the samples that can be parsed successfully meet the data structure of the multimedia format. Those that cannot be parsed will have obvious error output (keywords such as failed). Therefore, the specified content can be keywords such as failed in the parsing result. If there are keywords such as failed in the above parsing result, the verification result of the file is passed.

[0074] Therefore, through the above verification process, the candidate samples that have passed the verification can be filtered out from the candidate samples of the target multimedia file format and used as the samples to be tested in the target multimedia file format.

[0075] S205: Modify the sample data of the sample to be tested.

[0076] It should be noted that the sample to be tested can be regarded as a data set composed of multiple sample data, usually as an array, and the sample data of the sample to be tested can be changed by changing the numerical value in the data. Exemplarily, by modifying the sample to be tested, multiple mutated sample data can be obtained on the basis of a file. Specifically, the sample data of the sample to be tested is randomly changed to obtain at least one modified sample data of the sample to be tested. Exemplarily, the random change of the sample data of the sample to be tested can be to change the sample data of the sample to be tested by random data replacement, insertion, deletion, numerical increase or decrease, boundary value replacement, copy and overwrite. Figure 6 As shown, "33" and "44" in a set of data [11,22,33,44,55,66] in the sample data of the sample to be tested are changed to "FF" and "BF" respectively, so that the updated sample data of the sample to be tested is [11,22,FF,BF,55,66].

[0077] Specifically, Figure 7As shown, first determine whether random mutation is supported (i.e., execute is_random_mutate). If random mutation is not supported, obtain the seed template, i.e., run seed_pits_template(file), so as to obtain the existing pit file, which can be the test data used for multimedia file testing, and further, can be the test data used for multimedia file testing and abnormal. Then, mutate the pixel data of the obtained seed template (i.e., pit file) (i.e., execute mutate_pit_xml) to obtain the mutated seed template, and then perform the operation of matching the file type template (i.e., match file type template), i.e., find the mutated seed template matching the target multimedia file format as the test data. If random mutation is supported, perform the operation of obtaining the seed generator, i.e., seed_radamsa(file), and then select the random mutation pattern (i.e., choose a random mutation pattern), modify the sample data of the sample to be tested in combination with the seed generator, and obtain at least one modified sample data of the sample to be tested.

[0078] Therefore, by means of random mutation, the sample to be tested can be used as a seed to obtain a plurality of modified sample data of the sample to be tested.

[0079] S206: Obtain at least one test data in the target multimedia file format based on at least one modified sample data of the sample to be tested.

[0080] As an implementation manner, at least one modified sample data of the sample to be tested obtained by modifying the sample data of the sample to be tested can be directly used as test data in the target multimedia file format.

[0081] However, considering that the randomly mutated sample data is very likely to destroy the basic data structure of the specific format file of the sample to be tested, it will not be able to pass the detection of the specific field in the codec, and thus cannot be successfully decoded, and further cannot be successfully used for testing. Therefore, as an implementation method, the text data structure of the feature can be mutated. Specifically, based on the modified sample data of the sample to be tested, at least one test data of the target multimedia file format is obtained in the following implementation method: based on a text template, at least one of the modified sample data of the sample to be tested is generated into at least one designated file; based on the parsing tool corresponding to the text template, at least one designated file is parsed; and the data corresponding to the designated file obtained after successful parsing is used as a test data of the target multimedia file format.

[0082] Exemplarily, the text template is a template of a description document of the target multimedia file format, for example, a pit file of peach, and then, at least one specified file is generated from at least one of the modified sample data of the sample to be tested through the text template. Exemplarily, a structure of the sample data of the sample to be tested is obtained, and then, the structure is described with a text template (for example, a pit file) to obtain a specified file.

[0083] For example, the structure of the sample data of the sample to be tested is as follows:

[0084]

[0085] Wherein, len is the modified sample data of the sample to be tested, which is an array.

[0086] The structure is described in the pit file as follows:

[0087]

[0088] After obtaining the designated file, at least one of the designated files is parsed based on the parsing tool corresponding to the text template, and then the data corresponding to the designated file obtained after successful parsing is used as a test data of the target multimedia file format, while the designated file that is not successfully parsed will be discarded, that is, it cannot be used for subsequent test data, so as to avoid the sample data of the modified sample to be tested from being unable to be successfully decoded later due to the destruction of the file format.

[0089] For example, after the pit file is written, it can be parsed with peach to generate sample data. The test data generated by the pit file is as follows. It can be seen that the generated data is consistent with the default value described in the pit file.

[0090] ID:78 59 34 12;

[0091] Size:08 00 00 00;

[0092] Data:aa bb cc dd ee ff 11 22

[0093] Therefore, through the analysis and research (automation) of each multimedia file format, template pits files for various file formats can be well written. Combined with the peach tool, the tested samples can be mutated to obtain test data for each multimedia file format.

[0094] S207: Based on at least one test data of the target multimedia file format, perform a fuzzy test on the target multimedia file format by using a fuzzy testing tool to obtain a test result of the target multimedia file format.

[0095] S208: When it is detected that the test result is abnormal, determine the test data where the abnormality occurs and the log file corresponding to the test data.

[0096] It should be noted that, in the process of fuzzy testing at least one test data in the target multimedia file format, the working status corresponding to each test data will be monitored in real time, wherein the working status corresponding to the test data refers to the working status of the test object of the analog-to-digital test when running the test data. Assuming that the test data is an MP4 video file 1 and the test object is a video playback software, the working status corresponding to the test data is the working status of the video playback software when the video playback software plays the MP4 video file 1.

[0097] Since the working status of each test data is monitored in real time, once the working status is detected to be abnormal, the test data corresponding to the abnormal status can be immediately determined. Then, the log file corresponding to the test data is obtained, that is, the working log of the object to be tested simulated by the fuzz testing tool when running the test data. Figure 8 As shown in the figure, during the process of fuzzing the test object, there will be a monitor Monitor (log detection tool: specifically parse the changes in the logs in the / data / tombstone / directory) that will always detect the working status of the target. Once an exception occurs, the console will output the abnormal status information in order to more quickly and accurately determine the corresponding relationship between the deformed poc file (i.e., test data) and the log that caused the target exception.

[0098] S209: Analyze the log file and locate the abnormal code.

[0099] As an implementation method, the log analysis module of the poc---log---locates the abnormal code position, thereby realizing the vulnerability location function. Specifically, combined with the symbolic processing tool llvm-symbolizer that comes with Android, it can be parsed to the specific line number, that is, the abnormal code is located.

[0100] Specifically, Fig. 9As shown, first, get the seed (i.e. execute get_seed(url_list)), determine whether the seed passes the verification (i.e. execute is_validity_check), if the verification passes, then execute to get the screened seed (i.e. execute get_seed_list), if the verification fails, then discard it, after getting the screened seed (i.e. execute get_seeds_list), execute the compilation algorithm (mutation algorithm), get the structure ( / system / bin / stagefright), then further get the test data (all_sorts_decode(seed)), then, perform the test (crash), get the tested file (get_uniq_crash), continue to use the tested file as the screened seed, continue to mutate, and get new test data. During the test process, monitor the working status of the test data (i.e. execute monitor), output the abnormal status (watch_tombstones(path)), and then determine the abnormal content (get_poc_bugline(tombstones)), that is, obtain the abnormal test data and the log file corresponding to the test data, then locate the code line (locate_bug_line), and output the report (reported), which may include abnormal code, test data and log file.

[0101] Therefore, the embodiment of the present application can not only target all multimedia file formats supported by the target operating system, but also automatically generate corresponding test data for any multimedia file format in the supported multimedia file format, and test the multimedia file format, so there is no need to manually write the multimedia file format in advance, and can automatically generate corresponding test data for each supported multimedia file format and complete the test. In addition, the problem of seed file validity can be solved by pre-processing verification in the system simulator (ensuring that the checksum, fixed field or length in the file format are limited to the logical verification in the multimedia codec); the new technology based on the data structure template of the multimedia file structure solves the validity and richness of seed file variation, and the new technology includes two aspects: automatically generating a variation template of the file type data structure for multimedia files of different file formats; the generated file type template can be used to automatically generate Seed file variation to provide input for fuzzy testing; finally, the working status of the target is always detected through Monitor (log monitoring module: specifically analyzing the changes in the log in the / data / tombstone / directory), and once an exception occurs, the stack information of the crash log will be automatically symbolized, and the specific location of the specific problem code will be finally located.

[0102] See also Fig.10 , which shows a structural block diagram of a testing device 1000 provided in an embodiment of the present application, the device may include: a first determination unit 1001, a second determination unit 1002, an acquisition unit 1003, a variation unit 1004 and a testing unit 1005.

[0103] The first determining unit 1001 is used to determine various multimedia file formats supported by the target operating system for testing.

[0104] Furthermore, the first determination unit 1001 is also used to obtain a first multimedia file list of the target operating system, the first multimedia file list including identifiers of all multimedia file formats that the target operating system supports decoding; obtain a second multimedia file list corresponding to the fuzzy testing tool, the second multimedia file list including identifiers of all multimedia file formats that the fuzzy testing tool supports testing; and obtain each multimedia file format supported by the target operating system for testing by comparing the first multimedia file list with the second multimedia file list.

[0105] Furthermore, the first determination unit 1001 is also used to obtain a third multimedia file list of the native operating system as the second multimedia file list, the third multimedia file list including identifiers of all multimedia file formats supported for decoding by the native operating system, wherein the fuzz testing tool is developed based on the native operating system.

[0106] The second determining unit 1002 is configured to determine at least one multimedia file format from among the multimedia file formats supported for testing as a target multimedia file format.

[0107] The acquisition unit 1003 is used to acquire the sample to be tested in the target multimedia file format.

[0108] Furthermore, the acquisition unit 1003 is further configured to acquire a sample to be selected in the target multimedia file format; if the sample to be selected passes verification, the sample to be selected is used as a sample to be tested in the target multimedia file format.

[0109] Furthermore, the acquisition unit 1003 is also used to parse the sample to be selected through the instruction tool to obtain a parsing result; obtain a verification result of the sample to be selected based on the parsing result; if the verification result is a passed verification, the sample to be selected is used as a sample to be tested in the target multimedia file format.

[0110] Further, the acquisition unit 1003 is further used to determine whether there is a specified content in the parsing result; if there is no specified content, the verification result of the sample to be selected is determined to be passed; if there is a specified content, the verification result of the sample to be selected is determined to be failed. The command tool is the stagefright command line.

[0111] The mutation unit 1004 is configured to obtain at least one test data in the target multimedia file format based on the sample to be tested.

[0112] Furthermore, the mutation unit 1004 is further configured to modify the sample data of the sample to be tested; and obtain at least one test data in the target multimedia file format based on at least one modified sample data of the sample to be tested.

[0113] Furthermore, the mutation unit 1004 is also used to generate at least one designated file based on a text template from at least one of the modified sample data of the sample to be tested; parse at least one of the designated files based on a parsing tool corresponding to the text template; and use the data corresponding to the designated file obtained after successful parsing as a test data of the target multimedia file format.

[0114] Furthermore, the mutation unit 1004 is further configured to randomly change the sample data of the sample to be tested to obtain at least one modified sample data of the sample to be tested.

[0115] The testing unit 1005 is used to perform a fuzzy test on the target multimedia file format through a fuzzy testing tool based on at least one test data of the target multimedia file format to obtain a test result of the target multimedia file format.

[0116] Furthermore, the test unit 1005 is used to determine the test data with the anomaly and the log file corresponding to the test data when the test result is detected to be abnormal; and analyze the log file to locate the abnormal code.

[0117] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described devices and modules can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here.

[0118] In several embodiments provided in the present application, the coupling between modules may be electrical, mechanical or other forms of coupling.

[0119] In addition, each functional module in each embodiment of the present application can be integrated into a processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The above integrated modules can be implemented in the form of hardware or software functional modules.

[0120] Please refer to Fig.11 , which shows a structural block diagram of an electronic device provided in an embodiment of the present application. The electronic device 100 may be an electronic device capable of running applications, such as a smart phone, a tablet computer, an e-book, etc. The electronic device 100 in the present application may include one or more of the following components: a processor 110, a memory 120, and one or more applications, wherein the one or more applications may be stored in the memory 120 and configured to be executed by one or more processors 110, and the one or more programs are configured to execute the method described in the aforementioned method embodiment.

[0121] The processor 110 may include one or more processing cores. The processor 110 uses various interfaces and lines to connect various parts of the entire electronic device 100, and executes various functions and processes data of the electronic device 100 by running or executing instructions, programs, code sets or instruction sets stored in the memory 120, and calling data stored in the memory 120. Optionally, the processor 110 can be implemented in at least one hardware form of digital signal processing (DSP), field-programmable gate array (FPGA), and programmable logic array (PLA). The processor 110 can integrate one or a combination of a central processing unit (CPU), a graphics processing unit (GPU), and a modem. Among them, the CPU mainly processes the operating system, user interface, and application programs; the GPU is responsible for rendering and drawing display content; and the modem is used to process wireless communications. It can be understood that the above-mentioned modem may not be integrated into the processor 110, but may be implemented separately through a communication chip.

[0122] The memory 120 may include a random access memory (RAM) or a read-only memory (ROM). The memory 120 may be used to store instructions, programs, codes, code sets or instruction sets. The memory 120 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for implementing at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the following various method embodiments, etc. The data storage area may also store data (such as a phone book, audio and video data, chat record data) created by the electronic device 100 during use.

[0123] Please refer to Fig.12 , which shows a structural block diagram of a computer-readable medium provided in an embodiment of the present application. The computer-readable medium 800 stores program codes, which can be called by a processor to execute the method described in the above method embodiment.

[0124] The computer readable medium 800 can be an electronic memory such as a flash memory, an EEPROM (electrically erasable programmable read-only memory), an EPROM, a hard disk, or a ROM. Optionally, the computer readable medium 800 includes a non-transitory computer-readable storage medium. The computer readable medium 800 has storage space for program code 810 that performs any method steps in the above method. These program codes can be read from or written to one or more computer program products. The program code 810 can be compressed, for example, in an appropriate form.

[0125] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A testing method, characterized in that: The method is applied to fuzzy testing of multimedia file formats, and comprises: Determine the various multimedia file formats supported by the target operating system for testing; Determine at least one multimedia file format from among the multimedia file formats supported for testing as a target multimedia file format; Obtaining a sample to be tested in the target multimedia file format; Based on the sample to be tested, obtaining at least one test data in the target multimedia file format; Based on at least one test data of the target multimedia file format, a fuzzy test is performed on the target multimedia file format through a fuzzy test tool to obtain a test result of the target multimedia file format.

2. The method according to claim 1, characterized in that The step of determining each multimedia file format supported by the target operating system for testing includes: Acquire a first multimedia file list of the target operating system, wherein the first multimedia file list includes identifiers of all multimedia file formats supported for decoding by the target operating system; Obtaining a second multimedia file list corresponding to the fuzz testing tool, wherein the second multimedia file list includes identifiers of all multimedia file formats supported by the fuzz testing tool for testing; By comparing the first multimedia file list with the second multimedia file list, various multimedia file formats supported by the target operating system for testing are obtained.

3. The method according to claim 2, characterized in that The obtaining of a second multimedia file list corresponding to the fuzz testing tool includes: A third multimedia file list of the native operating system is obtained as the second multimedia file list, wherein the third multimedia file list includes identifiers of all multimedia file formats that the native operating system supports decoding, wherein the fuzz testing tool is developed based on the native operating system.

4. The method according to claim 1, characterized in that: The step of obtaining the sample to be tested in the target multimedia file format includes: Obtaining a sample to be selected in the target multimedia file format; If the sample to be selected passes the verification, the sample to be selected is used as a sample to be tested in the target multimedia file format.

5. The method according to claim 4, characterized in that If the sample to be selected passes the verification, the sample to be selected is used as a sample to be tested in the target multimedia file format, including: Analyze the sample to be selected by the command tool to obtain an analysis result; Obtaining a verification result of the sample to be selected based on the analysis result; If the verification result is that the verification is passed, the sample to be selected is used as a sample to be tested in the target multimedia file format.

6. The method according to claim 5, characterized in that The obtaining the verification result of the sample to be selected based on the analysis result includes: Determine whether specified content exists in the parsing result; If the specified content does not exist, determining that the verification result of the sample to be selected is passed; If the specified content exists, it is determined that the verification result of the sample to be selected is verification failure.

7. The method according to claim 5, characterized in that The command tool is the stagefright command line.

8. The method according to claim 1, characterized in that The obtaining, based on the sample to be tested, at least one test data in the target multimedia file format comprises: Modifying the sample data of the sample to be tested; Based on at least one modified sample data of the sample to be tested, at least one test data in the target multimedia file format is obtained.

9. The method according to claim 8, characterized in that The step of obtaining at least one test data in the target multimedia file format based on the modified sample data of the sample to be tested comprises: Based on the text template, generating at least one designated file from at least one of the modified sample data of the sample to be tested; Parsing at least one of the specified files based on a parsing tool corresponding to the text template; The data corresponding to the designated file obtained after successful parsing is used as test data of the target multimedia file format.

10. The method according to claim 8, characterized in that The modifying the sample data of the sample to be tested comprises: The sample data of the sample to be tested is randomly changed to obtain at least one modified sample data of the sample to be tested.

11. The method according to claim 1, characterized in that: After performing a fuzzy test on the target multimedia file format by using a fuzzy test tool based on at least one test data of the target multimedia file format and obtaining a test result of the target multimedia file format, the method further includes: When the test result is detected to be abnormal, the test data in which the abnormality occurs and the log file corresponding to the test data are determined; Analyze the log file and locate the abnormal code.

12. A testing device, characterized in that: The device is applied to fuzzy testing of multimedia file formats, and comprises: A first determining unit, used to determine each multimedia file format supported by the target operating system for testing; A second determining unit, configured to arbitrarily determine at least one multimedia file format from among the multimedia file formats supported for testing as a target multimedia file format; An acquisition unit, used for acquiring a sample to be tested in the target multimedia file format; A mutation unit, configured to obtain at least one test data in the target multimedia file format based on the sample to be tested; The testing unit is used to perform a fuzzy test on the target multimedia file format through a fuzzy testing tool based on at least one test data of the target multimedia file format to obtain a test result of the target multimedia file format.

13. An electronic device, characterized in that: include: one or more processors; Memory; One or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the one or more processors, and the one or more applications are configured to execute the method according to any one of claims 1-11.

14. A computer readable medium, characterized in that The computer-readable medium stores a program code executable by a processor, and when the program code is executed by the processor, the processor executes the method according to any one of claims 1 to 11.

Images