Front-end control coverage detection method, device and equipment

Abstract

The present invention discloses a front-end control coverage detection method, device and equipment. It is mainly based on image recognition technology to realize the recognition and disassembly of the controls contained in the front-end interface image, and insert pre-made tags into each control code. After the front-end request is monitored and the request is executed, if the code runs to the tag, the pre-stored operation record identifier of the control is updated. In this way, the current operation record identifier of each control can be finally read out, and then compared with the control execution standard, the coverage of the control element of the front-end interface can be completed. The present invention can quickly and conveniently identify and disassemble the front-end control information, and realize the detection of code coverage in conjunction with the inserted control tag and the set operation record identifier. The present invention does not need to consume labor costs, nor does it need to add additional testing resources, and when the interactive interface or interactive logic changes, the coverage detection can also be completed conveniently and efficiently.

Description

Technical Field

[0001] The present invention relates to the field of front-end development technology, and in particular to a front-end control coverage detection method, device and equipment. Background Art

[0002] With the widespread application of various businesses involving human-computer interaction, the demand for front-end interface testing has also increased sharply. At the same time, users have higher and higher requirements for interactive functions and ease of use. When the interface involves interaction with users, there will inevitably be a variety of different interaction scenarios due to interactive operation logic or changes in the interactive interface.

[0003] The existing front-end testing methods not only take up a lot of resources, but are also cumbersome to operate and difficult to reasonably control costs. Therefore, how to conveniently and efficiently ensure the quality of front-end code under limited testing resources and accurately evaluate the coverage of front-end controls for various scenarios has become the focus of front-end development. Summary of the invention

[0004] In view of the above, the present invention aims to provide a front-end control coverage detection method, device and equipment, and accordingly proposes a computer-readable storage medium and a computer program product, which are used to economically, efficiently, conveniently and accurately test the scene coverage of front-end controls during the front-end development process.

[0005] The technical solution adopted by the present invention is as follows:

[0006] In a first aspect, the present invention provides a front-end control coverage detection method, comprising:

[0007] Obtain control information of each control contained in the image of the target interface;

[0008] According to the control information, parse out the code of each control;

[0009] Inserting a control tag into the code of each control;

[0010] Monitor user input requests in real time and execute corresponding control functions;

[0011] When the request is processed and the code runs to the control mark of the corresponding control, the preset running record mark of the control is updated;

[0012] Get the final operation record identifier of each control on the target interface, compare it with the set front-end control execution standard, and generate coverage detection results.

[0013] In at least one possible implementation, the acquiring of control information of each control contained in the image of the target interface based on the image recognition technology includes:

[0014] Collect target interface images;

[0015] Extracting features of the target interface image, and determining a number of processing objects in combination with a preset processing object identification strategy;

[0016] The features of each processing object are extracted, and the type information and location information of each control are determined in combination with a preset control identification strategy.

[0017] In at least one possible implementation, the method further includes:

[0018] Generate a control mark corresponding to each control in a one-to-one manner according to the control information and preset rules;

[0019] The control information of each control and the corresponding control mark are stored.

[0020] In at least one possible implementation, the method further includes:

[0021] Compare the generated control tags with the stored control tags;

[0022] If the two are repeated, the currently generated control tag is discarded and a new control tag is regenerated;

[0023] Continue to compare the new control tag with the stored control tag to store the generated and non-duplicate control tag.

[0024] In at least one possible implementation, the front-end control execution standard includes:

[0025] Preliminarily build a control knowledge graph based on various front-end frameworks, various front-end controls, and a large number of front-end control call records, and use the control knowledge graph as the front-end control execution standard;

[0026] or,

[0027] Based on the control information, control knowledge data corresponding to each control of the target interface is extracted from the control knowledge graph, and the front-end control execution standard is formed.

[0028] In at least one possible implementation manner, the operation record identifier includes: an execution identifier; or, the operation record identifier includes: an execution identifier and an activation identifier.

[0029] In at least one possible implementation manner, updating the operation record identifier of the control includes:

[0030] Updating the execution flag of the control;

[0031] or,

[0032] The execution flag and the enabling flag of the control are updated.

[0033] In at least one possible implementation, the operation record identifier further includes: a statistical identifier;

[0034] The comparison with the set front-end control execution standard includes: comparing each control of the target interface with the front-end control execution standard one by one, and after comparing each control, updating the statistical identifier of the control.

[0035] In a second aspect, the present invention provides a front-end control coverage detection device, comprising:

[0036] A control disassembly module is used to obtain control information of each control contained in the image of the target interface;

[0037] A code parsing module, used to parse the code of each control according to the control information;

[0038] A tag insertion module, used for inserting a control tag into the code of each control;

[0039] The request processing module is used to monitor the user input requests in real time and execute the corresponding control functions;

[0040] An identification updating module, used for updating a preset running record identification of a control when the request processing is completed and the code runs to the control tag of the corresponding control;

[0041] The comparison detection module is used to obtain the final operation record identification of each control in the target interface, and compare it with the set front-end control execution standard to generate coverage detection results.

[0042] In at least one possible implementation, the control disassembly module includes:

[0043] An image acquisition unit, used for acquiring an image of a target interface;

[0044] A processing object identification unit, used to extract features of the target interface image and determine a number of processing objects in combination with a preset processing object identification strategy;

[0045] The control identification unit is used to extract the features of each processing object and determine the type information and location information of each control in combination with a preset control identification strategy.

[0046] In at least one possible implementation, the device further includes: a control mark generation module; the control mark generation module specifically includes:

[0047] A tag generating unit, used for generating a control tag corresponding to each control in a one-to-one manner according to the control information and preset rules;

[0048] The information storage unit is used to store the control information of each control and the corresponding control mark.

[0049] In at least one possible implementation, the device further includes: a tag uniqueness checking module; the tag uniqueness checking module specifically includes:

[0050] A tag comparison unit, used to compare the generated control tag with the stored control tag;

[0051] A tag regeneration unit is used to discard the currently generated control tag and regenerate a new control tag if the two are repeated;

[0052] The unique tag storage unit is used to continue comparing the new control tag with the stored control tag to store the generated and non-repetitive control tag.

[0053] In at least one possible implementation, the front-end control execution standard includes:

[0054] Preliminarily build a control knowledge graph based on various front-end frameworks, various front-end controls, and a large number of front-end control call records, and use the control knowledge graph as the front-end control execution standard;

[0055] or,

[0056] Based on the control information, control knowledge data corresponding to each control of the target interface is extracted from the control knowledge graph, and the front-end control execution standard is formed.

[0057] In at least one possible implementation manner, the operation record identifier includes: an execution identifier; or, the operation record identifier includes: an execution identifier and an activation identifier.

[0058] In at least one possible implementation manner, the identification updating module is specifically used to:

[0059] Updating the execution flag of the control;

[0060] or,

[0061] The execution flag and the enabling flag of the control are updated.

[0062] In at least one possible implementation, the operation record identifier further includes: a statistical identifier;

[0063] The comparison and detection module comprises:

[0064] A one-by-one comparison unit, used to compare each control of the target interface one by one with the front-end control according to the standard;

[0065] The identification updating unit is used to update the statistical identification of each control after comparing the control.

[0066] In a third aspect, the present invention provides a front-end control coverage detection device, comprising:

[0067] One or more processors, a memory, and one or more computer programs, wherein the memory may adopt a non-volatile storage medium, wherein the one or more computer programs are stored in the memory, and the one or more computer programs include instructions, and when the instructions are executed by the device, the device executes the method as described in the first aspect or any possible implementation manner of the first aspect.

[0068] In a fourth aspect, the present invention provides a computer-readable storage medium, which stores a computer program, which, when executed on a computer, enables the computer to execute the method as described in the first aspect or any possible implementation of the first aspect.

[0069] In a fifth aspect, the present invention further provides a computer program product, which, when executed by a computer, is used to execute the method in the first aspect or any possible implementation manner of the first aspect.

[0070] In a possible design of the fifth aspect, the relevant programs involved in the product can be stored in whole or in part on a memory packaged together with the processor, or can be stored in whole or in part on a storage medium not packaged together with the processor.

[0071] The idea of ​​the present invention is to realize the recognition and disassembly of the controls contained in the front-end interface image based on image recognition technology, so as to obtain the codes of each control, and then insert a pre-made mark into each control code, and when the front-end request is monitored and the request is executed, if the code runs to the mark, the pre-stored operation record mark of the control is updated, and so on. When all requests are executed, the final operation record mark of each control can be read out, and then compared with the set control execution standard, the detection of the coverage of the control elements of the front-end interface can be completed, that is, the code quality corresponding to the front-end interface can be evaluated. The present invention can quickly and conveniently identify and disassemble the front-end control information, and cooperate with the inserted control mark to realize the control in the process of executing the front-end request. Changes to the operation record mark one by one, and then the code coverage can be compared based on the execution standard after the test. The present invention does not need to consume labor costs, nor does it need to increase additional testing resources, and when the interactive interface or interactive logic changes, it can also complete the coverage detection conveniently and efficiently. BRIEF DESCRIPTION OF THE DRAWINGS

[0072] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention will be further described below with reference to the accompanying drawings, in which:

[0073] Figure 1 A flowchart of an embodiment of a front-end control coverage detection method provided by the present invention;

[0074] Figure 2 A flowchart of a preferred embodiment of the present invention for identifying controls from an image;

[0075] Figure 3 A block diagram of an embodiment of a front-end control coverage detection device provided by the present invention;

[0076] Figure 4 A schematic diagram of an embodiment of the front-end control coverage detection device provided by the present invention. DETAILED DESCRIPTION

[0077] Embodiments of the present invention are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be interpreted as limiting the present invention.

[0078] Before introducing the technical solution of the present invention, it can be explained that the front end can also be tested by installing a plug-in, that is, injecting the code detection script into the front-end code and deploying it for execution, but this process requires the newly generated file to be introduced into the project again, and the output file is obtained, and the test results are viewed by running the output. In actual use, it is found that this method has to frequently verify the deployment and execution results of each step to determine that the code detection script has been injected into the project and deployed successfully at each stage. The disadvantage of this method is that it is relatively cumbersome, not easy to operate, and the verification efficiency is not high.

[0079] In addition, the present invention further analyzes that the interactive interface is very easy to change according to different business needs. When the controls involved in the user interface are changed, based on the traditional end-to-end testing form, it depends to a large extent on the execution results of the server and the relationship between the entire environment and other modules. This will increase the overall investment cost of the front-end testing, especially when the multi-module testing chain is lengthened, the situation is more obvious.

[0080] In view of the above, the present invention aims at how to simply and economically overcome the problem of poor detection of multi-scenario coverage of front-end controls caused by changes in the interactive interface or interactive operations, and proposes to implement interface control parsing based on image recognition technology, and insert tags into the control code to accurately update the corresponding operation record identifier after executing the front-end request. Finally, by comparing the operation record identifier of each control element with the set standard, the coverage of the front-end control for multiple scenarios can be evaluated concisely and accurately.

[0081] The present invention provides at least one embodiment of a front-end control coverage detection method, such as Figure 1 As shown, the following steps may be included:

[0082] Step S1: Acquire control information of each control contained in the image of the target interface.

[0083] Since the birth of artificial intelligence, the application field has been continuously expanding. When designing the present invention, it is considered that image recognition technology is a major category of artificial intelligence, and images are the main source of human acquisition and exchange of information. Therefore, it can combine image recognition and information comprehensive processing capabilities to solve the problems of frequent changes in interactive interfaces and multiple interactive logics, and then promote the evolution of front-end technology to meet the actual development needs of the business. Accordingly, the present invention proposes to use image recognition technology to identify controls on the target interface to be processed. Of course, in actual operation, other preferred links can also be designed in the image recognition process, such as pre-processing the collected image, such as but not limited to denoising, smoothing, transformation and other operations. This link can enhance the boundary and detail information of the image, which helps to improve the accuracy of subsequent control recognition. In addition, it is also possible to consider regional processing of the interface image, that is, splitting the entire interface into several image areas, and then performing subsequent control recognition detection for each image area. This link can unitize the recognition task and perform recognition operations on each area to simplify the computational complexity of the recognition process.

[0084] As for the specific image recognition operation, a variety of different implementation methods can also be designed, such as directly based on the image of the target interface, capturing the specific information of each control element in it, or adopting a step-by-step processing strategy, such as first locking several control objects from the interface, and then further identifying the specific type and location of the control. For this, please refer to Figure 2 The embodiment shown specifically includes the following steps:

[0085] Step S11, collecting target interface images;

[0086] Step S12, extracting features of the target interface image, and determining a number of processing objects in combination with a preset processing object identification strategy;

[0087] Step S13: extract the features of each processing object, and determine the type information and location information of each control in combination with a preset control identification strategy.

[0088] In actual operation, the front-end image data may be obtained (or the images of each region obtained by image segmentation may be obtained), and then the pre-built processing object recognition model may be used to recognize the processing object of the collected image. The processing object recognition model construction method may include:

[0089] ①. Obtain the front-end interface image and mark the parts to be identified;

[0090] ②. Use the training set to train the processing object recognition model to determine which parts of the image belong to the control elements; wherein the processing object recognition model can be but is not limited to being trained based on the InceptionResNetV2 neural network.

[0091] Then, a pre-built control recognition model is used to identify the control of the aforementioned processing object. The control recognition model construction method may include:

[0092] ①, obtain multiple images containing known control information and process the object, and mark them;

[0093] ②. Train a control recognition model based on the labeled training images, so as to determine the control information of each control on the target interface; wherein the control recognition model can be, but is not limited to, trained based on the YOLOv3 neural network.

[0094] The aforementioned image feature extraction and recognition strategy itself can draw on mature technologies in the field of image processing, which will not be elaborated here. What needs to be further explained in conjunction with the technical tasks of the present invention is that the control information may specifically refer to the functional type of the control, and of course may also include the location of the control. The type information of the control can be specifically referred to as follows: dialog box, pop-up box, radio button, multiple-choice box, input box, switch, slider, date and time selector, clickable submit button, password field, etc., and specific reference may be made to the front-end component library involved in this field. In short, through the above-mentioned image recognition processing, even if the interface changes and the control is adjusted, it is possible to quickly disassemble the information such as what type a certain control in the interface belongs to and where it is in the interface.

[0095] Step S2: parsing the code of each control according to the control information;

[0096] Step S3: inserting a control tag into the code of each control.

[0097] After the control information is disassembled in the above steps, the code of each control can be obtained from the interface code. This embodiment proposes to insert one or more control tags into the code of each control, for example, inserting a control tag that can represent the "identity" of the control or a control tag that represents the completion of code execution into all conditions of each front-end control code. As for the control tag, based on the above control information and preset rules, a control tag corresponding to each control can be generated. The rules here can be adjusted according to actual needs. For example, in a preferred embodiment of the present invention, "control element + system time + control position + element number" is used as the rule condition for generating control tags.

[0098] Furthermore, in order to facilitate the detection process to query and check the control records and each control, as well as to facilitate more accurate comparison and evaluation of the coverage, the control information of each control and the control mark corresponding to each control may be recorded. For example, the control information of each control and the corresponding control mark may be stored in a database. More preferably, a mark record table may be constructed in the database, which can be used to store the generated control marks.

[0099] Based on this, the present invention further considers that in actual operation, there may be more control elements involved or the interface may be changed, or controls may be added or adjusted. Therefore, before storing the control mark, the generated control mark may be compared with the stored control mark. If the two are repeated, the currently generated control mark is discarded, and a new control mark may be regenerated based on the aforementioned rule conditions and the current control information. The new control mark is then compared with the stored control mark. The final effect of this method is to store the generated and non-duplicate control mark, which ensures the one-to-one correspondence between the control mark and the control and the uniqueness of the mark to avoid affecting the implementation of subsequent steps.

[0100] Continuing from the above, step S4, real-time monitoring of user input requests and executing corresponding control functions;

[0101] Step S5: When the request processing is completed and the code runs to the control mark of the corresponding control, the preset running record mark of the control is updated.

[0102] During the detection process, the front end keeps a listening state so as to obtain the request instruction issued by the user. Of course, it can be understood that the request input by the user can be based on various forms such as voice, text, image, etc. Taking the voice request as an example, the text corresponding to the voice stream can be further obtained through the transcription engine, and the meaning of the user's instruction can be determined in combination with the semantic understanding technology, so that the corresponding control can be called to operate to process the front-end request. The present invention does not limit the front-end request form and the process of processing the request, but it should be emphasized that the operation record identification of the control can be updated only when two conditions are met. The two conditions refer to the completion of the front-end request and the operation to the aforementioned control mark, so as to accurately ensure that the operation state of the control has changed. The operation record identification described here can include an execution identification, and can also include an execution identification and an enable identification, which can be set based on different control types. For example, a control operation state record table (or code operation record table, etc.) is set for the target interface in the aforementioned database. Of course, the control operation state record table can also be pre-constructed into a prefabricated table that can cover more control types and more identification types. When the current target interface is detected, targeted changes can be made in the more comprehensive prefabricated table.

[0103] Specifically, updating the running record flag of the control may include: updating the execution flag of the control; or updating the execution flag and the enabling flag of the control. For example, when the above two conditions are met, the execution flag of the corresponding control may be changed from 0 to 1, and the enabling flag of the enabled control may be changed from 0 to 1, while the enabling flag of the unenabled control (such as the checkbox in the form filling interface, when one option is selected, the other option is grayed out) remains at 0.

[0104] Step S6: Obtain the final operation record identifier of each control on the target interface, and compare it with the set front-end control execution standard to generate a coverage detection result.

[0105] In the aforementioned steps, all requests for the target interface are executed and completed, and the final operation record identification of each control of the target interface is obtained. Specifically, in the coverage detection link mentioned in this step, the keywords, calling methods, descriptions and other information in the front-end request can be obtained, the type information and location information of each front-end control can be obtained, and the current execution identification and activation identification of each front-end control in the database can be obtained. Then, this information is structured and combined to obtain the control execution result of the current target interface. For example, the above-mentioned various information of the target interface is filled into a structured template to obtain a multi-scenario coverage report of the target interface. The front-end control execution standard used as the comparison standard can be pre-built based on various front-end frameworks, various front-end controls and a large number of front-end control call records to construct a control knowledge graph (or control knowledge base, etc.), and the control knowledge graph is used as the front-end control execution standard. The specific method of constructing the control knowledge graph can be referred to as follows:

[0106] ①, based on the definition, usage, calling rules, etc. of various front-end controls (such as the various known control components mentioned above);

[0107] ②. Obtain a large number of previous front-end control call records, extract the description text and standardize it into control description;

[0108] ③. Definitions, usage, calling rules, etc. based on various front-end frameworks (such as Angular, React, and Vue).

[0109] It can also be further explained that in other preferred schemes of the present invention, the detection targeting can be further considered. Based on the control information of the current target interface, the control knowledge data corresponding to each control of the target interface is extracted from the aforementioned more comprehensive control knowledge graph, and the front-end control execution standard is formed, that is, a specific comparison standard is generated to evaluate the coverage of the control execution results of the current interface.

[0110] Finally, it can be added that in at least one preferred embodiment of the present invention, the aforementioned operation record identifier can also include: a statistical identifier. Therefore, in the final detection coverage stage, the control of the target interface can be compared with the front-end control execution standard one by one, and after comparing each control, the statistical identifier of the control is updated to ensure the uniqueness of the detection object and avoid repeated comparison and confusion of the detection results.

[0111] In summary, the concept of the present invention is to realize the recognition and disassembly of the controls contained in the front-end interface image based on image recognition technology, so as to obtain the codes of each control, and then insert a pre-made mark into each control code, and when the front-end request is monitored and the request is executed, if the code runs to the mark, the pre-stored operation record identifier of the control is updated, and so on. When all requests are executed, the final operation record identifier of each control can be read out, and then compared with the set control execution standard, the detection of the coverage of the control elements of the front-end interface can be completed, that is, the code quality corresponding to the front-end interface can be evaluated. The present invention can quickly and conveniently identify and disassemble the front-end control information, and cooperate with the inserted control mark to realize the control in the process of executing the front-end request. The operation record identifier is changed sequentially, and then the code coverage can be compared based on the execution standard after the test. The present invention does not need to consume labor costs, nor does it need to add additional testing resources, and when the interactive interface or interactive logic changes, it can also complete the coverage detection conveniently and efficiently.

[0112] Corresponding to the above embodiments and preferred solutions, the present invention also provides an embodiment of a front-end control coverage detection device, such as Figure 3 As shown, it may specifically include the following components:

[0113] The control disassembly module 1 is used to obtain the control information of each control contained in the image of the target interface;

[0114] A code parsing module 2 is used to parse the code of each control according to the control information;

[0115] A tag inserting module 3, used for inserting a control tag into the code of each control;

[0116] Request processing module 4, used to monitor the requests input by users in real time and execute corresponding control functions;

[0117] The identification updating module 5 is used to update the preset running record identification of the control when the request processing is completed and the code runs to the control mark of the corresponding control;

[0118] The comparison and detection module 6 is used to obtain the final operation record identifier of each control in the target interface, and compare it with the set front-end control execution standard to generate a coverage detection result.

[0119] In at least one possible implementation, the control disassembly module includes:

[0120] An image acquisition unit, used for acquiring an image of a target interface;

[0121] A processing object identification unit, used to extract features of the target interface image and determine a number of processing objects in combination with a preset processing object identification strategy;

[0122] The control identification unit is used to extract the features of each processing object and determine the type information and location information of each control in combination with a preset control identification strategy.

[0123] In at least one possible implementation, the device further includes: a control mark generation module; the control mark generation module specifically includes:

[0124] A tag generating unit, used for generating a control tag corresponding to each control in a one-to-one manner according to the control information and preset rules;

[0125] The information storage unit is used to store the control information of each control and the corresponding control mark.

[0126] In at least one possible implementation, the device further includes: a tag uniqueness checking module; the tag uniqueness checking module specifically includes:

[0127] A tag comparison unit, used to compare the generated control tag with the stored control tag;

[0128] A tag regeneration unit is used to discard the currently generated control tag and regenerate a new control tag if the two are repeated;

[0129] The unique tag storage unit is used to continue comparing the new control tag with the stored control tag to store the generated and non-repetitive control tag.

[0130] In at least one possible implementation, the front-end control execution standard includes:

[0131] Preliminarily build a control knowledge graph based on various front-end frameworks, various front-end controls, and a large number of front-end control call records, and use the control knowledge graph as the front-end control execution standard;

[0132] or,

[0133] Based on the control information, control knowledge data corresponding to each control of the target interface is extracted from the control knowledge graph, and the front-end control execution standard is formed.

[0134] In at least one possible implementation manner, the operation record identifier includes: an execution identifier; or, the operation record identifier includes: an execution identifier and an activation identifier.

[0135] In at least one possible implementation manner, the identification updating module is specifically used to:

[0136] Updating the execution flag of the control;

[0137] or,

[0138] The execution flag and the enabling flag of the control are updated.

[0139] In at least one possible implementation, the operation record identifier further includes: a statistical identifier;

[0140] The comparison and detection module comprises:

[0141] A one-by-one comparison unit, used to compare each control of the target interface one by one with the front-end control according to the standard;

[0142] The identification updating unit is used to update the statistical identification of each control after comparing the control.

[0143] It should be understood that the above Figure 3 The division of the various components in the front-end control coverage detection device shown is only a division of logical functions. In actual implementation, they can be fully or partially integrated into one physical entity, or they can be physically separated. And these components can all be implemented in the form of software calling through processing elements; they can also be all implemented in the form of hardware; some components can also be implemented in the form of software calling through processing elements, and some components can be implemented in the form of hardware. For example, one of the above modules can be a separately established processing element, or it can be integrated in a chip of an electronic device. The implementation of other components is similar. In addition, all or part of these components can be integrated together, or they can be implemented independently. In the implementation process, each step of the above method or the above components can be completed by the hardware integrated logic circuit in the processor element or the instructions in the form of software.

[0144] For example, the above components may be one or more integrated circuits configured to implement the above methods, such as one or more application specific integrated circuits (ASIC), or one or more microprocessors (DSP), or one or more field programmable gate arrays (FPGA). For another example, these components may be integrated together to be implemented in the form of a system-on-a-chip (SOC).

[0145] Based on the above embodiments and their preferred embodiments, those skilled in the art can understand that, in actual operation, the present invention is applicable to a variety of implementations. The present invention uses the following carrier as a schematic illustration:

[0146] (1) A front-end control coverage detection device, which may include:

[0147] One or more processors, a memory, and one or more computer programs, wherein the one or more computer programs are stored in the memory, and the one or more computer programs include instructions, and when the instructions are executed by the device, the device performs the steps / functions of the aforementioned embodiment or equivalent implementation.

[0148] Figure 4 The schematic diagram of the structure of an embodiment of the front-end control coverage detection device of the present invention is shown in FIG. 1 , wherein the device can be an electronic device or a circuit device built into the above electronic device. The above electronic device can be a PC, a server, an intelligent terminal (a mobile phone, a tablet, etc.), and a dedicated front-end development and testing device. This embodiment does not limit the specific form of the front-end control coverage detection device.

[0149] Specific as Figure 4 As shown, the front-end control coverage detection device 900 includes a processor 910 and a memory 930. The processor 910 and the memory 930 can communicate with each other through an internal connection path to transmit control and / or data signals. The memory 930 is used to store computer programs, and the processor 910 is used to call and run the computer program from the memory 930. The above-mentioned processor 910 and the memory 930 can be combined into a processing device, or more commonly, they are independent components. The processor 910 is used to execute the program code stored in the memory 930 to implement the above-mentioned functions. In specific implementation, the memory 930 can also be integrated in the processor 910, or independent of the processor 910.

[0150] In addition, in order to make the function of the front-end control coverage detection device 900 more complete, the device 900 may also include one or more of an input unit 960, a display unit 970, an audio circuit 980, a camera 990, and a sensor 901, and the audio circuit may also include a speaker 982, a microphone 984, etc. Among them, the display unit 970 may include a display screen.

[0151] Furthermore, the front-end control coverage detection device 900 may also include a power supply 950 for providing electrical energy to various devices or circuits in the device 900 .

[0152] It should be understood that Figure 4 The front-end control coverage detection device 900 shown can implement each process of the method provided in the aforementioned embodiment. The operation and / or function of each component in the device 900 can be respectively used to implement the corresponding process in the aforementioned method embodiment. For details, please refer to the description of the embodiments of the method, device, etc. in the previous text. To avoid repetition, the detailed description is appropriately omitted here.

[0153] It should be understood that Figure 4 The processor 910 in the front-end control coverage detection device 900 shown can be a system on chip SOC, and the processor 910 can include a central processing unit (Central Processing Unit; hereinafter referred to as: CPU), and can further include other types of processors, such as: image processor (Graphics Processing Unit; hereinafter referred to as: GPU), etc., which will be specifically introduced below.

[0154] In summary, the various processors or processing units within the processor 910 can work together to implement the previous method flow, and the corresponding software programs of the various processors or processing units can be stored in the memory 930.

[0155] (2) A readable storage medium having a computer program or the above-mentioned device stored thereon, which, when executed, enables a computer to execute the steps / functions of the aforementioned embodiment or an equivalent implementation.

[0156] In several embodiments provided by the present invention, if any function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, some technical solutions of the present invention are essentially or part of the technical solution that contributes to the prior art or part of the technical solution can be embodied in the form of the software product described below.

[0157] (3) A computer program product (which may include the above-mentioned apparatus), which, when running on a terminal device, enables the terminal device to execute the front-end control coverage detection method of the above-mentioned embodiment or an equivalent implementation.

[0158] It can be known from the description of the above implementation methods that those skilled in the art can clearly understand that all or part of the steps in the above implementation method can be implemented by means of software plus the necessary general hardware platform. Based on this understanding, the above computer program product may include but is not limited to APP; continuing from the previous text, the above device / terminal may be a computer device (such as a mobile phone, PC terminal, cloud platform, server, server cluster or network communication device such as media gateway, etc.). In addition, the hardware structure of the computer device may also specifically include: at least one processor, at least one communication interface, at least one memory and at least one communication bus; the processor, communication interface and memory can all communicate with each other through the communication bus. Among them, the processor may be a central processing unit CPU, DSP, microcontroller or digital signal processor, and may also include GPU, embedded neural network processor (Neural-network Process Units; hereinafter referred to as: NPU) and image signal processor (Image Signal Processing; hereinafter referred to as: ISP). The processor may also include a specific integrated circuit ASIC, or one or more integrated circuits configured to implement the embodiments of the present invention, etc. In addition, the processor may have the function of operating one or more software programs, and the software programs may be stored in a storage medium such as a memory; and the aforementioned memory / storage medium may include: non-volatile memory (non-volatile memory), such as a non-removable disk, U disk, mobile hard disk, optical disk, etc., as well as read-only memory (Read-Only Memory; hereinafter referred to as: ROM), random access memory (Random Access Memory; hereinafter referred to as: RAM), etc.

[0159] In the embodiments of the present invention, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B may represent the existence of A alone, the existence of A and B at the same time, and the existence of B alone. A and B may be singular or plural. The character " / " generally indicates that the associated objects before and after are in an "or" relationship. "At least one of the following" and similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b and c may represent: a, b, c, a and b, a and c, b and c, or a and b and c, where a, b, c may be single or multiple.

[0160] Those skilled in the art will appreciate that the various modules, units, and method steps described in the embodiments disclosed in this specification can be implemented by a combination of electronic hardware, computer software, and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0161] In addition, each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other. In particular, for embodiments such as devices and equipment, since they are basically similar to method embodiments, the relevant parts can refer to the partial description of the method embodiments. The embodiments of the devices and equipment described above are merely schematic, wherein the modules, units, etc. described as separate components may or may not be physically separated, that is, they may be located in one place, or they may be distributed in multiple places, such as nodes of a system network. Specifically, some or all of the modules and units may be selected according to actual needs to achieve the purpose of the above-mentioned embodiment scheme. Those skilled in the art can understand and implement it without paying creative labor.

[0162] The above describes in detail the structure, features and effects of the present invention based on the embodiments shown in the drawings, but the above is only a preferred embodiment of the present invention. It should be noted that the technical features involved in the above embodiments and their preferred methods can be reasonably combined and matched into a variety of equivalent schemes by those skilled in the art without departing from or changing the design ideas and technical effects of the present invention; therefore, the present invention is not limited to the scope of implementation shown in the drawings, and all changes made in accordance with the concept of the present invention, or modifications to equivalent embodiments with equivalent changes, which still do not exceed the spirit covered by the specification and drawings, should be within the protection scope of the present invention.

Claims

1. A front-end control coverage detection method, characterized in that: include: Obtain control information of each control contained in the image of the target interface; According to the control information, parse out the code of each control; Inserting a control tag into the code of each control; After starting the detection, monitor the user input request in real time and execute the corresponding control function; When the request is processed and the code runs to the control mark of the corresponding control, the preset running record mark of the control is updated; Obtain the final operation record identifier of each control on the target interface, and compare it with the set front-end control execution standard to generate coverage test results; The front-end control execution standard includes: a control knowledge graph constructed in advance based on various front-end frameworks, various front-end controls, and a large number of front-end control call records; Alternatively, based on the control information, control knowledge data corresponding to each control of the target interface is extracted from the control knowledge graph.

2. The front-end control coverage detection method according to claim 1 is characterized in that: Based on image recognition technology, the control information of each control contained in the image of the target interface is obtained, including: Collect target interface images; Extracting features of the target interface image, and determining a number of processing objects in combination with a preset processing object identification strategy; The features of each processing object are extracted, and the type information and location information of each control are determined in combination with a preset control identification strategy.

3. The front-end control coverage detection method according to claim 1 is characterized in that: The method further comprises: Generate a control mark corresponding to each control in a one-to-one manner according to the control information and preset rules; The control information of each control and the corresponding control mark are stored.

4. The front-end control coverage detection method according to claim 3 is characterized in that: The method further comprises: Compare the generated control tags with the stored control tags; If the two are repeated, the currently generated control tag is discarded and a new control tag is regenerated; Continue to compare the new control mark with the stored control mark, and store the generated and non-duplicate control mark.

5. The front-end control coverage detection method according to any one of claims 1 to 4, characterized in that: The operation record identification includes: an execution identification; or, The operation record identifier includes: an execution identifier and an activation identifier.

6. The front-end control coverage detection method according to claim 5 is characterized in that: The updating of the running record identifier of the control includes: Updating the execution flag of the control; or, The execution flag and the enabling flag of the control are updated.

7. The front-end control coverage detection method according to claim 5, characterized in that: The operation record identification also includes: a statistical identification; The comparison with the set front-end control execution standard includes: comparing each control of the target interface with the front-end control execution standard one by one, and after comparing each control, updating the statistical identifier of the control.

8. A front-end control coverage detection device, characterized in that: include: A control disassembly module is used to obtain control information of each control contained in the image of the target interface; A code parsing module, used to parse the code of each control according to the control information; A tag insertion module, used for inserting a control tag into the code of each control; The request processing module is used to monitor the user input request in real time and execute the corresponding control function after the detection starts; An identification updating module, used for updating a preset running record identification of a control when the request processing is completed and the code runs to the control tag of the corresponding control; The comparison and detection module is used to obtain the final operation record identification of each control in the target interface, and compare it with the set front-end control execution standard to generate coverage detection results; The front-end control execution standard includes: a control knowledge graph constructed in advance based on various front-end frameworks, various front-end controls, and a large number of front-end control call records; Alternatively, based on the control information, control knowledge data corresponding to each control of the target interface is extracted from the control knowledge graph.

9. A front-end control coverage detection device, characterized in that: include: One or more processors, a memory and one or more computer programs, wherein the one or more computer programs are stored in the memory, and the one or more computer programs include instructions, and when the instructions are executed by the device, the device executes the front-end control coverage detection method as described in any one of claims 1 to 8.

Images