U shield display information acquisition method, device, equipment, medium and program product applied to U shield testing

Abstract

The present disclosure provides a method for obtaining U shield display information applied to U shield testing, which can be applied to the fields of big data technology and artificial intelligence technology. The method for obtaining U shield display information applied to U shield testing includes: determining the click coordinate information of the U shield according to the U shield to be tested; performing a click action corresponding to the click coordinate information on the U shield according to the received image acquisition operation instruction, and detecting the display interface image of the U shield corresponding to the click action; and parsing the display interface image to obtain the U shield display information. The present disclosure also provides a device, equipment, medium and program product for obtaining U shield display information applied to U shield testing.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and more specifically to a method, device, equipment, medium and program product for obtaining U-shield display information applied to U-shield testing. Background Art

[0002] In the business application systems provided by banks and other financial enterprises, for transactions involving account changes, in order to ensure the security of transactions and prevent impersonation, financial enterprises (such as banks) in the prior art usually provide U-shield as proof of transaction user identity authentication. When conducting transactions, users need to insert U-shield on terminal devices such as computers to conduct related transactions in the system.

[0003] With the acceleration of the pace of version development, automated verification of business application systems has become the preferred solution in the testing process. Since the transactions involving the above-mentioned need to insert a USB shield into the local computer as a security medium are generally high-risk key transactions, and are also the key verification content of each version test, in the process of traditional automated testing, there will be restrictions on hardware security media operations (plugging and unplugging, physical button clicking, etc.), and more user manual operation links need to be added to complete the normal operation test process of a transaction. In other words, the automated testing of such high-risk business transactions has always required manual intervention for connection, and the automation cost is high, which greatly limits the efficient automated testing of such high-risk business transactions. Summary of the invention

[0004] In view of at least one of the technical problems existing in the process of automated testing for high-risk business transactions in the prior art, the present disclosure provides a method, device, equipment, medium and program product for obtaining U-shield display information for U-shield testing, which can achieve a more efficient level of automated testing.

[0005] According to a first aspect of the present disclosure, a method for obtaining U-shield display information applied to U-shield testing is provided, which includes: determining the click coordinate information of the U-shield according to the U-shield to be tested; performing a click action corresponding to the click coordinate information on the U-shield, and detecting a display interface image of the U-shield corresponding to the click action; and parsing the display interface image to obtain the U-shield display information.

[0006] According to an embodiment of the present disclosure, in determining the click coordinate information of the U-shield according to the U-shield to be tested, it includes: determining the physical slot number of the U-shield device placement cabinet where the U-shield is located according to the physical U-shield number of the U-shield to be tested; and determining the click coordinate information of the U-shield according to the physical slot number.

[0007] According to an embodiment of the present disclosure, determining the click coordinate information of the U-shield according to the physical slot number includes: detecting the U-shield slot image of the U-shield corresponding to the physical slot number; determining the model of the U-shield according to the U-shield slot image; and determining the click coordinate information of the U-shield according to the first click offset determined by the U-shield model and the U-shield coordinate information determined by the physical slot number.

[0008] According to an embodiment of the present disclosure, before executing a click action corresponding to the click coordinate information on the U-shield and detecting the display interface image of the U-shield corresponding to the click action, it also includes: determining a first movement trajectory relative to the U-shield device placement cabinet for performing a first click operation based on the first click coordinate information; determining a first click position corresponding to the first click coordinate information based on the first movement trajectory; wherein the click coordinate information includes the first click coordinate information, and the click action includes the first click action.

[0009] According to an embodiment of the present disclosure, a click action corresponding to click coordinate information is executed on the U-shield, and a display interface image of the U-shield corresponding to the click action is detected, including: before executing a first click action corresponding to first click coordinate information on the U-shield, detecting a first display interface image of the U-shield according to an image acquisition operation instruction; in response to the first display interface image, alternately executing a first click action and an image detection action on the U-shield to obtain at least two second display interface images of the U-shield; wherein the display interface image includes a first display interface image and at least two second display interface images.

[0010] According to an embodiment of the present disclosure, in executing a click action corresponding to the click coordinate information on the U-shield and detecting the display interface image of the U-shield corresponding to the click action, it also includes: determining the second click coordinate information used to execute the second click action based on the first click coordinate information and the second click offset corresponding to the model of the U-shield; determining the second movement trajectory relative to the U-shield device placement cabinet for executing the second click operation based on the second coordinate information; determining the second click position corresponding to the second click action based on the second movement trajectory; wherein the click coordinate information also includes the second click coordinate information, and the click action also includes the second click action.

[0011] According to an embodiment of the present disclosure, after executing a click action corresponding to the click coordinate information on the U shield and detecting a display interface image of the U shield corresponding to the click action, it also includes: executing a second click action on a second click position of the U shield.

[0012] According to an embodiment of the present disclosure, in parsing the display interface image to obtain U-shield display information, it includes: in response to the execution of the second click action, sorting and identifying the display interface image to obtain the text content in the display interface image; performing splicing analysis on the text content to obtain the U-shield display information.

[0013] According to a second aspect of the present disclosure, a device for acquiring U shield display information applied to U shield testing is provided, which includes a coordinate determination module, a click detection module and an information acquisition module. The coordinate determination module is used to determine the click coordinate information of the U shield according to the U shield to be tested; the click detection module is used to perform a click action corresponding to the click coordinate information on the U shield, and detect the display interface image of the U shield corresponding to the click action; and the information acquisition module is used to parse and process the display interface image to acquire the U shield display information.

[0014] The third aspect of the present disclosure provides an electronic device, comprising: one or more processors; a memory for storing one or more programs, wherein, when the one or more programs are executed by the one or more processors, the one or more processors execute the above-mentioned U-shield display information acquisition method applied to U-shield testing.

[0015] The fourth aspect of the present disclosure also provides a computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, causes the processor to execute the above-mentioned method for obtaining U-shield display information applied to U-shield testing.

[0016] The fifth aspect of the present disclosure also provides a computer program product, including a computer program, which, when executed by a processor, implements the above-mentioned method for obtaining U-shield display information applied to U-shield testing. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The above contents and other objects, features and advantages of the present disclosure will become more apparent through the following description of the embodiments of the present disclosure with reference to the accompanying drawings, in which:

[0018] Figure 1 The following schematically shows an application scenario diagram of a method, device, equipment, medium and program product for obtaining U-shield display information applied to U-shield testing according to an embodiment of the present disclosure;

[0019] Figure 2 The flowchart of the method for obtaining U-shield display information applied to U-shield testing according to an embodiment of the present disclosure is schematically shown;

[0020] Figure 3A A system composition diagram of a method for obtaining U-shield display information applied to U-shield testing according to an embodiment of the present disclosure is schematically shown;

[0021] Figure 3B A schematic diagram shows a system of a method for obtaining U-shield display information applied to a U-shield test according to an embodiment of the present disclosure and a composition diagram of a U-shield device placement cabinet;

[0022] Figure 4The flowchart of the method for obtaining U-shield display information applied to U-shield testing according to an embodiment of the present disclosure is schematically shown;

[0023] Figure 5A-5G Schematically showing a U-shield interface diagram to be tested corresponding to multiple display interface images during the image detection process according to an embodiment of the present disclosure;

[0024] Figure 6 A structural block diagram of a U-shield display information acquisition device applied to a U-shield test according to an embodiment of the present disclosure is schematically shown; and

[0025] Figure 7 A block diagram of an electronic device suitable for implementing a method for acquiring U-shield display information applied to U-shield testing according to an embodiment of the present disclosure is schematically shown. DETAILED DESCRIPTION

[0026] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present disclosure. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiments of the present disclosure. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of the concepts of the present disclosure.

[0027] The terms used herein are only for describing specific embodiments and are not intended to limit the present disclosure. The terms "comprise", "include", etc. used herein indicate the existence of the features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.

[0028] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0029] When using expressions such as "at least one of A, B, and C, etc.", they should generally be interpreted according to the meaning of the expression commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0030] In the existing technology, in the automated testing scenario of U-Shield, there are two main situations involving high-risk business transactions: the first is the situation of clicking the physical buttons on the U-Shield device; the second is the situation of switching the U-Shield currently in use. Among them, for the first situation, the usual solution in the industry is to use a physical peripheral solution, that is, in the entire business transaction process, the physical button click operation of the U-Shield is realized by clicking the physical button on the U-Shield through the external device; for the second situation, in the current actual work, it can only be realized by the tester manually plugging and unplugging to switch the U-Shield, otherwise it can only be realized by further developing related U-Shield routing connection devices or modes.

[0031] Among them, in the case of clicking the above-mentioned U-shield button, there are usually two clicking schemes: one is an uncontrolled blind clicking scheme, and the other is a controlled targeted clicking scheme.

[0032] In the uncontrolled blind click mode, blind clicks are usually performed by continuously clicking the physical buttons on the U-Shield using physical peripherals. Because the blind clicks are not targeted, even if there are no or only a few U-Shields that need to be clicked in a certain period of time, all U-Shield devices are in a clicked state, resulting in a huge amount of invalid click operations, resulting in waste of resources such as electricity and unnecessary equipment loss, especially more U-Shield buttons will be damaged due to multiple clicks, wasting test resources.

[0033] In the controlled targeted click mode, usually one clicker is used for each U-shield device. As the scale of the test increases, more and more U-shield devices need to be prepared for testing. When the number of U-shield devices reaches a certain scale, the mode of equipping each U-shield with a clicker creates certain difficulties in the deployment and management of the clicker. At the same time, because not all U-shiels need to be clicked at the same time, more resources of the U-shield clicker devices are wasted.

[0034] Accordingly, in the current mode existing in the prior art, when it comes to automated test transactions involving U-Shield testing, regardless of whether a physical peripheral is used to continuously click on the physical button on the U-Shield, or a controlled click or a cloud-based physical deployment of the U-Shield, the existing process only simulates the user's click operation on the U-Shield device. During the actual transaction process, the information displayed by the U-Shield device cannot be accurately collected, and therefore it is impossible to verify whether the information displayed by the U-Shield is correct, whether the prompts to the user meet expectations, and whether the user operation information is provided normally. This results in a verification blind spot in the test process, and it is impossible to effectively implement accurate and targeted testing, and it is impossible to complete the verification of a complete business closed-loop test scenario.

[0035] In view of at least one of the technical problems existing in the process of automated testing for high-risk business transactions in the prior art, the present disclosure provides a method, device, equipment, medium and program product for obtaining U-shield display information for U-shield testing, which can achieve a more efficient level of automated testing.

[0036] It should be noted that the above-mentioned U-shield display information acquisition method and device for U-shield testing disclosed in the present invention can be used in the fields of big data technology and artificial intelligence technology, and can also be used in the financial field and any field outside the financial field. The application field of the U-shield display information acquisition method and device for U-shield testing disclosed in the present invention is not limited.

[0037] In the technical solution disclosed in this disclosure, the collection, storage, use, processing, transmission, provision, disclosure and application of user personal information and other data involved are in compliance with the relevant laws and regulations, necessary confidentiality measures are taken, and they do not violate public order and good morals. Before obtaining or collecting user personal information, the user's authorization or consent is obtained.

[0038] An embodiment of the present disclosure provides a method for obtaining U-shield display information applied to U-shield testing, which includes: determining the click coordinate information of the U-shield according to the U-shield to be tested; performing a click action corresponding to the click coordinate information on the U-shield, and detecting a display interface image of the U-shield corresponding to the click action; and parsing the display interface image to obtain the U-shield display information.

[0039] Figure 1 The application scenario diagram schematically shows a method, device, equipment, medium and program product for obtaining U-shield display information applied to U-shield testing according to an embodiment of the present disclosure.

[0040] like Figure 1 As shown, the application scenario 100 according to this embodiment may include terminal devices 101-103, a network 104 and a server 105. The network 104 is used to provide a medium for communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0041] Users can use terminal devices 101, 102, 103 to interact with server 105 through network 104 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 101, 102, 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc. (only examples).

[0042] The terminal devices 101 , 102 , and 103 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, and desktop computers.

[0043] The server 105 may be a server that provides various services, such as a background management server (only as an example) that provides support for websites browsed by users using the terminal devices 101, 102, and 103. The background management server may analyze and process the received data such as user requests, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal device.

[0044] It should be noted that the U shield display information acquisition method applied to the U shield test provided in the embodiment of the present disclosure can generally be executed by the server 105. Accordingly, the U shield display information acquisition device applied to the U shield test provided in the embodiment of the present disclosure can generally be set in the server 105. The U shield display information acquisition method applied to the U shield test provided in the embodiment of the present disclosure can also be executed by a server or server cluster that is different from the server 105 and can communicate with the terminal device 101, 102, 103 and / or the server 105. Correspondingly, the U shield display information acquisition device applied to the U shield test provided in the embodiment of the present disclosure can also be set in a server or server cluster that is different from the server 105 and can communicate with the terminal device 101, 102, 103 and / or the server 105.

[0045] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0046] The following will be based on Figure 1 The scene described by Figure 2~Figure 6 The U shield display information acquisition method, device, and equipment applied to the U shield test in the disclosed embodiment are described in detail. Among them, the U shield in the disclosed embodiment has a display interface for displaying business transaction information or business process information, and the physical structure of the U shield device has a click key or button for click operation, etc., and these keys are manually or mechanically clicked to complete the business transaction or business process test in the operation and maintenance of the U shield device.

[0047] Figure 2 The flowchart of the method for obtaining U-shield display information applied to U-shield testing according to an embodiment of the present disclosure is schematically shown.

[0048] like Figure 2 As shown, the method for obtaining U-shield display information applied to U-shield testing of this embodiment includes operations S201 to S203.

[0049] In operation S201, the click coordinate information of the USB shield is determined according to the USB shield to be tested;

[0050] In operation S202, a click action corresponding to the click coordinate information is performed on the U shield, and a display interface image of the U shield corresponding to the click action is detected;

[0051] In operation S203, the display interface image is parsed to obtain U shield display information.

[0052] In the embodiments of the present disclosure, the U-shield as the object to be tested can generally be arranged in an array to form a U-shield array set in a specific area. In this specific area, due to the different positions of multiple U-shields to be tested in the U-shield array, the position information of each U-shield is unique relative to the specific area (which can be understood as a reference system). Furthermore, for each U-shield, the click position (such as a key or button, etc.) it has for being clicked must also be unique. Therefore, the click coordinate information of each U-shield is the position information of the click position on the U-shield relative to the specific area where the U-shield is located. It can be seen that according to the position of the specific area where the U-shield to be tested is located, the click position of the U-shield can be confirmed.

[0053] Therefore, the device location information of the U-shield and the click location information of the U-shield can be uniquely determined, so that the locations of multiple U-shields in a specific area can be managed in the cloud, thereby achieving unified click management of multiple U-shields through only one clicker.

[0054] Corresponding to the above-mentioned click coordinate information, a click action can be performed on the click position corresponding to the click coordinate information according to the corresponding U shield click operation instruction. Among them, the U shield click operation instruction is a control instruction for performing a click operation on the specified U shield to be tested, and according to the U shield click operation instruction, it can be used to control a click tool such as a clicker to click on the U shield to be tested. In the process of executing the click action of the U shield, the display interface image of the display area of ​​the U shield during the click process can also be further detected according to the image acquisition operation instruction. Among them, the image acquisition operation instruction is a control instruction for performing an image acquisition process on the specified U shield to be tested, and the image acquisition can be a process of generating image data such as pictures or videos by means such as taking pictures and recording videos. In the clicking process, the display interface of the U shield to be tested is obtained by taking pictures, recording videos, etc., so as to realize the image detection of the corresponding click action, wherein the image data such as pictures or videos obtained can be used as the storage data of the above-mentioned display interface image, and the display interface image is the picture or video itself presented in the display interface of the U shield to be tested. It can be seen that the automatic movement process of the clicker during the click action and image detection process can be improved according to the click coordinate information to ensure a closed-loop test of the entire test process.

[0055] Furthermore, after parsing the display interface image, the display information presented in the display interface of the USB shield to be tested during the click operation can be obtained, that is, the USB shield display information. Therefore, the USB shield display information can include display content such as patterns, text, numbers, and punctuation. These display contents can be used to feedback the business content presented by the USB shield to be tested after each click during the test process.

[0056] Therefore, the above-mentioned method for obtaining U-shield information in the disclosed embodiment solves the problem in the prior art that when a targeted automated testing U-shield click solution mode is adopted, if a large number of U-shield devices need to be automated tested, and each U-shield test device needs to prepare a clicker device, the resulting waste of clicker resources and large maintenance workload are large. By defining the coordinates of the location of the U-shield, the U-shield devices can be centrally and uniformly managed to achieve automated testing of the cloud service of the U-shield, thereby realizing the cloud mode of the U-shield by reusing only the same set of clicker devices for the U-shield devices in a specific area of ​​the same U-shield device, and being able to provide a cloud mode automated testing U-shield click service. Therefore, the number of U-shield device clickers and resource costs required are greatly reduced, and the workload of U-shield device maintenance is reduced, while also meeting the flexible deployment requirements of one U-shield device corresponding to one U-shield clicker.

[0057] In addition, through the U-Shield device information collection and processing function, the information displayed in the U-Shield device is collected during the device click process, which can avoid the test blind spots due to the lack of judgment on the U-Shield display information during the test verification process. In the case of abnormal functions of some application systems or abnormal U-Shield drivers, the risk of inconsistency between the prompt information of the U-Shield device and the corresponding system operation after the actual user operation is avoided. Through the U-Shield interactive display information collection, the display information of the U-Shield device during the U-Shield test is collected, so as to timely discover the inconsistency between the display information of the U-Shield device and the actual prompt information of the system, avoid misleading the end user, and realize a true closed loop of the test process.

[0058] Figure 3A A system composition diagram of a method for obtaining U-shield display information applied to U-shield testing according to an embodiment of the present disclosure is schematically shown; Figure 3B A schematic diagram shows a system of a method for obtaining U-shield display information applied to a U-shield test according to an embodiment of the present disclosure and a composition diagram of a U-shield device placement cabinet; Figure 4 The flowchart of the method for obtaining U-shield display information applied to U-shield testing according to an embodiment of the present disclosure is schematically shown; Figure 5A-5G The following schematically shows a U-shield interface diagram to be tested corresponding to multiple display interface images during the image detection process according to an embodiment of the present disclosure.

[0059] like Figure 3A-Figure 4 As shown, the system composition of the U shield display information acquisition method using the above-mentioned embodiment of the present disclosure mainly includes an execution machine external device management module 310, a U shield device routing connection module 320, a one-to-one U shield click management module 330, a U shield device display information acquisition module 340, a U shield display information analysis and comparison module 350, a U shield display information display module 360 ​​and a cloud service U shield click management module 380. Among them, the U shield device routing connection module 320 can be connected to multiple physical U shields 1-n to be tested, the one-to-one U shield click management module 330 can be connected to multiple clickers 1-n for testing clicks, and the system composition also includes an execution machine pool 370, and the execution machine pool 370 can include multiple execution machines 1-n. Among them, the physical U shield 1-n, the clicker 1-n and the execution machine 1-n can be set one-to-one through the one-to-one U shield click management module 330 to realize the traditional one-to-one U shield click solution.

[0060] The business process needs to be the business transaction process content related to the business transaction content to be tested, and different business transactions correspond to different business process needs. The execution machine external device management module 310 can send U-shield usage communication message information to the U-shield device routing connection module 320 according to different business processes, and is also responsible for sending U-shield click control communication message information to the one-to-one U-shield click management module 330. Among them, the U-shield usage communication message information includes basic attribute content such as U-shield number, operation instructions, sending time, execution machine name, and sending IP. Among them, the operation instructions include creating a new USB interface connection, releasing a USB interface connection, initializing U-shield routing configuration, resetting physical U-shield connection, and updating physical U-shield connection. In addition, the U-shield click control communication message information contains information such as the execution machine name, U-shield number, U-shield deployment category, sending time, sending IP, number of clicks, and click interval duration. Among them, the U-shield deployment categories include cloud service U-shield and non-cloud service U-shield. The cloud service U-shield can be a U-shield equipped with only one clicker in a specific area in the embodiment of the present disclosure to perform clicks according to the U-shield coordinate information, thereby realizing cloud service management. The non-cloud service U-shield is a traditional U-shield in which one clicker locates one U-shield to perform clicks.

[0061] like Figure 2-Figure 5G As shown, according to an embodiment of the present disclosure, in operation S201, determining the click coordinate information of the U shield according to the U shield to be tested includes:

[0062] According to the physical U-shield number of the U-shield to be tested, determine the physical slot number of the U-shield equipment placement cabinet where the U-shield is located;

[0063] Determine the click coordinate information of the USB shield based on the physical grid number.

[0064] The U shield device routing connection module 320 is used to establish a physical connection path between the automated execution machine 1-n and the U shield device 1-n to be tested, and to transmit instructions. It is mainly composed of a U shield information management submodule 321, a U shield device connection submodule 322, and a U shield data switching routing submodule 323.

[0065] Among them, the U shield information management submodule 321 mainly manages the U shield number, the comparison information of the U shield device connected to the USB interface, and the business purpose of the U shield and provides maintenance functions. The U shield device connection submodule 322 is used to manage and maintain the mapping relationship between the USB interface and the physical U shield. The U shield data switching routing submodule 323 is used to receive the U shield usage communication message information sent from the execution machine external device management module, establish or interrupt the connection between the execution machine and the USB physical interface, and synchronize the connection status, start time, end time, current occupied execution machine name, current occupied execution machine IP and other fields to update the U shield routing configuration table.

[0066] Therefore, the U-shield device routing connection module 320 can also establish a logical connection relationship between the U-shield to be tested, the physical interface and the basic information of the execution machine in the U-shield communication message. The logical connection relationship can be used to determine the U-shield number to be tested, the location of the specific test area and the corresponding click coordinate information.

[0067] like Figure 3B As shown, there are multiple U shield placement cabinets 1-n in the application system. The U shield placement cabinet is a hardware device mainly used to place and fix the U shield device. U shield device placement cabinets of different sizes can place physical U shield devices of different numbers and shapes. For example, U shield placement cabinet 1 has 100 physical U shield devices to be tested, from 1-1 to 10-10. Similarly, the nth U shield placement cabinet n also has 100 physical U shield devices to be tested, from 1-1 to 10-10. At this time, the U shield placement cabinet is used as a specific area for storing these U shields to be tested. As long as the 100 U shields to be tested in each U shield placement cabinet are arranged, stored and fixed, the U shield placement cabinet can be used as a reference coordinate system (such as establishing a horizontal and vertical coordinate axis to achieve the coordinate definition of a specified position) to uniquely define the coordinates of the U shield, that is, the U shield coordinate information, which can be specifically defined according to the location of the physical interface on the U shield placement cabinet to which it is connected. Correspondingly, the coordinates of the button on the U-shield used for testing relative to the U-shield placement cabinet where it is located are the U-shield click coordinate information.

[0068] Therefore, through the U-Shield deployment cabinet, cloud-based centralized management of multiple U-Shields can be achieved, so that each U-Shield deployment cabinet only needs to correspond to one clicker, that is, each clicker can click and test all U-Shields in the corresponding U-Shield deployment cabinet one by one, thereby solving the problem of waste of clicker resources caused by one clicker corresponding to a single U-Shield test in traditional technology through cloud-based U-Shield test service management.

[0069] Therefore, for the selected U-shield to be tested, according to the logical connection relationship determined by the U-shield device routing connection module 320, the physical U-shield number of the U-shield to be tested can be used to determine the physical interface number of the U-shield placement cabinet into which it is inserted, and the physical grid number of the U-shield placement cabinet where the U-shield is located can be determined according to the physical interface number. If the U-shield storage access area in the U-shield placement cabinet is grid-divided into multiple sub-areas, each sub-area corresponds to a U-shield storage position, then the storage position of a U-shield can be uniquely defined by numbering the sub-area, and the sub-area can be understood as a physical grid. Among them, the physical U-shield number of the U-shield can be used as a unique identification code of a U-shield to distinguish the U-shield from other U-shields, similar to a unique identity code. The physical grid number is related to the physical interface into which the U-shield is inserted, and each physical interface has a unique coordinate in the corresponding U-shield placement cabinet, and each coordinate has a number to distinguish other physical coordinates, so that the physical grid number can uniquely determine the U-shield coordinate information of the U-shield inserted into the physical interface.

[0070] At the same time, according to the model of the U-Shield, the basic information of the U-Shield, such as the type, version or brand of the U-Shield, can be determined, so that the click coordinate information of the U-Shield can be uniquely determined based on the U-Shield coordinate information determined by the physical slot number. For details, please refer to the confirmation process of the click coordinate information described below.

[0071] In addition, if Figure 5A-5G As shown, each U-shield to be tested has a corresponding display panel (such as an LCD panel, an LED panel, and a liquid crystal panel), which serves as the display interface of the U-shield to be tested. In addition, corresponding click buttons can be set around the display panel of the U-shield to be tested, such as an "OK" key for confirming the process, a "C" key for canceling the process or returning to the process, and a page-turning key for turning up or down pages, such as the "v" key can be a page-turning key. Among them, when the U-shield is inserted into the physical interface of the U-shield placement cabinet, the relative position coordinates of different buttons on the U-shield are also different. Among them, since the position of the U-shield button is fixed, the position relationship between each button of the same type of U-shield can be preset, and the position relationship between each button relative to the U-shield coordinate information of the U-shield in the corresponding physical position can also be preset, which can be specifically preset and fixed in the form of coordinates.

[0072] like Figure 2-Figure 5G As shown, according to an embodiment of the present disclosure, in determining the click coordinate information of the U shield according to the physical slot number, it includes: detecting the U shield slot image of the U shield corresponding to the physical slot number; determining the model of the U shield according to the U shield slot image; and determining the click coordinate information of the U shield by the first click offset determined by the U shield model and the U shield coordinate information determined by the physical slot number.

[0073] When the USB shield to be tested is inserted into the physical interface of the USB shield placement cabinet, an image detection can be performed on the physical slot where the physical interface of the USB shield is inserted in response to the detection instruction of the USB shield insertion, and the image of the USB shield on the physical slot is obtained as the USB shield slot image, wherein the USB shield slot image corresponds to the USB shield physical number, physical slot number and USB shield coordinate information of the physical slot. Specifically, the image detection can involve the detection operation of taking a photo or recording a video of the image in the state where the USB shield is inserted in the physical slot.

[0074] By performing recognition analysis on the U shield slot image, the model of the U shield in the physical slot corresponding to the U shield slot image can be determined according to the preset U shield model image data. The U shield model image data can be the corresponding front appearance image data of all the adopted U shield models, and the front appearance image refers to the image of the appearance of the U shield with operation buttons and display interface.

[0075] Therefore, the movement offset between each button on the U shield and the set center position of the U shield can be determined according to the model of the U shield, and the movement offset can be the first click offset of the corresponding click button. The first click offset can be reflected as the fixed offset distance reflected on the appearance of the U shield between the first click position corresponding to the first click action and the set center position of a certain model of the U shield and the offset angle of the center line of the set center position of the U shield.

[0076] Therefore, by combining the U-shield coordinate information of the physical slot number corresponding to the U-shield and the offset between the set center position of the U-shield, the click coordinate information of the button on the U-shield relative to the U-shield placement cabinet can be obtained.

[0077] Among them, the corresponding click buttons of U-shields of different types, versions and brands have different click coordinate positions relative to the U-shield placement cabinet due to the actual size and appearance design of the U-shield, but the displacement relationship between the U-shield coordinate information of the U-shield, the U-shield setting center position and its click coordinate information is generally fixed.

[0078] Therefore, the click coordinate information of the click position of the U shield can be quickly and accurately determined by image recognition according to the fact that the button of the click position of the U shield is fixed relative to the appearance of the U shield shell.

[0079] According to an embodiment of the present disclosure, before operation S201 performs a click action corresponding to the click coordinate information on the U shield and detects a display interface image of the U shield corresponding to the click action, it also includes:

[0080] Determine, according to the first click coordinate information, a first movement trajectory relative to the U-shield device placement cabinet for performing the first click operation;

[0081] Determine, according to the first movement trajectory, a first click position corresponding to the first click coordinate information;

[0082] The click coordinate information includes first click coordinate information, and the click action includes a first click action.

[0083] After determining the U-shield to be tested, according to the needs of the business process test, if the U-shield is clicked, it is necessary to determine the movement trajectory of the clicker based on the position coordinates of the current clicker in the U-shield placement cabinet where the U-shield is located and the click coordinates of the button at the click position of the U-shield to be tested. Among them, the first click coordinate information can be the click coordinate information of one of the buttons that needs to be clicked on the U-shield, such as the page turning key (such as the up page key or the down page key, generally the down page key). The click operation corresponding to the click of the first click coordinate information is the first click action, and the movement trajectory of the clicker executing the click process on the U-shield placement cabinet determined according to the first click coordinate information and the current position of the clicker is the first movement trajectory. Among them, according to the first movement trajectory, the first click coordinate information can be directly determined as the corresponding first click position.

[0084] Therefore, the first click action can correspond to a click on the page turning key, the first click coordinate information can correspond to the coordinate information of the page turning key relative to the U shield placement cabinet, and the first movement trajectory can correspond to the movement path of the clicker for the page turning key relative to the U shield placement cabinet. In this way, before executing the page turning key click on each U shield, the click position and click path of the clicker on the page turning key can be determined, so that the clicker can complete the click on the page turning key fully automatically, and realize the test click on the page turning function, thereby completely replacing the blind click operation or the traditional technology that requires manual execution of the click operation, greatly improving the click accuracy and test efficiency.

[0085] like Figure 3A and Figure 3B As shown, the cloud service U shield click management module 380 is mainly used to receive the click operation instruction of the U shield whose deployment mode is cloud centralized deployment submitted by the execution machine 1-n, and perform the click operation on the specified U shield according to the click operation instruction. It is mainly composed of the U shield device deployment configuration management submodule 381, the U shield device click addressing submodule 382, ​​the U shield device click operation control submodule 383 and the cloud service U shield click instruction processing submodule 384.

[0086] Among them, the cloud service U shield click instruction processing submodule 384 is mainly used to receive the cloud service U shield click control communication message sent by the execution machine external device management module 310, and parse and process the U shield click control communication message instruction. The U shield device placement configuration management submodule 381 is mainly used to record the relevant configuration information of each U shield, including the physical U shield number, the physical grid number where it is located, and the horizontal and vertical coordinate position of the click inside each physical grid. The U shield device click addressing submodule 382 calculates the sliding distance of the clicker according to the click position of the physical target U shield in the click instruction, and slides the clicker to the specified position. The U shield device click addressing submodule 382 stores the absolute position reference coordinate value of each physical grid on the U shield device placement cabinet, and obtains the size and key distribution of the U shield according to the information such as the U shield brand placed in each grid, thereby calculating the coordinates of the corresponding button on the U shield of the brand and the sliding distance of each grid to the reference coordinate, and reaches the specified U shield button position. In addition, the U-shield device click operation control submodule 383 is mainly used to operate the clicker to perform click operations and record click logs in the click operation log recording submodule 334.

[0087] It can be seen that through the cloud service U-shield click management module 380, when there are a large number of automated testing U-shield devices, cloud service management of the U-shield in the U-shield deployment cabinet can be realized. It only needs a single physical slot in the U-shield deployment cabinet to store the U-shield, and compatibility support for U-shields of different brands or types can be achieved. Therefore, centralized and unified management of U-shield devices can be achieved, and through the mode of reusing the same clicker for U-shield devices in a U-shield equipment cabinet, a cloud-based automated testing U-shield click service is provided. Compared with the traditional mode of one-to-one U-shield clickers, it greatly saves the number of resources of U-shield clickers, saves resource costs, improves the test efficiency and test accuracy during the U-shield test, and further reduces the workload of test operation and maintenance.

[0088] According to an embodiment of the present disclosure, in operation S202, a click action corresponding to the click coordinate information is performed on the U shield, and a display interface image of the U shield corresponding to the click action is detected, including:

[0089] Before executing a first click action corresponding to the first click coordinate information on the U shield, detecting a first display interface image of the U shield according to an image acquisition operation instruction;

[0090] In response to the first display interface image, alternately performing a first click action and an image detection action on the U shield to obtain at least two second display interface images of the U shield;

[0091] The display interface image includes a first display interface image and at least two second display interface images.

[0092] The first display interface image is the display image that the display interface of the U-shield to be tested displays according to the business process requirements before the first click action is executed, such as Figure 5A The information in the displayed image is the initial display information, which can be understood as the display content of the first page of the display interface. Since the USB shield to be tested is generally a small device, the display area of ​​the display interface is limited, so there may be content that cannot be fully displayed on the first page (such as Figure 5B , Figure 5C as well as Figure 5D The displayed contents shown may correspond to the 2nd, 3rd and 4th pages of displayed contents respectively). The page turning key of the USB shield needs to be clicked to realize the page turning operation of the displayed contents. The second display interface image is the display image displayed by the display interface of the USB shield to be tested according to the first click action response after the first click action starts to execute the click, such as Figure 5B , Figure 5C as well as Figure 5D Display image shown.

[0093] The U-shield device display information acquisition module 340 receives the click execution instruction issued by the cloud service U-shield click instruction processing submodule 384, and according to the sliding distance (i.e., the first moving trajectory) determined in the U-shield device click addressing submodule 382, ​​before each sliding clicker clicks the U-shield device, the image detection device (such as an information acquisition camera) on the U-shield device is called to perform a photo operation, and the image obtained by the photo is pre-processed by image recognition to determine whether it is necessary to perform a click operation on the page turning button on the U-shield. If it is necessary to click the page turning key, the information acquisition image detection device is called before each click to perform a photo or video recording operation and perform image preprocessing. When all operations that require the page turning key to be clicked are completed, the corresponding U-shield button is clicked according to the click command in the U-shield click operation instruction.

[0094] After obtaining the first display interface image before the first click action, in response to calling the clicker to move to the designated click position of the U shield to be tested according to the first movement trajectory, the first first click action is performed on the page turning key of the U shield to be tested. After the click action is completed, the image detection device is called again through the U shield device display information acquisition module 340 to detect the display interface of the U shield to obtain the second display interface image, which is the first second display interface image after the first first click action is performed on the U shield. Afterwards, the second first click action is performed on the U shield again. After the click action is completed, the display image of the display interface after the second first click action of the U shield is detected again as the third display interface image, that is, the second second display interface image. In this way, the first click action and the image detection action are performed alternately and repeatedly, and the respective display images of the U shield after the first click action at different times can be obtained as at least two second display interface images, that is, the page turning of the U shield is completed, and the U shield display during each page turning process, such as Figure 4 Operation S401 and operation S402 are shown.

[0095] Since the first click action is a click action on the page turning key, the page turning operation of the U-shield to be tested during the test process can be completed, thereby replacing the traditional solution of manually judging the display interface and manually turning pages, greatly saving human resources, reducing test time, and improving test efficiency.

[0096] Among them, in the process of alternately executing the first click action and the image detection action on the U shield, when the two adjacent display interface images respectively obtained by the image detection actions before and after one of the first click actions are consistent. Whether the first click action of the page turning key is terminated to complete the page turning click operation can be judged based on the consistency comparison result of the two adjacent display interface images. Specifically, Figure 5A-5D As shown, before the first click action of the U shield device is performed, a photo (i.e., image detection action) is taken of the content displayed on the screen (i.e., display interface) of the U shield device to form photo 1 (i.e., the first display interface image, as shown in FIG. Figure 5A As shown), after photo 1 is saved; click the same page turning key of the U shield for the second time with the first click action, turn the page of the displayed content of the U shield, and then take a photo of the content displayed on the U shield device screen to form photo 2 (i.e. the second display interface image, as shown Figure 5B After saving Photo 2, compare Photo 2 with Photo 1. If the contents of the two photos are not completely consistent, it proves that the content displayed on the current U-Shield device screen has not yet ended. You need to continue to perform the first click action on the same page turning key of the same U-Shield and take a photo of the content displayed on the U-Shield device screen again to form Photo 3 (i.e. the third display interface image, as shown in Figure 5CAs shown in the figure), after saving photo 3, compare photo 3 with photo 2 to determine whether the contents of the two photos are completely consistent... Repeat this process until the contents of the last two photos are completely consistent, such as when the 4th display interface image and the 5th display interface image are both consistent. Figure 5D If the displayed contents are consistent, it proves that the information displayed on the current U-Shield device screen is at the last page, all information has been displayed, and the page turning operation is completed. The subsequent second click action can be carried out.

[0097] like Figure 3A-3B As shown, the one-to-one U shield click management module 330 is mainly used to receive the operation instructions of the U shield device whose deployment mode is one U shield device and one clicker submitted by the execution machine, and perform click operations on the specified U shield according to the click operation instructions. It is mainly composed of a one-to-one U shield click instruction processing submodule 331, a clicker configuration information management submodule 332, a click operation control submodule 333 and a click operation log recording submodule 334.

[0098] Among them, the one-to-one U shield click instruction processing submodule 331 is mainly used to receive the non-cloud service U shield click control communication message sent by the execution machine external device management module 310, and parse and process the message instruction. The clicker configuration information management submodule 332 is mainly used to store the corresponding mapping relationship between the clicker and the physical U shield. The click operation control submodule 333 is mainly used to operate the specified clicker to perform a click operation according to the click instruction. The click operation log recording submodule 334 is mainly used to record the click operation log of the clicker, including the execution machine name, U shield number, execution machine IP, click operation start time, click operation number, and click interval duration.

[0099] Specifically, when the execution machine corresponding to the U-shield to be tested executes the step of U-shield clicking in the automated script, the execution machine external device management module 310 sends a non-cloud service U-shield click control communication message to the one-to-one U-shield click management module 330, and the one-to-one U-shield click instruction processing submodule 331 receives the message and parses it, and finds the clicker number corresponding to the physical U-shield in the clicker configuration information management submodule 332 according to the U-shield number in the message, and then the click operation control submodule 333 controls the corresponding clicker to complete the U-shield click operation. Finally, the click operation log recording submodule 334 completes the log recording operation in the whole process.

[0100] Regarding the acquisition of the above-mentioned display interface image, the U-shield device display information acquisition module 340 can be used to receive the click execution instruction issued by the cloud service U-shield click instruction processing submodule 384, and according to the sliding distance determined in the U-shield device click addressing submodule 382, ​​before each sliding clicker clicks the U-shield device, the U-shield device information acquisition camera centrally deployed in the cloud is first called, and the camera slides to the specified position to perform the photo operation, and the photo is pre-processed by image recognition to determine whether to perform the page turning button click operation on the U-shield. If the page turning button needs to be clicked, the information collection camera is called before each click to perform the photo operation and image preprocessing. When all operations that require the page turning button to be clicked are completed, the corresponding U-shield button is clicked according to the click command in the U-shield click operation instruction.

[0101] like Figure 5A-5D As shown in the figure, for a specific example, when testing the transaction process of purchasing the ultra-short-term corporate RMB wealth management product "Weekly Dividends" on the corporate online banking, after receiving the basic information such as the completed purchase amount filled in by the user on the terminal device and clicking the submit button, the U-Shield device can display the purchased wealth management product information (such as Figure 5A-5D Before executing the U-shield click operation instruction, the U-shield device display information collection module will call the U-shield device information collection camera deployed in the cloud to slide to the specified position according to the sliding distance determined in the U-shield device click addressing submodule 382, ​​take a picture of the U-shield display information, and obtain the following information: Figure 5A-5D The display interface image shown.

[0102] The image detection device can accurately detect the display information of the display interface of the USB shield to be tested, so as to provide accurate feedback on the display information of the USB shield.

[0103] like Figure 2-Figure 5G As shown, according to an embodiment of the present disclosure, in operation S202, the U shield performs a click action corresponding to the click coordinate information, and detects the display interface image of the U shield corresponding to the click action, further comprising:

[0104] Determine second click coordinate information for executing a second click action according to the first click coordinate information and the second click offset corresponding to the model of the USB shield;

[0105] Determine, according to the second coordinate information, a second movement trajectory for performing a second click operation relative to the U-shield device placement cabinet;

[0106] Determining a second click position corresponding to a second click action according to the second movement trajectory;

[0107] The click coordinate information also includes second click coordinate information, and the click action also includes a second click action.

[0108] Generally, after the display of business transaction information is completed through page turning and clicking operations, it is necessary to perform a confirmation operation of the business transaction information according to the business transaction process, such as confirming that the information is correct (corresponding to the OK key test click) or confirming that the information is incorrect (corresponding to the C key test click). Therefore, after completing at least one first click action for the page turning key and alternately performing image detection to obtain the second display interface image, and confirming that the page turning process is completed, it is not necessary to perform the first click action for the U-shield page turning key click again, and then perform a click operation on the corresponding non-page turning key (such as the OK key or the C key) on the U-shield according to the click command in the U-shield click operation instruction.

[0109] Among them, the second click action can correspond to a click operation on other buttons other than the page-turning key, and the second click coordinate information can correspond to the coordinate information of other buttons other than the page-turning key relative to the U-shield placement cabinet. The second movement trajectory is the movement trajectory of the clicker on the U-shield placement cabinet when the clicker performs the second click action process determined according to the second click coordinate information and the position of the first click coordinate information where the current clicker is located.

[0110] Among them, taking the OK key as the click object of the second click action as an example, according to different models of U shield, the second click offset can be generated according to the fixed position relationship between the OK key of the U shield and the page turning key corresponding to the first click action of the U shield, and the coordinate information of the OK key relative to the U shield placement cabinet can be uniquely determined by combining the first click coordinate information with the above second click offset, that is, the second click coordinate information. Among them, the second click offset can be reflected as the fixed offset distance reflected on the appearance of a certain model of U shield between the second click position corresponding to the second click action and the first click position corresponding to the first click action, and the offset angle of the center line of the center position set by the reference U shield.

[0111] Therefore, after determining the second click coordinate information, the movement trajectory of the clicker from the first click position to the second click position can also be determined based on the first click coordinate information. The second click position is the position of the click object button relative to the U shield placement cabinet when the second click action is executed. It can be seen that the specific content of the second click position can be determined through the second movement trajectory.

[0112] Therefore, not only can the first click of the page turning process be realized, but on this basis, the second click of the confirmation process of the business process test information displayed by the U-shield can be further completed, thereby basically realizing the click test closed loop of the business process test process, and directly realizing the accurate acquisition of business process transaction test information. While reducing click test resources, it effectively guarantees efficient and accurate testing efficiency.

[0113] like Figure 2-Figure 5G As shown, according to an embodiment of the present disclosure, after executing a click action corresponding to the click coordinate information on the U shield and detecting a display interface image of the U shield corresponding to the click action, it also includes:

[0114] Perform a second click action on the second click position of the U shield.

[0115] After the page turning of the USB shield to be tested is completed, the USB shield device display information acquisition module 340 has completed the acquisition of the display interface image, and then in response to the completion of the acquisition process, according to the click command in the USB shield click operation instruction, the second click action is performed on other keys on the USB shield other than the page turning key. The second click action can be a click action on a non-page turning key on the USB shield to be tested, such as a confirmation key (i.e., OK key), a cancel key (i.e., C key), etc., generally a click action on the confirmation key.

[0116] Specifically, after the page turning operation is completed, if the business transaction content of the displayed information corresponding to the first click action is confirmed to be correct, the second click action is performed on the confirmation key of the U shield, and the image of the U shield display interface is as follows: Figure 5E As shown, "Operation successful" is displayed. On this basis, the image of the U shield display interface can also automatically display the next display interface, as shown in the following example. Fig. 5F The display shows "Please confirm again", and based on this, the confirmation OK key is further clicked for a second time to complete the confirmation operation. In addition, if the business transaction content of the displayed information corresponding to the first click action is confirmed to be correct, the cancel C key of the U shield is clicked for a second time, and the image of the U shield display interface is as follows Figure 5G As shown, "Operation Canceled" is displayed.

[0117] Accordingly, the U shield display information acquisition module 340 can acquire the display interface image of the U shield display interface after each second click action is performed. Therefore, the OK button on the U shield is clicked according to the click command in the U shield click operation instruction, thereby completing the entire U shield confirmation process.

[0118] like Figure 2-Figure 5G As shown, according to an embodiment of the present disclosure, in operation S203, the display interface image is parsed and processed to obtain the U shield display information, including:

[0119] In response to the execution of the second click action, the display interface image is sorted and identified to obtain text content in the display interface image;

[0120] Perform splicing and parsing on the text content to obtain the U shield display information.

[0121] like Figure 4 In the operations S403-S404 shown, the U shield display information parsing and comparison module 350 receives the image information (display interface image) captured by the U shield device display information acquisition module 340, such as a camera or other image detection device, and sorts and identifies the first display interface image and multiple second display interface images of the U shield device according to the image acquisition time (such as the photo taking time) through OCR image recognition technology, obtains the display content information corresponding to each display interface image, and parses, removes duplicates, and splices the U shield device display information to form corresponding structured text information as the U shield display information. Among them, the splicing parsing rules can be processed according to the variable name-variable value separator and the separator between variables.

[0122] Therefore, after obtaining the corresponding U shield display information, the structured text information obtained by parsing can be compared with the preset test target information, so as to feedback the comparison result as the test result, as shown in Table 1 below.

[0123] Table 1

[0124]

[0125] like Figure 4As shown, in operation S403, the U shield device display information acquisition module 340 sends the collected image data to the U shield display information parsing and comparison module 350 for text recognition and deduplication processing, in operation S404, all information items are intelligently identified and intelligent distinction of information items is realized, in operation S405, it is judged whether each information item can correspond to the parameter name in the automation script, and in operation S406, it is judged whether the value corresponding to the information item is consistent with the value corresponding to the parameter name in the script. Among them, if the identified information item cannot correspond to the parameter name defined in the automation script, then the feedback message is: the comparison failed, the information to be compared is undefined, and all corresponding information is recorded in the U shield display information display module 360. If the identified information item can correspond to the parameter name defined in the automation script, but the value of the information item is inconsistent with the corresponding parameter value in the automation script, then the feedback message is: the comparison failed, the value of the compared information item is inconsistent with the preset value, and all corresponding information is recorded in the U shield display information display module 360. If the identified information item can correspond to the parameter name defined in the automation script, and the value of the information item is consistent with the corresponding parameter value in the automation script, a feedback message is given: the comparison is successful, and all corresponding information is recorded in the U-shield display information display module 360.

[0126] Specifically, if a U-shield test is performed on a transaction in the process of purchasing an ultra-short-term corporate financial product "Weekly Dividends" on the corporate online banking, all images collected by the U-shield device display information collection module 340 are sorted in the order of collection, and the content of the transaction information display area in the U-shield is recognized through the OCR image recognition technology, and the complete U-shield display information to be processed is obtained as "Product Name: XX Bank "Weekly Dividends" Ultra-short-term Corporate Financial Product, Product Currency: xx Currency, Fund Account: 020004850920066XXXX, Fund Account Currency: xx Currency, Banknote Mark: Cash, Purchase Amount: 100,000.00 Yuan, (XX Bank)" (such as Figure 5A-5D content shown).

[0127] The parsed information is deduplicated and different information items are intelligently distinguished. In the above parsed text, the distinguishable information items are "product name", "product currency", "fund account", "fund account currency", "banknote exchange mark" and "purchase amount"; the actual values ​​corresponding to the information items are "XX Bank's 'weekly dividend' ultra-short-term corporate RMB wealth management product", "RMB", "020004850920066XXXX", "RMB", "cash" and "100,000.00 yuan". When the information items displayed in the U-Shield can all be found in the automation script to correspond to the defined parameter names, and the actual values ​​corresponding to the information items are the same as the parameter values ​​corresponding to the parameter names, it means that the actual displayed information on the U-Shield is consistent with the expected information, and the comparison is successful. On the contrary, if the information item cannot find the corresponding parameter name, or the parameter name can be found, but the actual value corresponding to the information item is different from the parameter value corresponding to the parameter name, the comparison fails.

[0128] In addition, if Figure 3A and Figure 3B As shown, the U-shield display information display module 360 ​​is mainly used to display the display information and comparison feedback information of the U-shield device during the execution of the test case to the testers or operation and maintenance personnel during the testing process of the U-shield, mainly including the clicker number, U-shield number, execution machine name, execution machine IP, U-shield device expected display information, actual U-shield display information, comparison results between expected information and actual display information, U-shield interaction photos and pictures, comparison time and other information.

[0129] Therefore, through the above-mentioned method of the embodiment of the present invention regarding the U-shield device information collection and processing function, it is realized that during the process of clicking the U-shield device, the click position and click path of the clicker are confirmed, and the information displayed in the U-shield device is collected, and the information displayed by the U-shield device is compared with the preset test acceptance rules, thereby avoiding blind spots in the test verification process. In the case of abnormal functions of some application systems or abnormal U-shield drivers, there is a risk of inconsistency between the prompt information of the U-shield device and the corresponding system operation after the actual user operation. Through the U-shield interactive information collection device, the collection of the displayed information of the U-shield device during the U-shield test is realized, and it is matched and compared with the preset information, so as to promptly discover the problem of inconsistency between the displayed information of the U-shield device and the actual prompt information of the system, thereby avoiding misleading the end user and realizing a true closed loop of the test process.

[0130] Specifically, according to the above method of the embodiment of the present disclosure, when there are a large number of U-shield devices in the automated test, the number of resources of the U-shield clicker can be greatly saved compared to the one-to-one U-shield clicker mode. Moreover, as long as U-shield devices of different brands and models can be placed in a single physical slot in the U-shield placement cabinet, they can be compatible and supported. In addition, through the cloud mode automated test U-shield click device, centralized and unified management of U-shield devices is realized, and through the mode of reusing the same set of click devices by the U-shield devices in the same U-shield equipment cabinet, a cloud-based mode is realized, and a cloud mode automated test U-shield click service is provided, which reduces costs and the workload of U-shield device maintenance. At the same time, it solves the problem of user misleading in the business stage due to the fact that the U-shield device information is not collected and compared during the automated test, resulting in the inconsistency between the actual display information of the U-shield and the display information required for the target transaction. It can be seen that according to the above method of the embodiment of the present disclosure, all interactive information in the automated test process of the U-shield device can be captured more comprehensively, so that the entire test process automation realizes a complete closed loop.

[0131] Based on the above-mentioned method for obtaining U shield display information applied to U shield test, the present disclosure also provides a device for obtaining U shield display information applied to U shield test. Figure 6 The device is described in detail.

[0132] Figure 6 The structural block diagram of a U-shield display information acquisition device 600 applied to a U-shield test according to an embodiment of the present disclosure is schematically shown.

[0133] like Figure 6 As shown, the U shield display information acquisition device applied to the U shield test of this embodiment includes a coordinate determination module 610, a click detection module 620 and an information acquisition module 630.

[0134] The coordinate determination module 610 is used to determine the click coordinate information of the USB shield according to the USB shield to be tested. In one embodiment, the coordinate determination module 610 can be used to perform the operation S201 described above, which will not be repeated here.

[0135] The click detection module 620 is used to perform a click action corresponding to the click coordinate information on the U shield, and detect the display interface image of the U shield corresponding to the click action. In one embodiment, the click detection module 620 can be used to perform the operation S202 described above, which will not be repeated here.

[0136] The information acquisition module 630 is used to parse the display interface image and obtain the U shield display information. In one embodiment, the information acquisition module 630 can be used to perform the operation S203 described above, which will not be repeated here.

[0137] According to an embodiment of the present disclosure, any multiple modules of the coordinate determination module 610, the click detection module 620, and the information acquisition module 630 can be combined into one module for implementation, or any one of the modules can be split into multiple modules. Alternatively, at least part of the functions of one or more of these modules can be combined with at least part of the functions of other modules and implemented in one module. According to an embodiment of the present disclosure, at least one of the coordinate determination module 610, the click detection module 620, and the information acquisition module 630 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application-specific integrated circuit (ASIC), or can be implemented by hardware or firmware such as any other reasonable way of integrating or packaging the circuit, or implemented in any one of the three implementation methods of software, hardware, and firmware, or in any appropriate combination of any of them. Alternatively, at least one of the coordinate determination module 610, the click detection module 620, and the information acquisition module 630 can be at least partially implemented as a computer program module, and when the computer program module is run, the corresponding function can be executed.

[0138] Figure 7 A block diagram of an electronic device 700 suitable for implementing a method for acquiring U-shield display information applied to a U-shield test according to an embodiment of the present disclosure is schematically shown.

[0139] An embodiment of the present disclosure also provides an electronic device, comprising: one or more processors; a memory for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors execute the above-mentioned U-shield display information acquisition method applied to U-shield testing.

[0140] like Figure 7 As shown, the electronic device 700 according to an embodiment of the present disclosure includes a processor 701, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 702 or a program loaded from a storage part 708 to a random access memory (RAM) 703. The processor 701 may include, for example, a general-purpose microprocessor (such as a CPU), an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (for example, an application-specific integrated circuit (ASIC)), etc. The processor 701 may also include an onboard memory for caching purposes. The processor 701 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present disclosure.

[0141] In RAM 703, various programs and data required for the operation of electronic device 700 are stored. Processor 701, ROM 702 and RAM 703 are connected to each other via bus 704. Processor 701 performs various operations of the method flow according to the embodiment of the present disclosure by executing the program in ROM 702 and / or RAM 703. It should be noted that the program can also be stored in one or more memories other than ROM 702 and RAM 703. Processor 701 can also perform various operations of the method flow according to the embodiment of the present disclosure by executing the program stored in the one or more memories.

[0142] According to an embodiment of the present disclosure, the electronic device 700 may further include an input / output (I / O) interface 705, which is also connected to the bus 704. The electronic device 700 may further include one or more of the following components connected to the I / O interface 705: an input portion 706 including a keyboard, a mouse, etc.; an output portion 707 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage portion 708 including a hard disk, etc.; and a communication portion 709 including a network interface card such as a LAN card, a modem, etc. The communication portion 709 performs communication processing via a network such as the Internet. A drive 710 is also connected to the I / O interface 705 as needed. A removable medium 711, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 710 as needed, so that a computer program read therefrom is installed into the storage portion 708 as needed.

[0143] An embodiment of the present disclosure further provides a computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, causes the processor to execute the above-mentioned method for obtaining U-shield display information applied to U-shield testing.

[0144] The computer-readable storage medium of the embodiments of the present disclosure may be included in the device / apparatus / system described in the above embodiments; or it may exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiments of the present disclosure is implemented.

[0145] According to an embodiment of the present disclosure, a computer-readable storage medium may be a non-volatile computer-readable storage medium, for example, may include but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, an apparatus or a device. For example, according to an embodiment of the present disclosure, a computer-readable storage medium may include the ROM and / or RAM described above and / or one or more memories other than ROM and RAM.

[0146] An embodiment of the present disclosure further provides a computer program product, including a computer program, which, when executed by a processor, implements the above-mentioned method for obtaining U-shield display information applied to U-shield testing.

[0147] The computer program product of the embodiment of the present disclosure includes a computer program, which contains program codes for executing the method shown in the flowchart. When the computer program product is run in a computer system, the program codes are used to enable the computer system to implement the above method of the embodiment of the present disclosure.

[0148] When the computer program is executed by the processor, the above functions defined in the system / device of the embodiment of the present disclosure are performed. According to the embodiment of the present disclosure, the system, device, module, unit, etc. described above can be implemented by a computer program module.

[0149] In one embodiment, the computer program may rely on a tangible storage medium such as an optical storage device, a magnetic storage device, etc. In another embodiment, the computer program may also be transmitted and distributed in the form of a signal on a network medium, and downloaded and installed through a communication part, and / or installed from a removable medium. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0150] In such an embodiment, the computer program can be downloaded and installed from the network through the communication part, and / or installed from a removable medium. When the computer program is executed by the processor, the above-mentioned functions defined in the system of the embodiment of the present disclosure are performed. According to the embodiment of the present disclosure, the system, device, means, module, unit, etc. described above can be implemented by a computer program module.

[0151] According to an embodiment of the present disclosure, the program code for executing the computer program provided by the embodiment of the present disclosure can be written in any combination of one or more programming languages. Specifically, these computing programs can be implemented using high-level process and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, Java, C++, python, "C" language or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on the remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (for example, using an Internet service provider to connect through the Internet).

[0152] The flow charts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0153] It will be appreciated by those skilled in the art that the features described in the various embodiments and / or claims of the present disclosure may be combined and / or combined in a variety of ways, even if such combinations and / or combinations are not explicitly described in the present disclosure. In particular, the features described in the various embodiments and / or claims of the present disclosure may be combined and / or combined in a variety of ways without departing from the spirit and teachings of the present disclosure. All of these combinations and / or combinations fall within the scope of the present disclosure.

[0154] The embodiments of the present disclosure are described above. However, these embodiments are only for illustrative purposes and are not intended to limit the scope of the present disclosure. Although the embodiments are described above separately, this does not mean that the measures in the various embodiments cannot be used in combination to advantage. The scope of the present disclosure is defined by the attached claims and their equivalents. Without departing from the scope of the present disclosure, those skilled in the art may make a variety of substitutions and modifications, which should all fall within the scope of the present disclosure.

Claims

1. A method for obtaining U-shield display information applied to U-shield testing, wherein: include: Determine the click coordinate information of the U-shield according to the U-shield to be tested; The click coordinate information is the position information of the click position on the U shield relative to the specific area where the U shield is located; Executing a click action corresponding to the click coordinate information on the U shield, and detecting a display interface image of the U shield corresponding to the click action; as well as Analyze the display interface image to obtain the U shield display information; In the determining the click coordinate information of the U-shield according to the U-shield to be tested, it includes: determining the physical slot number of the U-shield device placement cabinet where the U-shield is located according to the physical U-shield number of the U-shield to be tested; and determining the click coordinate information of the U-shield according to the physical slot number.

2. The method according to claim 1, wherein: In determining the click coordinate information of the U shield according to the physical slot number, the following steps are included: Detecting a U-shield slot image of the U-shield corresponding to the physical slot number; Determining the model of the U-Shield according to the U-Shield grid image; and Determine the click coordinate information of the U shield by using the first click offset determined by the model of the U shield and the U shield coordinate information determined by the physical slot number; Among them, the U-shield coordinate information uniquely defines the coordinates of the U-shield with the U-shield device placement cabinet as the reference coordinate system; the first click offset is the fixed offset distance reflected on the appearance of the U-shield between the first click position corresponding to the first click action and the set center position of the current model of the U-shield and the offset angle of the center line with reference to the set center position of the U-shield.

3. The method according to claim 2, wherein: Before executing a click action corresponding to the click coordinate information on the U shield and detecting a display interface image of the U shield corresponding to the click action, the method further includes: Determine, according to the first click coordinate information, a first movement trajectory relative to the U-shield device placement cabinet for performing the first click operation; Determining a first click position corresponding to the first click coordinate information according to the first movement trajectory; The click coordinate information includes first click coordinate information, and the click action includes the first click action; the first click coordinate information is click coordinate information of one of the buttons on the U shield that needs to be clicked.

4. The method according to claim 3, wherein: In the performing a click action corresponding to the click coordinate information on the U shield and detecting a display interface image of the U shield corresponding to the click action, the method includes: Determine second click coordinate information for performing a second click action according to the first click coordinate information and a second click offset corresponding to the model of the USB shield; Determine, according to the second click coordinate information, a second movement trajectory for performing a second click operation relative to the U-shield device placement cabinet; Determining a second click position corresponding to the second click action according to the second movement trajectory; Among them, the click coordinate information also includes second click coordinate information, and the click action also includes a second click action; the second click offset is a fixed offset distance between the second click position corresponding to the second click action and the first click position corresponding to the first click action, which is reflected on the appearance of the current model of the U-shield, and an offset angle of the center line with reference to the center position set by the U-shield.

5. The method according to claim 4, wherein: After executing a click action corresponding to the click coordinate information on the U shield and detecting a display interface image of the U shield corresponding to the click action, the method further includes: A second click action is performed on a second click position of the U shield.

6. The method according to claim 5, wherein: The step of parsing the display interface image to obtain the U-shield display information includes: In response to the execution of the second click action, performing sorting and recognition processing on the display interface image to obtain text content in the display interface image; The text content is spliced ​​and parsed to obtain the U shield display information.

7. A U shield display information acquisition device applied to U shield testing, wherein: include: A coordinate determination module, used to determine the click coordinate information of the U shield according to the U shield to be tested; The click coordinate information is the position information of the click position on the U shield relative to the specific area where the U shield is located; The coordinate determination module is further used to determine the physical slot number of the U-shield device placement cabinet where the U-shield is located according to the physical U-shield number of the U-shield to be tested; and determine the click coordinate information of the U-shield according to the physical slot number; A click detection module, used to perform a click action corresponding to the click coordinate information on the U shield, and detect a display interface image of the U shield corresponding to the click action; as well as The information acquisition module is used to parse and process the display interface image to obtain the U shield display information.

8. An electronic device comprising: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors execute the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, causes the processor to execute the method according to any one of claims 1 to 6.

10. A computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images