Screen smoothness determination method, device, equipment and medium

Abstract

The present application discloses a method, apparatus, device and medium for determining screen fluency, and relates to the field of image processing technology. The specific implementation scheme is: obtaining a video shot by a shooting device; wherein the video is shot when a sliding operation is performed on the screen interface of the device to be tested; for at least two video frames in the video whose similarity is greater than a set threshold and are continuous, the at least two video frames are deleted so that there is one video frame remaining among the at least two video frames; based on the total number of video frames remaining in the video, the screen fluency of the device to be tested is determined. The embodiment of the present application achieves the effect of determining the screen fluency of the device to be tested, and can be used to test the screen fluency of any device to be tested, expanding the scope of application of the screen fluency test.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of computer technology, specifically to image processing technology, and in particular to a method, device, equipment and medium for determining screen smoothness. Background Art

[0002] Currently, it is necessary to test the screen fluency of devices with screens. The main test method is: connect a personal computer to the device under test, and after the tester enters a command line on the personal computer, the personal computer sends the command line to the device under test. The operating system of the device under test returns the screen frame rate data of the device under test based on the received command line, so that the degree of screen freeze can be determined based on the screen frame rate data, and then the fluency can be determined, so that developers can optimize the device under test from the software level based on the fluency.

[0003] Since the operating systems on competing devices often block command lines sent by other devices, the above fluency testing method is not applicable to fluency testing of any device, and thus comparative testing is not possible. Summary of the invention

[0004] The embodiments of the present application provide a method, apparatus, device and storage medium for determining screen fluency to solve the problem that the existing screen fluency testing method is not suitable for determining the screen fluency of any device to be tested and has a narrow scope of application.

[0005] According to a first aspect, a method for determining screen fluency is provided, the method comprising:

[0006] Obtaining a video captured by a camera; wherein the video is captured when a sliding operation is performed on a screen interface of the device to be tested;

[0007] For at least two consecutive video frames in the video whose similarity is greater than a set threshold, deleting the at least two video frames so that one video frame remains among the at least two video frames;

[0008] The screen fluency of the device under test is determined according to the total number of video frames remaining in the video.

[0009] According to a second aspect, a device for determining screen fluency is provided, the device comprising:

[0010] A video acquisition module, used to acquire a video captured by a shooting device; wherein the video is captured when a sliding operation is performed on the screen interface of the device to be tested;

[0011] A video frame deletion module, configured to delete at least two video frames in the video whose similarity is greater than a set threshold and are continuous, so that one video frame remains among the at least two video frames;

[0012] The fluency determination module is used to determine the screen fluency of the device under test according to the total number of video frames remaining in the video.

[0013] According to a third aspect, there is provided an electronic device, comprising:

[0014] at least one processor; and

[0015] a memory communicatively connected to the at least one processor; wherein,

[0016] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the screen fluency determination method of the blockchain as described in any of the embodiments of the present application.

[0017] According to a fourth aspect, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to cause the computer to execute the method for determining screen fluency of a blockchain as described in any one of the embodiments of the present application.

[0018] The technology of the present application realizes the effect of determining the screen fluency of the device to be tested, and can be used to test the screen fluency of any competing device, thereby expanding the scope of application of the screen fluency test.

[0019] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present application.

[0021] Figure 1 is a flow chart of a method for determining screen fluency according to an embodiment of the present application;

[0022] Figure 2A is a flow chart of a method for determining screen fluency according to an embodiment of the present application;

[0023] Figure 2B is a schematic diagram of stacking a video frame according to an embodiment of the present application;

[0024] Figure 3is a structural schematic diagram of a device for determining screen fluency according to an embodiment of the present application;

[0025] Figure 4 It is a block diagram of an electronic device according to the method for determining screen fluency of an embodiment of the present application. DETAILED DESCRIPTION

[0026] The following is a description of exemplary embodiments of the present application in conjunction with the accompanying drawings, including various details of the embodiments of the present application to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present application. Similarly, for the sake of clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0027] During the research and development process, the applicant discovered that the existing screen fluency test methods can specifically include the following four methods: 1) Use the device under test to access the test website page or test software, and the test website or test software will give the corresponding screen frame rate data. 2) The tester manually slides the screen of the device under test and observes the smoothness of the screen of the device under test with the naked eye. 3) The script of the device under test of the IOS system executes the operation, and then uses the Instruments tool provided by XCODE to output the screen frame rate data. 4) Execute the automated script, record the screen frame processing, and obtain the screen frame rate data.

[0028] The disadvantage of the above method 1) is that it can only determine the frame rate data under the set scenarios of the test website or test software, and it is impossible to design specific scenarios to test the fluency according to the characteristics of the device to be tested. The disadvantage of the above method 2) is that errors are inevitable and observation by the naked eye is highly subjective and not convincing. The disadvantage of the above method 3) is that it relies on the IOS system development environment and engineering source code, requires script writing, requires basic programming skills, and has a high learning cost. The disadvantage of the above method 4) is that in some closed systems, device developers often block the developer entrance, resulting in the inability to perform screen fluency tests.

[0029] From the above, it can be seen that there is currently no comprehensive and mature screen fluency testing method to determine the screen fluency of competing devices.

[0030] Figure 1 This is a flowchart of a method for determining screen fluency according to an embodiment of the present application. This embodiment can be applied to the case of determining the screen fluency of a device under test. The method of this embodiment can be executed by a screen fluency determination device, which can be implemented in software and / or hardware and can be integrated on any electronic device with computing capabilities, such as a server.

[0031] like Figure 1 As shown, the method for determining screen fluency disclosed in this embodiment may include:

[0032] S101, obtaining a video captured by a capturing device; wherein the video is captured when a sliding operation is performed on a screen interface of a device to be tested.

[0033] The shooting device includes but is not limited to any device with video shooting function such as a camera, a video recorder and a smart phone. In order to ensure the accuracy of the final screen smoothness, the resolution of the video shot by the shooting device shall not be less than 720p, and the optional resolution is 1080p resolution or 2k resolution.

[0034] In one embodiment, the shooting device is installed in a fixture, and the fixture is fixedly placed above the device to be tested, and the angle formed with the center point of the device to be tested is a preset angle. The optional fixture is fixedly placed directly above the device to be tested, that is, the angle formed with the center point of the device to be tested is 0 degrees. When the screen interface of the device to be tested performs a sliding operation, the video shooting function of the shooting device is turned on at the same time to shoot the screen interface of the device to be tested in real time. When the screen interface of the device to be tested completes the sliding operation, the video shooting function of the shooting device is stopped at the same time, and the shot video is sent to the screen smoothness determination device.

[0035] By acquiring the video captured by the shooting device, a foundation is laid for subsequently performing a video frame deletion operation on at least two consecutive video frames in the video whose similarity is greater than a set threshold.

[0036] S102: For at least two consecutive video frames in the video whose similarity is greater than a set threshold, delete the at least two video frames so that one video frame remains among the at least two video frames.

[0037] In one embodiment, the screen fluency determination device performs frame processing on the acquired video to obtain each video frame, and uses the existing image similarity calculation algorithm, including but not limited to the structural similarity measurement method, the cosine similarity method, the histogram-based image similarity calculation method, the mutual information-based image similarity calculation method, or the feature point-based image similarity calculation method, to perform image similarity calculation on the continuous video frames in all video frames to determine the similarity between each continuous video frame. If the similarity between at least two continuous video frames is greater than a set threshold, a video deletion operation is performed on the at least two continuous video frames, and one video frame remains. The threshold can be set arbitrarily according to the actual situation, and this embodiment does not make a specific limitation.

[0038] Optionally, the video deletion operation includes: deleting the video frame with a later shooting time among at least two consecutive video frames, and leaving the video frame with the earliest shooting time.

[0039] By deleting at least two video frames whose similarity is greater than a set threshold and are continuous in the video, so that at least one video frame remains from the at least two video frames, a foundation is laid for subsequently determining the total number of remaining video frames in the video.

[0040] S103: Determine the screen fluency of the device under test according to the total number of remaining video frames in the video.

[0041] In one embodiment, after the video frames are deleted through S102, the total number of video frames remaining in the video is counted, and according to the size of the total number, the screen smoothness of the device under test is qualitatively analyzed. The more the total number of remaining video frames is, the fewer the continuous video frames in the video whose similarity is greater than the set threshold are, that is, the screen of the device under test does not display the same display content for a long time during the sliding process, that is, the screen smoothness of the device under test is better; correspondingly, the fewer the total number of remaining video frames is, the more the continuous video frames in the video whose similarity is greater than the set threshold are, that is, the screen of the device under test displays the same display content for a long time during the sliding process, that is, the screen smoothness of the device under test is worse.

[0042] Optionally, a first quantity threshold, a second quantity threshold, a third quantity threshold and a fourth quantity threshold are preset, wherein the first quantity threshold is greater than the second quantity threshold, the second quantity threshold is greater than the third quantity threshold, and the third quantity threshold is greater than the fourth quantity threshold. If the total number of video frames remaining in the video is greater than or equal to the first quantity threshold, the screen fluency of the device under test is determined to be "very good"; if the total number of video frames remaining in the video is less than the first quantity threshold, and greater than or equal to the second quantity threshold, the screen fluency of the device under test is determined to be "good"; if the total number of video frames remaining in the video is less than the second quantity threshold, and greater than or equal to the third quantity threshold, the screen fluency of the device under test is determined to be "average"; if the total number of video frames remaining in the video is less than the third quantity threshold, and greater than or equal to the fourth quantity threshold, the screen fluency of the device under test is determined to be "poor"; if the total number of video frames remaining in the video is less than the fourth quantity threshold, the screen fluency of the device under test is determined to be "very poor".

[0043] By determining the screen fluency of the device under test according to the total number of remaining video frames in the video, the effect of testing the screen fluency of the device under test is achieved.

[0044] According to the technical solution of this embodiment, for at least two video frames in the acquired captured video, whose similarity is greater than a set threshold and are continuous, at least two video frames are deleted so that at least two video frames have one video frame left, and finally the screen fluency of the device to be tested is determined according to the total number of video frames remaining in the video, thereby achieving the effect of determining the screen fluency of the device to be tested, and compared with the existing screen fluency test method, it also has the following beneficial effects: 1. The test scene is not restricted and can be carried out in a variety of test scenes, and the test diversity is good. 2. No human participation is required, and the determined screen fluency results are not affected by human subjective judgment, which is very convincing. 3. There is no need to write a test script in advance, and the programming foundation and learning cost requirements for relevant technical personnel are low. 4. It does not rely on the support of the operating system of the device to be tested, so there will be no problem of being blocked by the operating system of the competing device. It can be used to test the screen fluency of any competing device, expanding the scope of application of the screen fluency test.

[0045] Based on the above embodiment, the device under test includes a smart speaker.

[0046] The smart speaker includes a touch screen, and the user can perform touch operations on the screen, including but not limited to clicking, double-clicking and dragging, etc. The smart speaker responds to the user's touch operations and executes corresponding functions.

[0047] By using the smart speaker as the device to be tested, the screen fluency of the smart speaker can be tested, so that the tester can optimize and adjust the screen of the smart speaker according to the test results.

[0048] Figure 2A It is a flowchart of a method for determining screen fluency according to an embodiment of the present application, which is further optimized and expanded based on the above technical solution and can be combined with the above optional implementation methods. Figure 2A As shown, the method may include:

[0049] S201 . Acquire control parameters, control a robotic arm to perform a sliding operation on a screen interface of a device to be tested according to the control parameters, and control a shooting device to shoot a video of the screen.

[0050] In one embodiment, a technician pre-sets control parameters for controlling the movement of the robotic arm, and the control parameters include but are not limited to a sliding frequency control parameter, a sliding amplitude control parameter, a sliding direction control parameter, and a sliding start time control parameter. The screen fluency determination device obtains the control parameters and controls the robotic arm to perform a sliding operation on the screen interface of the device under test according to the control parameters. At the same time, the screen fluency determination device controls the shooting device to start shooting the video when the robotic arm starts to perform the sliding operation according to the code pre-written by the technician, and stops shooting the video when the robotic arm stops performing the interactive operation.

[0051] Optionally, the subject that performs the sliding operation on the screen interface of the device under test can be the above-mentioned mechanical arm, or it can be a human body, such as a human finger, that is, a sliding operation is performed on the screen interface of the device under test manually, and the shooting device is controlled to shoot a video of the screen.

[0052] By obtaining control parameters, controlling the robotic arm to perform a sliding operation on the screen interface of the device to be tested according to the control parameters, and controlling the shooting device to take a video of the screen, the sliding operation on the screen interface and the shooting device to take a video of the screen are automatically realized without human intervention, thereby reducing labor costs and improving accuracy and efficiency.

[0053] S202, obtaining a video captured by a capturing device; wherein the video is captured when a sliding operation is performed on a screen interface of the device to be tested.

[0054] S203: For each video frame in the video, crop a region of the video frame that includes a subject performing the sliding operation.

[0055] In one implementation, entity recognition is performed on the video frame, and the area of ​​the identified subject of the sliding operation is cropped.

[0056] Optionally, S203 includes the following two implementation methods A or B:

[0057] A. Determine a preset position area in the video frame as an area containing a subject performing the sliding operation, and cut the preset position area from the video frame.

[0058] In one implementation, a technician presets a position area, where the preset position area can be a rectangular area at the lower left corner of a video frame or a rectangular area at the lower right corner of a video frame. This embodiment does not impose any limitation on the position area. After the video is acquired, for each video frame, the corresponding preset position area in each video frame is cropped.

[0059] This cropping method is more efficient and easier to implement by cropping the preset position area in the video frame.

[0060] B. Identify, based on image features of the video frame, an area in the video frame that contains the subject performing the sliding operation, and crop the identified area from the video frame.

[0061] In one embodiment, for each video frame, a preset artificial intelligence algorithm is used to identify the area of ​​the video frame containing the subject performing the sliding operation according to the image features of the video frame, wherein the artificial intelligence algorithm includes but is not limited to a classifier-based target detection algorithm or a candidate region-based target detection algorithm, etc., and the area of ​​the subject performing the sliding operation identified in each video frame is cropped.

[0062] By identifying the area containing the subject performing the sliding operation and cropping it out, the cropped area is smaller and has less impact on the original video frame, so the final screen smoothness determination result will be more accurate.

[0063] By cropping the area of ​​the video frame containing the subject performing the sliding operation in each video frame in the video, the influence of the subject performing the sliding operation on the video frame is avoided, thereby ensuring the accuracy of the final screen smoothness determination result.

[0064] S204, push the first captured video frame into a stack; for the nth captured video frame, if the similarity between the nth video frame and the video frame currently on the top of the stack is greater than a preset threshold, delete the nth video frame, otherwise, push the nth video frame into a stack; wherein n is an integer greater than 1 and less than or equal to N, and N is the total number of video frames in the captured video.

[0065] In one embodiment, the video frames are sorted in the order of shooting time, and the video frames are first stored in a sequence table according to the sorting order, and the first video frame with the earliest shooting time is pushed into the stack first according to the obtained sequence table, that is, the first video frame is the top video frame of the stack. Next, the remaining video frames in the sequence table are calculated in similarity with the top video frame of the stack in sequence according to the shooting order. For example, when n is 2, it means that the second video frame is calculated similarity with the top video frame of the stack, and when n is N, it means that the last video frame is calculated similarity with the top video frame of the stack. According to the obtained similarity calculation result, each video frame whose similarity with the top video frame of the stack is greater than a preset threshold is deleted, and each video frame whose similarity with the top video frame of the stack is less than or equal to the preset threshold is pushed into the stack.

[0066] For example, assuming that the preset threshold is 0.6, and the stack includes the second video frame, the third video frame, the fourth video frame, and the fifth video frame in addition to the top video frame. If the similarities between the second video frame, the third video frame, the fourth video frame, and the fifth video frame and the top video frame are 0.8, 0.7, 0.5, and 0.4, respectively, the second video frame and the third video frame are deleted, and the fourth video frame and the fifth video frame are pushed into the stack.

[0067] like Figure 2B As shown, Figure 2B 20 is a schematic diagram of stacking video frames according to an embodiment of the present application, wherein 20 is a sequential stack in this embodiment, 21 represents the first video frame captured, after video frame 21 is stacked, video frame 21 is located at the top of the stack, 22 represents the ath video frame whose similarity to video frame 21 is less than or equal to a preset threshold, after video frame 22 is stacked, video frame 22 is located at the top of the stack, 23 represents the bth video frame whose similarity to video frame 21 is less than or equal to a preset threshold, wherein b is greater than a, that is, video frame 23 is a video frame whose shooting time is after video frame 22. After video frame 23 is stacked, video frame 23 is located at the top of the stack,

[0068] The first captured video frame is pushed into a stack, and then the nth video frame is captured in sequence. If the similarity between the nth video frame and the video frame currently on the top of the stack is greater than a preset threshold, the nth video frame is deleted; otherwise, the nth video frame is pushed into a stack. Since there is no need to calculate the similarities of all adjacent video frames, the time for similarity calculation is saved, thereby improving the efficiency of deleting similar video frames.

[0069] S205: Determine the screen fluency of the device under test according to the total number of remaining video frames in the video.

[0070] In one implementation, the total number of all stacked video frames is counted as the total number of remaining video frames in the video.

[0071] S206 , generating a test report according to the determined screen fluency of the multiple devices under test, wherein the test report includes a test scenario, a total number of remaining video frames corresponding to each device under test, and a comparative test conclusion of the screen fluency of each device under test.

[0072] In one implementation, after S201 to S205 are performed on multiple devices to be tested to determine the screen fluency of the multiple devices to be tested, a test report is generated accordingly, and the test report includes:

[0073] 1. Test scenarios: Test scenarios include but are not limited to web page test scenarios, application software test scenarios, and system software test scenarios.

[0074] 2. The total number of remaining video frames corresponding to each device under test.

[0075] 3. Comparative test conclusions of the screen fluency of each device under test.

[0076] Among them, according to the total number of remaining video frames corresponding to each device under test, a comparative test conclusion of the screen fluency of each device under test is obtained. For example, the devices under test include device A under test, device B under test and device C under test, the total number of remaining video frames corresponding to device A under test is 20, the total number of remaining video frames corresponding to device B under test is 30, and the total number of remaining video frames corresponding to device C under test is 40. Then the comparative test conclusion of the screen fluency is: the screen fluency of device C under test is the highest, the screen fluency of device B under test is second, and the screen fluency of device A under test is the lowest.

[0077] By generating a test report based on the screen fluency of multiple devices to be tested, testers can more intuitively understand the difference in screen fluency between their own devices and competing devices, so that testers can optimize and adjust their own devices in a timely manner.

[0078] According to the technical solution of this embodiment, a technical effect of more accurately and quickly determining the screen fluency of the device to be tested is achieved, and it can be used to test the screen fluency of any competing device, thereby expanding the scope of application of the screen fluency test.

[0079] Figure 3 This is a schematic diagram of a device for determining screen fluency according to an embodiment of the present application. This embodiment can be applied to the case of determining the screen fluency of a device under test. The device of this embodiment can be implemented using software and / or hardware, and can be integrated on any electronic device with computing capabilities, such as a server.

[0080] like Figure 3 As shown, the screen fluency determination device 30 disclosed in this embodiment may include a video acquisition module 31, a video frame deletion module 32 and a fluency determination module 33, wherein:

[0081] Optionally, the device further includes a control parameter acquisition module, which is specifically used to:

[0082] A control parameter is obtained, and according to the control parameter, the robot arm is controlled to perform a sliding operation on the screen interface of the device to be tested, and a shooting device is controlled to shoot a video of the screen.

[0083] Optionally, the device further includes a region clipping module, specifically configured to:

[0084] For each video frame in the video, a region in the video frame including a subject performing the sliding operation is cropped.

[0085] Optionally, the region cropping module is further used to:

[0086] Determine a preset position area in the video frame as an area containing the subject performing the sliding operation, and cut out the preset position area from the video frame; or,

[0087] An area in the video frame containing a subject performing the sliding operation is identified according to image features of the video frame, and the identified area is cropped from the video frame.

[0088] Optionally, the video frame deletion module 32 is specifically used to:

[0089] Push the first captured video frame into the stack;

[0090] For the nth video frame shot in sequence, if the similarity between the nth video frame and the video frame currently on the top of the stack is greater than a preset threshold, the nth video frame is deleted, otherwise, the nth video frame is pushed into the stack; wherein n is an integer greater than 1 and less than or equal to N, and N is the total number of video frames in the video shot.

[0091] Optionally, the device further includes a test report generating module, which is specifically used to:

[0092] A test report is generated according to the determined screen fluency of the multiple devices under test, wherein the test report includes a test scenario, a total number of remaining video frames corresponding to each device under test, and a comparative test conclusion of the screen fluency of each device under test.

[0093] Optionally, the device under test includes a smart speaker.

[0094] The screen fluency determination device 30 disclosed in the embodiment of the present application can execute the screen fluency determination method disclosed in the embodiment of the present application, and has the corresponding functional modules and beneficial effects of the execution method. For the contents not described in detail in this embodiment, please refer to the description in any embodiment of the screen fluency determination method of the present application.

[0095] According to an embodiment of the present application, the present application also provides an electronic device and a readable storage medium.

[0096] like Figure 4, is a block diagram of an electronic device according to a method for determining screen fluency according to an embodiment of the present application. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present application described and / or required herein.

[0097] like Figure 4 As shown, the electronic device includes: one or more processors 401, a memory 402, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. The various components are interconnected using different buses and can be installed on a common mainboard or installed in other ways as needed. The processor can process instructions executed in the electronic device, including instructions stored in or on the memory to display graphical information of a GUI on an external input / output device (such as a display device coupled to an interface). In other embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple electronic devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 4 A processor 401 is taken as an example.

[0098] The memory 402 is a non-transient computer-readable storage medium provided in the present application. The memory stores instructions executable by at least one processor to enable the at least one processor to perform the screen fluency determination method provided in the present application. The non-transient computer-readable storage medium of the present application stores computer instructions, which are used to enable a computer to perform the screen fluency determination method provided in the present application.

[0099] The memory 402 is a non-transient computer-readable storage medium that can be used to store non-transient software programs, non-transient computer executable programs and modules, such as the program instructions / modules corresponding to the screen fluency determination method in the embodiment of the present application (for example, the attached Figure 3 The processor 401 executes various functional applications and data processing of the server by running the non-transient software programs, instructions and modules stored in the memory 402, that is, the screen fluency determination method in the above method embodiment is implemented.

[0100] The memory 402 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created by the use of an electronic device according to the screen fluency determination method, etc. In addition, the memory 402 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory 402 may optionally include a memory remotely arranged relative to the processor 401, and these remote memories may be connected to the electronic device of the screen fluency determination method via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0101] The electronic device of the screen fluency determination method may further include: an input device 403 and an output device 404. The processor 401, the memory 402, the input device 403 and the output device 404 may be connected via a bus or other means. Figure 4 The example of connecting through bus is taken in the following.

[0102] The input device 403 can receive input digital or character information, and generate key signal input related to user settings and function control of the electronic device of the screen fluency determination method, such as a touch screen, a keypad, a mouse, a track pad, a touch pad, an indicator rod, one or more mouse buttons, a trackball, a joystick and other input devices. The output device 404 may include a display device, an auxiliary lighting device (e.g., an LED) and a tactile feedback device (e.g., a vibration motor), etc. The display device may include, but is not limited to, a liquid crystal display (LCD), a light emitting diode (LED) display and a plasma display. In some embodiments, the display device may be a touch screen.

[0103] Various implementations of the systems and techniques described herein can be realized in digital electronic circuit systems, integrated circuit systems, dedicated ASICs (application specific integrated circuits), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0104] These computer programs (also referred to as programs, software, software applications, or code) include machine instructions for programmable processors and can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, device, and / or means (e.g., disk, optical disk, memory, programmable logic device (PLD)) for providing machine instructions and / or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal for providing machine instructions and / or data to a programmable processor.

[0105] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0106] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0107] A computer system may include clients and servers. Clients and servers are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship to each other.

[0108] According to the technical solution of the embodiment of the present application, the effect of determining the screen fluency of the device under test is achieved, and because the method does not rely on the support of the operating system of the device under test, the problem of being blocked by the operating system of the competing device will not occur. It can be used to test the screen fluency of any competing device, thereby expanding the scope of application of the screen fluency test.

[0109] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this application can be executed in parallel, sequentially or in different orders, as long as the expected results of the technical solution disclosed in this application can be achieved, and this document is not limited here.

[0110] The above specific implementations do not constitute a limitation on the protection scope of this application. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of this application should be included in the protection scope of this application.

Claims

1. A method for determining screen fluency, the method comprising: Acquire control parameters, control a mechanical arm to perform a sliding operation on a screen interface of a device under test according to the control parameters, and control a camera to perform video shooting of the screen, including: the camera is installed in a fixture, the fixture is fixedly placed directly above the device under test, and when a sliding operation is performed on the screen interface of the device under test according to a pre-written code, the video shooting function of the camera is simultaneously turned on, and when the sliding operation is completed on the screen interface of the device under test, the video shooting function of the camera is simultaneously stopped; wherein the control parameters are pre-set, including a sliding frequency control parameter, a sliding amplitude control parameter, a sliding direction control parameter, and a sliding start time control parameter; Acquire a video captured by the capturing device; wherein the video is captured when a sliding operation is performed on the screen interface of the device under test; For at least two consecutive video frames in the video whose similarity is greater than a set threshold, deleting the at least two video frames so that one video frame remains among the at least two video frames; Determining the screen fluency of the device under test according to the total number of video frames remaining in the video, including: presetting a first number threshold, a second number threshold, a third number threshold and a fourth number threshold, if the total number of video frames remaining in the video is greater than or equal to the first number threshold, determining that the screen fluency of the device under test is very good; if the total number of video frames remaining in the video is less than the first number threshold, and greater than or equal to the second number threshold, determining that the screen fluency of the device under test is good; if the total number of video frames remaining in the video is less than the second number threshold, and greater than or equal to the third number threshold, determining that the screen fluency of the device under test is average; if the total number of video frames remaining in the video is less than the third number threshold, and greater than or equal to the fourth number threshold, determining that the screen fluency of the device under test is poor; if the total number of video frames remaining in the video is less than the fourth number threshold, determining that the screen fluency of the device under test is very poor; Generate a test report according to the determined screen fluency of the multiple devices under test; the test report includes the test scene, the total number of remaining video frames corresponding to each device under test, and the comparative test conclusion of the screen fluency of each device under test; The device under test includes a touch screen, and touch operations include but are not limited to clicking, double-clicking, and dragging. The device under test executes a corresponding function in response to the touch operation.

2. The method according to claim 1, wherein: Before deleting at least two consecutive video frames in the video whose similarity is greater than a set threshold, the method further includes: For each video frame in the video, a region in the video frame including a subject performing the sliding operation is cropped.

3. The method according to claim 2, wherein: Cropping a region of the video frame containing a subject performing the sliding operation, comprising: Determine a preset position area in the video frame as an area containing the subject performing the sliding operation, and cut out the preset position area from the video frame; or, An area in the video frame containing a subject performing the sliding operation is identified according to image features of the video frame, and the identified area is cropped from the video frame.

4. The method according to claim 1, wherein: For at least two consecutive video frames in the video whose similarity is greater than a set threshold, deleting the at least two video frames so that one video frame remains among the at least two video frames, including: Push the first captured video frame into the stack; For the nth video frame shot in sequence, if the similarity between the nth video frame and the video frame currently on the top of the stack is greater than a preset threshold, the nth video frame is deleted, otherwise, the nth video frame is pushed into the stack; wherein n is an integer greater than 1 and less than or equal to N, and N is the total number of video frames in the video shot.

5. The method according to any one of claims 1 to 4, wherein: The device to be tested includes a smart speaker.

6. A device for determining screen fluency, the device comprising: A control parameter acquisition module, used to acquire control parameters, control the mechanical arm to perform a sliding operation on the screen interface of the device to be tested according to the control parameters, and control the shooting device to shoot a video of the screen; The control parameter acquisition module is specifically used for installing the camera in a fixture, and the fixture is fixedly placed directly above the device under test. When a sliding operation is performed on the screen interface of the device under test according to a pre-written code, the video shooting function of the camera is simultaneously turned on, and when the sliding operation is completed on the screen interface of the device under test, the video shooting function of the camera is simultaneously stopped; wherein the control parameters are pre-set, including a sliding frequency control parameter, a sliding amplitude control parameter, a sliding direction control parameter, and a sliding start time control parameter; A video acquisition module, used to acquire the video captured by the shooting device; wherein the video is captured when a sliding operation is performed on the screen interface of the device under test; A video frame deletion module, configured to delete at least two video frames in the video whose similarity is greater than a set threshold and are continuous, so that one video frame remains among the at least two video frames; A fluency determination module, used to determine the screen fluency of the device under test according to the total number of video frames remaining in the video; The fluency determination module is specifically used to pre-set a first quantity threshold, a second quantity threshold, a third quantity threshold and a fourth quantity threshold. If the total number of video frames remaining in the video is greater than or equal to the first quantity threshold, the screen fluency of the device under test is determined to be very good; if the total number of video frames remaining in the video is less than the first quantity threshold, and greater than or equal to the second quantity threshold, the screen fluency of the device under test is determined to be good; if the total number of video frames remaining in the video is less than the second quantity threshold, and greater than or equal to the third quantity threshold, the screen fluency of the device under test is determined to be average; if the total number of video frames remaining in the video is less than the third quantity threshold, and greater than or equal to the fourth quantity threshold, the screen fluency of the device under test is determined to be poor; if the total number of video frames remaining in the video is less than the fourth quantity threshold, the screen fluency of the device under test is determined to be very poor; A test report generation module is used to generate a test report according to the screen fluency of the determined multiple devices under test; the test report includes the test scene, the total number of remaining video frames corresponding to each device under test, and the comparative test conclusion of the screen fluency of each device under test; The device under test includes a touch screen, and touch operations include but are not limited to clicking, double-clicking, and dragging. The device under test executes a corresponding function in response to the touch operation.

7. The device according to claim 6, wherein: The device also includes a region clipping module, which is specifically used for: For each video frame in the video, a region in the video frame including a subject performing the sliding operation is cropped.

8. The device according to claim 7, wherein: The region clipping module is further specifically used for: Determine a preset position area in the video frame as an area containing the subject performing the sliding operation, and cut out the preset position area from the video frame; or, An area in the video frame containing a subject performing the sliding operation is identified according to image features of the video frame, and the identified area is cropped from the video frame.

9. The device according to claim 6, wherein: The video frame deletion module is specifically used for: Push the first captured video frame into the stack; For the nth video frame shot in sequence, if the similarity between the nth video frame and the video frame currently on the top of the stack is greater than a preset threshold, the nth video frame is deleted, otherwise, the nth video frame is pushed into the stack; wherein n is an integer greater than 1 and less than or equal to N, and N is the total number of video frames in the video shot.

10. The device according to any one of claims 6 to 9, wherein: The device to be tested includes a smart speaker.

11. An electronic device, wherein: include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the screen fluency determination method according to any one of claims 1 to 5.

12. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to enable the computer to execute the screen smoothness determination method according to any one of claims 1 to 5.

Images