Network awareness test method and device, electronic equipment and nonvolatile storage medium

Abstract

The invention discloses a network awareness test method and device, electronic equipment and a nonvolatile storage medium. The method comprises: in response to a service test instruction, performing a service test on a network corresponding to a mobile terminal, the service test being used for simulating a real service operation behavior of a user on the mobile terminal; obtaining an index value of a key quality index of the network in a service test process; and determining index scores corresponding to the index values of the multiple types of key quality indexes, and determining a target perception score according to the index scores corresponding to the multiple types of key quality indexes, the target perception score being used for representing the user perception quality degree of the network corresponding to the mobile terminal. According to the method and the device, the technical problem that a perception test mode in the related technology is not comprehensive in network quality evaluation and cannot effectively and accurately reflect the influence of a network on user perception of various services is solved.

Description

Technical Field

[0001] The present application relates to the technical field of user perception crowdfunding testing, and in particular to a network perception testing method, device, electronic device and non-volatile storage medium. Background Art

[0002] With the acceleration of the construction of shared telecommunications networks and the improvement of the mobile network perception evaluation system, user perception and KPI indicators are equally important. There are many perception test software commonly used in the market, but they cannot accurately and comprehensively present the current occupied cell situation of the user's location and the perception of the user's actual services such as upload, download, and video. In other words, the test methods in related technologies cannot fully reflect the network quality under real services, nor can they effectively and accurately reflect the impact of the network on user perception of various services.

[0003] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0004] The embodiments of the present application provide a network perception testing method, device, electronic device and non-volatile storage medium to at least solve the technical problem that the perception testing method in the related art is not comprehensive enough in evaluating the network quality and cannot effectively and accurately reflect the impact of the network on user perception of various services.

[0005] According to one aspect of an embodiment of the present application, a network perception testing method is provided, comprising: in response to a service testing instruction, performing a service test on a network corresponding to a mobile terminal, wherein the service test is used to simulate the user's actual service operation behavior on the mobile terminal; obtaining index values ​​of key quality indicators of the network during the service testing process, wherein the key quality indicators include: a first indicator for characterizing the perception state of instant messaging services, a second indicator for characterizing the perception state of web browsing services, and a third indicator for characterizing the perception state of video services; determining index scores corresponding to the index values ​​of multiple types of key quality indicators, and determining a target perception score based on the index scores corresponding to the multiple types of key quality indicators, wherein the target perception score is used to characterize the user's perception of the network corresponding to the mobile terminal.

[0006] Optionally, performing a service test on the network corresponding to the mobile terminal includes: connecting to a target speed test server via a hypertext transfer protocol to perform a network rate test, wherein key quality indicators corresponding to the network rate test include at least one of the following: average download rate, peak download rate, average upload rate, and peak upload rate; initiating an end-to-end access test on the uniform resource locator corresponding to the target web page, wherein key quality indicators corresponding to the access test include at least one of the following: domain name system resolution delay, first packet delay, page opening delay, connection establishment delay, request sending delay, and response receiving delay; performing end-to-end access to the target video address and monitoring the indicator values ​​of key quality indicators during the access process to perform a video online viewing test, wherein key quality indicators corresponding to the video online viewing test include at least one of the following: average video download rate, peak rate, and number of freezes.

[0007] Optionally, performing service testing on the network corresponding to the mobile terminal also includes: obtaining coordinate location information corresponding to the mobile terminal, and determining the geographical area corresponding to the coordinate location information; obtaining preset service testing rules corresponding to the geographical area, wherein the preset service testing rules are used to indicate the type of service testing applicable to the geographical area and corresponding test parameters; and performing service testing on the network corresponding to the mobile terminal according to the preset service testing rules.

[0008] Optionally, determining the target perception score based on the indicator scores corresponding to multiple categories of key quality indicators includes: determining the target interval range corresponding to the indicator value; determining the indicator score corresponding to the indicator value according to the scoring rules corresponding to the target interval range; determining the weight coefficient corresponding to each category of key quality indicators, and determining the target perception score based on the weight coefficient and the indicator score.

[0009] Optionally, determining the weight coefficients corresponding to various key quality indicators includes: obtaining historical user behavior data corresponding to the mobile terminal, wherein the historical user behavior data is used to characterize the historical business operation behaviors performed by the user on the mobile terminal; classifying the historical user behavior data according to the user behavior type, and performing feature extraction on the historical user behavior data corresponding to each user behavior type to obtain user behavior characteristics, wherein the user behavior types include: web browsing, video playback, and file transfer, and the user behavior characteristics are at least used to characterize the duration and frequency of the user performing various types of business operation behaviors; based on the user behavior characteristics, determining the weight coefficients corresponding to various key quality indicators, wherein the weight coefficients are at least used to characterize the degree of influence of the perceived experience of each type of business operation behavior on the overall user perception.

[0010] Optionally, each target interval range corresponds to a performance level; the method also includes: determining the performance level corresponding to the indicator value of each key quality indicator; counting the number of key quality indicators at each performance level, and determining the target words based on the quantitative distribution of the key quality indicators in the performance level, wherein the target words are used to reflect the network quality status to the user; sending the target words, the target perception score, and the indicator value of each key quality indicator to the front-end interactive interface for display.

[0011] Optionally, the method also includes: determining the role type and user permissions corresponding to the user account, and determining the scenario type for business testing of the mobile terminal; determining the front-end page data corresponding to the role type, user permissions, and scenario type, wherein the front-end page data corresponding to different role types and / or user permissions and / or scenario types contain different page components; and sending the front-end page data to the front-end interactive interface of the mobile terminal for rendering and display.

[0012] Optionally, the method also includes: uploading relevant data of service testing of the network corresponding to the mobile terminal to the background, wherein the relevant data includes at least one of the following: time information, location information, basic network information, signal quality records, and key quality indicators for the service testing, and the background is used to analyze the relevant data and determine the location of network problems.

[0013] According to another aspect of an embodiment of the present application, a network perception testing device is also provided, including: a service testing module, used to respond to a service testing instruction to perform a service test on the network corresponding to the mobile terminal, wherein the service test is used to simulate the user's actual service operation behavior on the mobile terminal; an indicator value acquisition module, used to obtain the indicator values ​​of key quality indicators of the network during the service testing process, wherein the key quality indicators include: a first indicator for characterizing the perception status of instant messaging services, a second indicator for characterizing the perception status of web browsing services, and a third indicator for characterizing the perception status of video services; a perception scoring module, used to determine the indicator scores corresponding to the indicator values ​​of multiple types of key quality indicators, and determine the target perception score based on the indicator scores corresponding to the multiple types of key quality indicators, wherein the target perception score is used to characterize the user perception of the network corresponding to the mobile terminal.

[0014] According to another aspect of the embodiments of the present application, there is also provided an electronic device, including: a memory and a processor, the processor being used to run a program stored in the memory, wherein the network awareness testing method is executed when the program is run.

[0015] According to another aspect of the embodiments of the present application, a non-volatile storage medium is provided, the non-volatile storage medium includes a stored computer program, wherein the device where the non-volatile storage medium is located executes the network perception testing method by running the computer program.

[0016] According to another aspect of the embodiments of the present application, a computer program product is provided, including a computer program, which implements the steps of the network awareness testing method when the computer program is executed by a processor.

[0017] In an embodiment of the present application, a service test is performed on a network corresponding to a mobile terminal in response to a service test instruction, wherein the service test is used to simulate the actual service operation behavior of a user on a mobile terminal; the index values ​​of key quality indicators of the network are obtained during the service test, wherein the key quality indicators include: a first index for characterizing the perception state of instant messaging services, a second index for characterizing the perception state of web browsing services, and a third index for characterizing the perception state of video services; the index scores corresponding to the index values ​​of multiple types of key quality indicators are determined, and the target perception score is determined according to the index scores corresponding to the multiple types of key quality indicators, wherein the target perception score is used to characterize the user perception of the network corresponding to the mobile terminal. By simulating the operation process of real user behavior on the mobile terminal, user perception data is intuitively displayed, especially various types of services with high usage rates, which can truly reflect the user's service perception, and achieve the purpose of comprehensively and accurately reflecting the user's perception of various types of services under the current network quality, thereby solving the technical problem that the perception test method in the related art is not comprehensive enough in evaluating the network quality and cannot effectively and accurately reflect the impact of the network on the user's perception of various types of services. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0019] Figure 1 is a schematic diagram of a mobile Internet service perception and analysis system provided according to an embodiment of the present application;

[0020] Figure 2 It is a hardware structure block diagram of a computer terminal (or electronic device) for implementing a method for network perception testing provided in an embodiment of the present application;

[0021] Figure 3 It is a schematic diagram of a method flow of a network perception test provided according to an embodiment of the present application;

[0022] Figure 4is a schematic diagram of an example of determining an indicator score provided according to an embodiment of the present application;

[0023] Figure 5 It is a schematic diagram of an example of an evaluation speech technique provided according to an embodiment of the present application;

[0024] Figure 6 It is a schematic diagram of an APP interface provided according to an embodiment of the present application;

[0025] Figure 7 is a schematic diagram of another APP interface provided according to an embodiment of the present application;

[0026] Figure 8 It is a schematic diagram of a system function networking architecture provided according to an embodiment of the present application;

[0027] Fig. 9 It is a structural diagram of a network perception testing device provided according to an embodiment of the present application. DETAILED DESCRIPTION

[0028] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0029] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0030] In order to facilitate those skilled in the art to better understand the embodiments of the present application, some technical terms or nouns involved in the embodiments of the present application are explained as follows:

[0031] KQI (Key Quality Indicator): is an indicator used to measure network quality. It is a standard for measuring network quality and is used to indicate network availability, reliability, and performance. KQI can be used to measure network throughput, latency, packet loss, jitter, and other perceptual indicators, and can also be used to measure network availability, reliability, and security.

[0032] RSRP (Reference Singnal Received Power) is the power value of the common reference signal (CRS) of the cell received by the terminal. The value is the linear average of the power of a single RE within the measurement bandwidth, reflecting the strength of the useful signal in the cell.

[0033] SINR (Signal to Interference Noise Ratio): The ratio of the reference signal power to the interference noise power within the UE detection bandwidth, that is, S / (I+N), where the signal power is the received power of the CRS, and I+N is the sum of the non-serving cell, adjacent channel interference and system internal thermal noise power on the reference signal. It reflects the current channel link quality and is an important target for measuring UE performance parameters.

[0034] MCC (Mobile Country Code): is a three-digit code that uniquely identifies the country or region where a mobile communication network is located.

[0035] MNC (Mobile Network Code): is a two- or three-digit code used in conjunction with MCC to uniquely identify a mobile operator's network.

[0036] TAC (Tracking Area Code): It is an identifier used for the tracking area in the LTE network. It helps the network identify the current geographical area where the user equipment is located to facilitate location update and paging.

[0037] ECI (eNodeB Cell Identity): is the unique identifier of a cell in the LTE network, which is composed of the eNodeB ID and Cell ID.

[0038] PCI (Physical Cell ID): used to identify a cell, especially when switching between cells. It is carried in the downlink reference signal (CRS) of the cell so that the UE (user equipment) can distinguish different cells.

[0039] BAND: Frequency band. In mobile communications, BAND refers to a specific frequency range. Different frequency bands are used for different mobile network technologies, such as 4G (LTE) or 5G NR.

[0040] EARFCN (E-UTRA Absolute Radio Frequency Channel Number): is a number used to identify a radio frequency channel in the LTE network. It contains information such as frequency band and carrier frequency, and is used for frequency configuration for communication between UE and the network.

[0041] In the related art, there are two main ways to obtain mobile Internet service perception data: Method 1 is the test APP (Application) method, and Method 2 is the signaling and DPI (Deep Packet Inspection) monitoring method. Among them, the mobile Internet service perception test APP is part of the mobile Internet service perception analysis system, and its location in the network is as follows: Figure 1 shown.

[0042] In the daily use of mobile networks, user perception data can be intuitively reflected by simulating real user behavior operations on mobile phones or monitoring the actual business operations of real users on mobile phones, especially for services with high usage rates such as page browsing, video playback and instant messaging, which can truly reflect user business perception.

[0043] The business perception test APP should have the end-to-end test function for page browsing, video and instant messaging services, collect relevant business perception KQI indicators, including the wireless environment information, coordinate location information and mobile phone information at that time, and parse, generate and merge data information according to the specification requirements, and provide data results to the data platform. However, the business perception test APP functions in related technologies are not comprehensive in use. Customers need to use multiple APPs to complete a task in the early stage, which is highly complicated to operate, and the test call records cannot be retrieved and analyzed.

[0044] In order to achieve the above requirements and solve the problems existing in the related technology, relevant solutions are provided in the embodiments of the present application, which are described in detail below.

[0045] According to an embodiment of the present application, a method embodiment of a network awareness test is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0046] The method embodiments provided in the embodiments of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Figure 2 The hardware structure block diagram of a computer terminal (or electronic device) for implementing a network perception test method is shown. Figure 2 As shown, the computer terminal 20 (or electronic device) may include one or more (202a, 202b, ..., 202n are used to illustrate) processors 202 (the processor 202 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 204 for storing data, and a transmission device 206 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the BUS bus), a network interface, a power supply and / or a camera. It can be understood by those skilled in the art that Figure 2 The structure shown is only for illustration and does not limit the structure of the above electronic device. Figure 2 More or fewer components as shown, or with Figure 2 Different configurations are shown.

[0047] It should be noted that the one or more processors 202 and / or other data processing circuits described above may generally be referred to herein as "data processing circuits". The data processing circuits may be embodied in whole or in part as software, hardware, firmware, or any other combination thereof. In addition, the data processing circuit may be a single independent processing module, or may be incorporated in whole or in part into any of the other components in the computer terminal 20 (or electronic device). As described in the embodiments of the present application, the data processing circuit acts as a processor control (e.g., selection of a variable resistor terminal path connected to an interface).

[0048] The memory 204 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the network perception test method in the embodiment of the present application. The processor 202 executes various functional applications and data processing by running the software programs and modules stored in the memory 204, that is, the above-mentioned network perception test method is realized. The memory 204 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 204 may further include a memory remotely arranged relative to the processor 202, and these remote memories may be connected to the computer terminal 20 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0049] The transmission device 206 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the computer terminal 20. In one example, the transmission device 206 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 206 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0050] The display may be, for example, a touch screen liquid crystal display (LCD) that enables a user to interact with a user interface of the computer terminal 20 (or electronic device).

[0051] In the above operating environment, the embodiment of the present application provides a network perception testing method. Figure 3 is a schematic diagram of a method flow of a network perception test provided according to an embodiment of the present application, such as Figure 3 As shown, the method comprises the following steps:

[0052] Step S302, in response to the service test instruction, performing a service test on the network corresponding to the mobile terminal, wherein the service test is used to simulate the actual service operation behavior of the user on the mobile terminal;

[0053] Step S304, obtaining the index values ​​of the key quality indicators of the network during the service test, wherein the key quality indicators include: a first index for characterizing the perception state of instant messaging services, a second index for characterizing the perception state of web browsing services, and a third index for characterizing the perception state of video services;

[0054] Step S306, determining the indicator scores corresponding to the indicator values ​​of the multiple types of key quality indicators, and determining the target perception score based on the indicator scores corresponding to the multiple types of key quality indicators, wherein the target perception score is used to characterize the user perception quality of the network corresponding to the mobile terminal.

[0055] Through the above steps, by simulating the operation process of real user behavior on the mobile terminal, the user perception data can be intuitively displayed, especially the various types of services with higher usage rates, which can truly reflect the user's business perception and achieve the purpose of comprehensively and accurately reflecting the user's perception of various types of services under the current network quality. It also solves the technical problem that the perception test method in the relevant technology is not comprehensive enough in evaluating the network quality and cannot effectively and accurately reflect the impact of the network on the user's perception of various types of services.

[0056] The following further introduces the network perception testing method in steps S302 to S306 of the embodiment of the present application.

[0057] In this embodiment, the network perception test method can be implemented in the form of an APP. The test APP has the end-to-end test function for page browsing, video and instant messaging services, collects relevant business perception KQI indicators, including the wireless environment information, coordinate location information and mobile phone information at that time, and parses, generates and merges data information according to the specification requirements, and provides data results to the data platform, so that each authorized user can become a network experience officer, which is used to evaluate the network perception situation in important scenarios of the telecommunications industry, compete with friendly business network performance, accurately analyze and locate potential network problems, speed up network operation and maintenance efficiency, and reduce the time of manual testing and analysis. The following is a detailed introduction.

[0058] In this embodiment, the APP software has an operation interface (front-end interactive interface), and the user can simulate real user behavior on the interface to actively initiate service testing, that is, trigger a service test instruction. Upon receiving the service test instruction, the APP software performs a service test on the network corresponding to the mobile terminal, and automatically collects relevant information of the tested service (that is, the indicator values ​​of key quality indicators of the network during the service test). The supported service tests include speed testing, latency, web browsing, video browsing, etc., as follows.

[0059] In some embodiments of the present application, performing a service test on a network corresponding to a mobile terminal includes the following steps: connecting to a target speed test server via a hypertext transfer protocol to perform a network rate test, wherein the key quality indicators corresponding to the network rate test include at least one of the following: average download rate, peak download rate, average upload rate, and peak upload rate; initiating an end-to-end access test on a uniform resource locator corresponding to a target web page, wherein the key quality indicators corresponding to the access test include at least one of the following: domain name system resolution delay, first packet delay, page opening delay, connection establishment delay, request sending delay, and response receiving delay; performing an end-to-end access to a target video address, and monitoring the indicator values ​​of key quality indicators during the access process to perform a video online viewing test, wherein the key quality indicators corresponding to the video online viewing test include at least one of the following: average video download rate, peak rate, and number of freezes.

[0060] Specifically, in this embodiment, the service test methods include but are not limited to: 1) Network speed test: The service perception test APP can test the network throughput, and the test process includes download speed test and upload speed test. The network speed test is performed through the HTTP (HyperText Transfer Protocol) protocol, and the speed test address can select the currently existing speed test server (i.e., the above-mentioned target speed test server). The indicators collected by the network speed test include but are not limited to: average download rate, peak download rate, average upload rate and peak upload rate, etc.; 2) Access test of browsing services: The business perception test APP can initiate end-to-end access test on mainstream page browsing websites / applications (i.e. the above-mentioned target webpages), initiate HTTP request to the URL at the application layer by specifying the URL (Uniform Resource Locator) and the number of tests, and capture the timestamps of each stage of the request and response process. After parsing, the indicators that can be obtained include but are not limited to: DNS (Domain Name System) resolution delay, first packet delay, page opening delay, connection establishment delay, request sending delay, response receiving delay, etc.; 3) Video online viewing test: The business perception test APP can perform playback test on online video services, perform end-to-end access by specifying the video address to be requested (i.e. the above-mentioned target video), and simulate the actual business of users watching videos online. During the video access process, the indicators that can be obtained include but are not limited to: average video download rate, peak rate, number of freezes, etc.

[0061] In addition, when conducting business testing, the business test content on the test APP has a remote synchronization function. Each time the APP is started, the on-site network data of the simulated test is uploaded to the background server platform with one click. Different test content can be issued according to different geographical areas (taking provinces as an example). That is, the embodiment of the present application can adaptively adjust the test strategies and parameters according to the geographical area such as the province where the mobile terminal is currently located, as follows.

[0062] In some embodiments of the present application, performing a service test on the network corresponding to the mobile terminal also includes: obtaining coordinate location information corresponding to the mobile terminal, and determining the geographical area corresponding to the coordinate location information; obtaining preset service test rules corresponding to the geographical area, wherein the preset service test rules are used to indicate the type of service test applicable to the geographical area and the corresponding test parameters; and performing a service test on the network corresponding to the mobile terminal according to the preset service test rules.

[0063] Specifically, when the user starts the service perception test APP, the APP first calls the positioning service to obtain the user's GPS (Global Positioning System) coordinate location information. The acquired coordinate location information is compared with the background geographic information database to determine the geographical area where the user is located. For example, the database pre-stores the longitude and latitude ranges of various provinces and cities, and determines the user's location by judging whether the longitude and latitude fall within these ranges. The geographic information database may also include network characteristic information of a specific area, such as average network speed, interference conditions, etc., so as to adjust the test parameters later. The server extracts applicable rules from the preset service test rule library according to the geographical area range determined by the user. These rules may include the test frequency, test duration, address of the test server, etc. of different service types. The APP tests the user's network environment according to the acquired preset service test rules. For example, more frequent video service tests may be required in a certain province, while more emphasis is placed on web browsing tests in other provinces.

[0064] For example, if the mobile terminal is located in a densely populated urban center, the preset service test rules may focus more on testing instant messaging and web browsing, because these services are used more frequently in this area; while in remote areas, testing of video services may be more important, because the stability of video streaming may be more affected by network conditions. This method can adjust the test strategy according to the geographical location, ensure that the test results are more in line with the user needs of the specific area, and solve the impact of network performance differences in different regions on the accuracy of test results, including but not limited to dynamically adjusting the test rules when the mobile terminal crosses different geographical areas, and conducting targeted tests in specific geographical areas (such as airports and railway stations) to evaluate the performance of the network in a high-traffic environment.

[0065] After obtaining the indicator values ​​of various key quality indicators, the indicator scores corresponding to the indicator values ​​of multiple types of key quality indicators can be determined, and the target perception scores can be determined based on the indicator scores. The specific steps are as follows.

[0066] In some embodiments of the present application, determining the target perception score based on the indicator scores corresponding to multiple categories of key quality indicators includes the following steps: determining the target interval range corresponding to the indicator value; determining the indicator score corresponding to the indicator value according to the scoring rules corresponding to the target interval range; determining the weight coefficient corresponding to each category of key quality indicators, and determining the target perception score based on the weight coefficient and the indicator score.

[0067] Specifically, in this embodiment, the above key quality indicators are selected as upload rate, download rate, ping, web page opening delay, and video browsing rate. Figure 4As shown, multiple intervals can be defined according to the network perception situation, wherein each interval corresponds to a performance level (i.e., four ranges of excellent, good, medium, and poor). After determining the target interval in which the measured indicator values ​​of each key quality indicator are located, the score corresponding to the indicator value can be calculated according to the scoring calculation rule corresponding to the target interval, for example, when the measured indicator value is excellent (i.e., located within the interval corresponding to the excellent level), the scoring rule can be: test item score = (measured value / maximum range of conditional value) * 30 + 70; when the measured indicator value is good (i.e., located within the interval corresponding to the good level), the scoring rule can be: test item score = IF (ISBLANK (measured value), 0, ((measured value-minimum range) / (maximum range of conditional value)) * 10 + 60), that is, first check whether the measured value is empty (i.e., no data). If it is empty, the test item is directly scored as 0, indicating that no effective test has been conducted or the test result is unavailable. If it is not empty, the indicator score is calculated according to the rule ((measured value - minimum range value) / (maximum range of conditional value))*10+60).

[0068] After obtaining the indicator scores corresponding to the indicator values ​​of each key quality indicator, the weight coefficients corresponding to each key quality indicator can be calculated and summed to obtain the target perception score.

[0069] The above weight coefficients may be pre-set or may be personalized for users of different mobile terminals, as described below.

[0070] As an optional implementation, determining the weight coefficients corresponding to various key quality indicators includes the following steps: obtaining historical user behavior data corresponding to the mobile terminal, wherein the historical user behavior data is used to characterize the historical business operation behaviors performed by the user on the mobile terminal; classifying the historical user behavior data according to the user behavior type, and performing feature extraction on the historical user behavior data corresponding to each user behavior type to obtain user behavior characteristics, wherein the user behavior types include: web browsing, video playback, and file transfer, and the user behavior characteristics are at least used to characterize the duration and frequency of the user performing various types of business operation behaviors; based on the user behavior characteristics, determining the weight coefficients corresponding to various key quality indicators, wherein the weight coefficients are at least used to characterize the degree of influence of the perceived experience of each type of business operation behavior on the overall user perception.

[0071] Specifically, the system first collects the user's historical behavior data from the mobile terminal or the backend server, including various business operations performed by the user on the mobile terminal, such as web browsing, video playback, file transfer, etc. These data usually include the duration and frequency of the user's use of various services in a specific time period, as well as possible service quality feedback. The system classifies the collected historical data according to the user behavior type, for example, classifying all web browsing operations, video playback operations, and file transfer operations separately. Then, feature extraction is performed for each type of operation, and the average usage time, operation frequency, and possible service satisfaction scores of users when performing such services are analyzed to construct user behavior features. These features can reveal the user's preference and sensitivity to different types of services. Based on the extracted user behavior features, the system can calculate the contribution of various business operations to the user's overall perceived experience. For example, if a user frequently plays videos and each playback time is long, the weight coefficient of the KQI indicators related to video playback (such as the average video download rate, the number of freezes, etc.) may be set higher, because these indicators have a greater impact on user perception. Conversely, if the user rarely uses file transfer services, the weight of the indicators related to file transfer may be lower.

[0072] For example, historical data shows that user A uses video playback services for nearly 2 hours a day, performs file transfer operations an average of 3 times a day, and browses the web relatively rarely, averaging less than half an hour a day. In this case, the system may set the KQI indicator weights related to video playback to 40%, the file transfer indicator weights to 30%, and the web browsing indicator weights to 10%. Through this personalized setting, the system can more accurately evaluate and reflect the specific user's perception of network performance, providing more targeted data support for network optimization.

[0073] In addition, in this embodiment, different speech words can be output according to the distribution of key quality indicators in different performance levels. The specific steps are as follows.

[0074] In some embodiments of the present application, each target interval range corresponds to a performance level; the method also includes the following steps: determining the performance level corresponding to the indicator value of each key quality indicator; counting the number of key quality indicators at each performance level, and determining the target words based on the quantitative distribution of the key quality indicators in the performance level, wherein the target words are used to reflect the network quality status to the user; sending the target words, the target perception score, and the indicator value of each key quality indicator to the front-end interactive interface for display.

[0075] Specifically, Figure 5As shown, the five dimensional test items can be colored green, blue, orange, and red according to the color ratio conditions, and the corresponding words can be output according to the color ratio conditions, and the perception results of this test can be directly analyzed and judged through the content of the words. This method can display the network quality status in a user-friendly way, improve the readability and comprehensibility of information, and solve the problem that the test results are difficult for users to intuitively understand, including but not limited to using charts, color coding, etc. in the front-end interactive interface to intuitively display the performance level of each key quality indicator, and dynamically adjust the performance level classification standards based on user feedback to better meet user perception.

[0076] After completing the test, scoring and evaluation, the embodiment of the present application will also return the call record data, as follows.

[0077] In some embodiments of the present application, the method also includes the following steps: uploading relevant data of service testing of the network corresponding to the mobile terminal to the background, wherein the relevant data includes at least one of the following: time information, location information, basic network information, signal quality records, and key quality indicators for the service testing, and the background is used to analyze the relevant data and determine the location of network problems.

[0078] After uploading the test data with one click at the front desk, the one-click test results of the APP are recorded synchronously. In order to facilitate the rapid analysis of the uploaded data, the back-end platform data presents the information of each test item in a corresponding table, and sets the conditional query function (including province, city, district / county, start time, and end time search conditions). The export function can be provided for various report data for easy archiving and viewing. It includes SIM card number, name, test time, role, scene, address, longitude and latitude, basic signal information of 4G / 5G network (such as MCC, MNC, TAC, ECI, PCI, BAND, EARFCN, etc.), signal quality record (SINR, RSRP), uplink and downlink rate, and detailed records of mainstream websites and mainstream App delays and rate results. The back-end can synchronously analyze the front-end test data. Through the analysis, processing, and closed loop of the accumulated call list, it is easy to quickly discover and locate the network depth coverage, interference and various network problems, quickly solve user perception problems, and improve user satisfaction in the region. And it can also provide a base station query module synchronously to improve the efficiency of network optimization and maintenance work.

[0079] Among them, the measurement of SINR of mobile phone terminals depends on the chipset and implementation algorithm of the mobile phone manufacturer. The SINR value of 4G is calculated by the API parsing program, and the SINR value of 5G is directly reported by the mobile phone terminal.

[0080] The following is an introduction to other functions of the APP in the embodiments of the present application.

[0081] First, users can apply for testing permissions from the backend administrator. After confirming the authorization, the user name can log in by entering the mobile phone number and the SMS verification code.

[0082] In this embodiment, the system can adaptively display and adjust the front-end interface of the APP according to the role type of the user account logged in by the user, etc. The specific steps are as follows.

[0083] In some embodiments of the present application, the method also includes the following steps: determining the role type and user permissions corresponding to the user account, and determining the scenario type for business testing of the mobile terminal; determining the front-end page data corresponding to the role type, user permissions, and scenario type, wherein the page components contained in the front-end page data corresponding to different role types and / or user permissions and / or scenario types are different; and sending the front-end page data to the front-end interactive interface of the mobile terminal for rendering and display.

[0084] Specifically, in this embodiment, users can customize information such as roles and scenes, where roles include but are not limited to: business district branch, community branch, rural branch, business hall, others, custom roles, etc.; scenes include but are not limited to: campsites, highlands, communication street, food street, custom scenes, etc.

[0085] For example, for non-professionals such as installation and maintenance, customer service, and users, the front-end interactive interface can be displayed as follows: Figure 6 The simplified version of the mobile network speed test shown in the figure shows the evaluation of each service perception after the one-click test is completed. For the use scenario where highly professional technicians such as optimization testers handle problems on site, the front-end interactive interface can be displayed as follows Figure 7 The interface of the professional version of the mobile network speed test is shown. In this interface, you can see the user's current occupied cell status and the indicators of each service perception test.

[0086] In addition, during the test process, the APP in this embodiment also supports the following functions: support 4G / 5G dual networks; support dual-card mobile phones for competitive testing, and users can also switch data cards by clicking the switch button on the test interface; at the same time, support the judgment of wifi on / off to avoid users using wifi traffic for testing, and will report an error when it detects that the user is testing in wifi connection state; support the judgment of the default traffic card, and can prompt the user to switch the data service card to ensure that the data of the network test is true and accurate; can comprehensively score the user's test results and give different words; after the test is completed, the pop-up evaluation can be used to intuitively judge the quality of the network through "data + color" for each test item, and can submit feedback on the network problems encountered on the day with one click, and provide a custom description for subsequent network analysis and judgment;

[0087] In terms of risk and privacy management, the APP in this embodiment supports remotely shutting down the data collection and upload functions of the APP to avoid further impact on users. Users can choose to manually shut down the data collection and upload functions. In the case of manual shutdown, the server cannot force it to open or the APP can open it by itself. In addition, the business perception information collected by the test APP and uploaded back to the server does not contain information involving the user's personal privacy (including but not limited to: the message content of social software, the content of the HTTP web pages visited, the user's mobile phone number collected without the user's permission, the user's text message content, etc.).

[0088] At the same time, the APP also has a corresponding background module, which includes role management (setting functional permissions corresponding to different roles), user management (adding test user permissions to facilitate the management of front-end test terminal numbers), 5G one-click test (querying detailed test data and problem descriptions uploaded at the front desk), personal settings (setting and modifying login passwords) and other functions. After uploading the test data with one click at the front desk, the one-click test results of the APP are recorded synchronously. In order to facilitate the rapid analysis of the uploaded data, the background platform data presents the information of each test item in a corresponding table, and sets conditional query functions (including province, city, district / county, start time, and end time search conditions). Export functions can be provided for various report data for easy archiving and viewing.

[0089] Figure 8 is a schematic diagram of a system function networking architecture provided according to an embodiment of the present application, such as Figure 8 As shown, in an embodiment of the present application, the mobile terminal API data is parsed based on the standardized 3GPP (3rd Generation Partnership Project) protocol to generate parameter fields required for network evaluation; the parameter fields are presented on the perception test APP with a fixed sampling period according to the presentation requirements; user perception performance indicators are obtained through FTP (File Transfer Protocol) upload and download, web page, video address speed measurement, etc.; the back-end network management is responsible for functions such as decentralization and domain division, call bill data storage, and mobile terminal parameter configuration.

[0090] The network perception test method of the present application performs network testing by simulating real business operation behaviors, obtains the index values ​​of key quality indicators, and determines the user perception of the network based on these index values, that is, the target perception score. This method can not only comprehensively evaluate the network quality, but also dynamically adjust the weight coefficients of key quality indicators according to user behavior characteristics, so that the test results are closer to the user's actual experience. In addition, by presenting the test results to the user in the form of performance levels and target words, the readability and comprehension of the information are improved, which helps users to intuitively understand the network quality status. Analyzing relevant data in the background can quickly locate network problems and provide data support for network optimization, thereby improving the overall network service quality and enhancing user experience. By identifying role types, user permissions and scenario types, personalized front-end page data can be provided to different users, further improving the accuracy and satisfaction of services.

[0091] According to an embodiment of the present application, an embodiment of a network awareness testing device is also provided. Fig. 9 Schematic diagram of a network perception test device provided according to an embodiment of the present application. Fig. 9 As shown, the device comprises:

[0092] The service test module 90 is used to perform a service test on the network corresponding to the mobile terminal in response to the service test instruction, wherein the service test is used to simulate the actual service operation behavior of the user on the mobile terminal;

[0093] The indicator value acquisition module 92 is used to obtain the indicator values ​​of the key quality indicators of the network during the service test, wherein the key quality indicators include: a first indicator for characterizing the perception state of instant messaging services, a second indicator for characterizing the perception state of web browsing services, and a third indicator for characterizing the perception state of video services;

[0094] The perception scoring module 94 is used to determine the indicator scores corresponding to the indicator values ​​of multiple categories of key quality indicators, and determine the target perception score based on the indicator scores corresponding to the multiple categories of key quality indicators, wherein the target perception score is used to characterize the user perception of the network corresponding to the mobile terminal.

[0095] Optionally, performing a service test on the network corresponding to the mobile terminal includes: connecting to a target speed test server via a hypertext transfer protocol to perform a network rate test, wherein key quality indicators corresponding to the network rate test include at least one of the following: average download rate, peak download rate, average upload rate, and peak upload rate; initiating an end-to-end access test on the uniform resource locator corresponding to the target web page, wherein key quality indicators corresponding to the access test include at least one of the following: domain name system resolution delay, first packet delay, page opening delay, connection establishment delay, request sending delay, and response receiving delay; performing end-to-end access to the target video address and monitoring the indicator values ​​of key quality indicators during the access process to perform a video online viewing test, wherein key quality indicators corresponding to the video online viewing test include at least one of the following: average video download rate, peak rate, and number of freezes.

[0096] Optionally, performing service testing on the network corresponding to the mobile terminal also includes: obtaining coordinate location information corresponding to the mobile terminal, and determining the geographical area corresponding to the coordinate location information; obtaining preset service testing rules corresponding to the geographical area, wherein the preset service testing rules are used to indicate the type of service testing applicable to the geographical area and corresponding test parameters; and performing service testing on the network corresponding to the mobile terminal according to the preset service testing rules.

[0097] Optionally, determining the target perception score based on the indicator scores corresponding to multiple categories of key quality indicators includes: determining the target interval range corresponding to the indicator value; determining the indicator score corresponding to the indicator value according to the scoring rules corresponding to the target interval range; determining the weight coefficient corresponding to each category of key quality indicators, and determining the target perception score based on the weight coefficient and the indicator score.

[0098] Optionally, determining the weight coefficients corresponding to various key quality indicators includes: obtaining historical user behavior data corresponding to the mobile terminal, wherein the historical user behavior data is used to characterize the historical business operation behaviors performed by the user on the mobile terminal; classifying the historical user behavior data according to the user behavior type, and performing feature extraction on the historical user behavior data corresponding to each user behavior type to obtain user behavior characteristics, wherein the user behavior types include: web browsing, video playback, and file transfer, and the user behavior characteristics are at least used to characterize the duration and frequency of the user performing various types of business operation behaviors; based on the user behavior characteristics, determining the weight coefficients corresponding to various key quality indicators, wherein the weight coefficients are at least used to characterize the degree of influence of the perceived experience of each type of business operation behavior on the overall user perception.

[0099] Optionally, each target interval range corresponds to a performance level; the network perception testing device is also used to: determine the performance level corresponding to the indicator value of each key quality indicator; count the number of key quality indicators at each performance level, and determine the target speech based on the quantitative distribution of the key quality indicators in the performance level, wherein the target speech is used to reflect the network quality status to the user; send the target speech, target perception score, and the indicator value of each key quality indicator to the front-end interactive interface for display.

[0100] Optionally, the network perception testing device is also used to: determine the role type and user permissions corresponding to the user account, and determine the scenario type for business testing of the mobile terminal; determine the front-end page data corresponding to the role type, user permissions, and scenario type, wherein the front-end page data corresponding to different role types and / or user permissions and / or scenario types contain different page components; and send the front-end page data to the front-end interactive interface of the mobile terminal for rendering and display.

[0101] Optionally, the network perception testing device is also used to upload relevant data of service testing of the network corresponding to the mobile terminal to the background, wherein the relevant data includes at least one of the following: time information, location information, basic network information, signal quality records, and key quality indicators for the service testing. The background is used to analyze the relevant data and determine the location of network problems.

[0102] It should be noted that the various modules in the above-mentioned network perception testing device can be program modules (for example, a set of program instructions that implement a certain specific function) or hardware modules. For the latter, it can be expressed in the following forms, but is not limited to this: the expression form of each of the above-mentioned modules is a processor, or the functions of each of the above-mentioned modules are implemented by a processor.

[0103] It should be noted that the network perception test device provided in this embodiment can be used to perform Figure 3 The network perception test method shown, therefore, the relevant explanations and instructions on the above-mentioned network perception test method are also applicable to the embodiments of the present application and will not be repeated here.

[0104] An embodiment of the present application also provides a non-volatile storage medium, the non-volatile storage medium includes a stored computer program, wherein the device where the non-volatile storage medium is located executes the following network perception test method by running the computer program: in response to a service test instruction, performing a service test on the network corresponding to the mobile terminal, wherein the service test is used to simulate the user's actual service operation behavior on the mobile terminal; obtaining the index values ​​of key quality indicators of the network during the service test, wherein the key quality indicators include: a first indicator for characterizing the perception state of instant messaging services, a second indicator for characterizing the perception state of web browsing services, and a third indicator for characterizing the perception state of video services; determining the index scores corresponding to the index values ​​of multiple types of key quality indicators, and determining the target perception score based on the index scores corresponding to the multiple types of key quality indicators, wherein the target perception score is used to characterize the user's perception of the network corresponding to the mobile terminal.

[0105] The embodiment of the present application also provides a computer program product, including a computer program, which, when executed by a processor, implements the steps of the network perception testing method described in each embodiment of the present application: in response to a service test instruction, performing a service test on the network corresponding to the mobile terminal, wherein the service test is used to simulate the user's actual service operation behavior on the mobile terminal; obtaining the index values ​​of key quality indicators of the network during the service test, wherein the key quality indicators include: a first index for characterizing the perception state of instant messaging services, a second index for characterizing the perception state of web browsing services, and a third index for characterizing the perception state of video services; determining the index scores corresponding to the index values ​​of multiple types of key quality indicators, and determining the target perception score based on the index scores corresponding to the multiple types of key quality indicators, wherein the target perception score is used to characterize the user's perception of the network corresponding to the mobile terminal.

[0106] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0107] In the above embodiments of the present application, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0108] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0109] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0110] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0111] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, disk or optical disk and other media that can store program codes.

[0112] The above is only a preferred embodiment of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A network perception testing method, characterized in that: include: In response to the service test instruction, a service test is performed on the network corresponding to the mobile terminal, wherein the service test is used to simulate the actual service operation behavior of the user on the mobile terminal; Acquire the index values ​​of the key quality indicators of the network during the service test, wherein the key quality indicators include: a first index for characterizing the perception state of instant messaging services, a second index for characterizing the perception state of web browsing services, and a third index for characterizing the perception state of video services; Determine the indicator scores corresponding to the indicator values ​​of multiple categories of key quality indicators, and determine the target perception score based on the indicator scores corresponding to the multiple categories of key quality indicators, wherein the target perception score is used to characterize the user perception quality of the network corresponding to the mobile terminal.

2. The network perception testing method according to claim 1, characterized in that: The service test of the network corresponding to the mobile terminal includes: Connecting to a target speed test server via a hypertext transfer protocol to perform a network speed test, wherein the key quality indicator corresponding to the network speed test includes at least one of the following: average download speed, peak download speed, average upload speed, and peak upload speed; Initiate an end-to-end access test for the uniform resource locator corresponding to the target web page, wherein the key quality indicators corresponding to the access test include at least one of the following: domain name system resolution delay, first packet delay, page opening delay, connection establishment delay, request sending delay, and response receiving delay; Perform end-to-end access to the target video address and monitor the index value of the key quality indicator during the access process to perform a video online viewing test, wherein the key quality indicator corresponding to the video online viewing test includes at least one of the following: average video download rate, peak rate, and number of freezes.

3. The network perception testing method according to claim 2, characterized in that: The service test of the network corresponding to the mobile terminal also includes: Acquire the coordinate location information corresponding to the mobile terminal, and determine the geographical area range corresponding to the coordinate location information; Obtain preset service test rules corresponding to the geographical area, wherein the preset service test rules are used to indicate the type of service test applicable to the geographical area and the corresponding test parameters; perform the service test on the network corresponding to the mobile terminal according to the preset service test rules.

4. The network perception testing method according to claim 1, characterized in that: Determining the target perception score according to the indicator scores corresponding to the multiple types of key quality indicators includes: Determine a target interval range corresponding to the indicator value; Determine the indicator score corresponding to the indicator value according to the scoring rule corresponding to the target interval range; Determine the weight coefficients corresponding to each type of key quality indicator, and determine the target perception score based on the weight coefficients and the indicator scores.

5. The network perception testing method according to claim 4, characterized in that: Determine the weight coefficients corresponding to each type of key quality indicators include: Acquire historical user behavior data corresponding to the mobile terminal, wherein the historical user behavior data is used to characterize historical business operation behaviors performed by the user on the mobile terminal; The historical user behavior data is classified according to the user behavior type, and the feature extraction is performed on the historical user behavior data corresponding to each type of the user behavior type to obtain the user behavior feature, wherein the user behavior type includes: web browsing, video playing, file transfer, and the user behavior feature is at least used to characterize the duration and frequency of the user performing each type of business operation behavior; According to the user behavior characteristics, weight coefficients corresponding to each type of key quality indicator are determined, wherein the weight coefficients are at least used to characterize the degree of influence of the perceived experience of each type of business operation behavior on the overall user perception.

6. The network perception testing method according to claim 4, characterized in that: Each of the target intervals corresponds to a performance level; the method further comprises: Determining the performance level corresponding to the indicator value of each of the key quality indicators; Counting the number of the key quality indicators at each performance level, and determining a target speech based on the distribution of the number of the key quality indicators at the performance level, wherein the target speech is used to reflect the network quality status to the user; The target speech, the target perception score, and the indicator value of each key quality indicator are sent to the front-end interactive interface for display.

7. The network perception testing method according to claim 4, characterized in that: The method further comprises: Determine the role type and user authority corresponding to the user account, and determine the scenario type for the mobile terminal to perform business testing; Determining front-end page data corresponding to the role type, the user authority, and the scenario type, wherein the front-end page data corresponding to different role types and / or user authorities and / or scenario types contain different page components; The front-end page data is sent to the front-end interactive interface of the mobile terminal for rendering and display.

8. The network perception testing method according to claim 1, characterized in that: The method further comprises: The relevant data of the service test on the network corresponding to the mobile terminal is uploaded to the background, wherein the relevant data includes at least one of the following: time information, location information, basic network information, signal quality records, and the key quality indicators of the service test. The background is used to analyze the relevant data and determine the location of network problems.

9. A network perception test device, characterized in that: include: A service testing module, configured to perform a service test on a network corresponding to a mobile terminal in response to a service testing instruction, wherein the service test is configured to simulate a real service operation behavior of a user on the mobile terminal; An indicator value acquisition module, used to obtain the indicator values ​​of the key quality indicators of the network during the service test, wherein the key quality indicators include: a first indicator for characterizing the perception state of instant messaging services, a second indicator for characterizing the perception state of web browsing services, and a third indicator for characterizing the perception state of video services; The perception scoring module is used to determine the indicator scores corresponding to the indicator values ​​of multiple categories of key quality indicators, and determine the target perception score based on the indicator scores corresponding to the multiple categories of key quality indicators, wherein the target perception score is used to characterize the user perception of the network corresponding to the mobile terminal.

10. An electronic device, characterized in that: include: A memory and a processor, wherein the processor is used to run a program stored in the memory, wherein the program executes the network perception testing method described in any one of claims 1 to 8 when running.

11. A non-volatile storage medium, characterized in that: The non-volatile storage medium includes a stored computer program, wherein the device where the non-volatile storage medium is located executes the network perception testing method described in any one of claims 1 to 8 by running the computer program.

12. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the network awareness testing method described in any one of claims 1 to 8 are implemented.

Images