Test method, system and device of edge computing system, server and medium

Abstract

The embodiment of the invention discloses a test method, system and device for an edge computing system, a data recharge server and a medium, and the method comprises the steps: storing road data uploaded by roadside collection equipment as original road data, the roadside collection equipment comprising a camera and / or a radar; marking the data information of each traffic participant in the original road data in the current measurement scene, and taking a marking result as a standard data result of each traffic participant; sending the original road data to a to-be-tested system; receiving a sensing result sent by the to-be-tested system, the sensing result being data information about each traffic participant identified by the to-be-tested system from the original road data; the difference value between the sensing result and the standard data result is calculated, the performance of the to-be-tested system is evaluated according to the difference value, and by means of the technical scheme, the problem that the edge computing system is low in testing efficiency is solved.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of edge computing technology, and more specifically, to a testing method, system, device, data re-injection server and medium for an edge computing system. Background Art

[0002] With the rapid development of the Internet of Things and big data technologies, more and more devices and sensors are connected to the network, generating a large amount of data. However, transmitting this data to the cloud for processing and analysis will face problems such as network latency and bandwidth bottlenecks. The emergence of edge computing technology can sink computing and data processing to the edge of the network, thereby reducing the time for data transmission and processing and improving the response speed and reliability of applications.

[0003] In related technologies, the testing of edge computing systems involves collecting data through roadside equipment such as cameras or radars, sending the collected data directly to the edge computing system for real-time processing, and then manually reviewing the processing results of the edge computing system to determine the performance of the edge computing system. This manual review method is time-consuming and labor-intensive, and has a relatively low efficiency. Summary of the invention

[0004] Embodiments of the present invention provide a testing method, system, device, data re-injection server and medium for an edge computing system, so as to overcome the problem of low testing efficiency of the edge computing system.

[0005] In a first aspect, the present invention provides a testing method for an edge computing system, which is applied to a data re-injection server, including:

[0006] storing the road data uploaded by the roadside collection device as the original road data, wherein the roadside collection device includes a camera and / or a radar;

[0007] Label the data information of each traffic participant in the original road data under the current test scenario, and use the labeling result as the standard data result of each traffic participant, wherein the traffic participant includes any one or more of pedestrians, vehicles, traffic signs and traffic lights, and the data information includes type, location and / or quantity;

[0008] Sending raw road data to the system under test;

[0009] Receiving a perception result sent by the system to be tested, wherein the perception result is data information about each traffic participant identified by the system to be tested from the original road data;

[0010] The difference value between the perception result and the standard data result is calculated to evaluate the performance of the system under test according to the difference value.

[0011] Optionally, a difference value between the perception result and the standard data result is calculated to evaluate the performance of the system under test according to the difference value, including:

[0012] Calculate the difference between the perception result and the standard data result. If the difference is within a preset value range, it is determined that the performance of the system under test meets the preset requirements.

[0013] If the difference value exceeds the preset value range, it is determined that the performance of the system under test does not meet the preset requirements.

[0014] Optionally, the current test scenario is to test the recognition accuracy of the system under test for the target traffic participant type;

[0015] Accordingly, the method provided by the embodiment of the present invention further includes:

[0016] After receiving the perception results sent by the system to be tested, the ratio of the number of target traffic participants to the total number of all traffic participants is calculated;

[0017] Accordingly, the difference between the perception result and the standard data result is calculated to evaluate the performance of the system under test according to the difference, including:

[0018] The difference value between the ratio value and the standard ratio value is calculated to evaluate the performance of the system under test according to the difference value.

[0019] Optionally, the current test scenario is to test the calculation accuracy of the vehicle position of the system under test; accordingly, the traffic participant is a vehicle, and accordingly, the standard data result is the real position vector of the i-th vehicle in the two-dimensional plane;

[0020] Accordingly, calculating the difference between the perception result and the standard data result includes:

[0021] The difference between the perception result and the standard data result is calculated according to the following formula:

[0022]

[0023] Among them, x i represents the measured position vector of the i-th vehicle in the two-dimensional plane calculated by the system under test, represents the true position vector of the i-th vehicle in the two-dimensional plane, and N represents the total number of vehicles.

[0024] Optionally, the current test scenario is to test the deviation of the time interval for the system to be tested to output the perception result;

[0025] Accordingly, the method provided by the embodiment of the present invention further includes:

[0026] According to the relationship between the receiving frequency of the sensing result and the standard receiving frequency within the set time period, determine the instantaneous error of the sensing result sent by the system under test;

[0027] Based on the preset time interval deviation algorithm, and according to the instantaneous error and the time interval and the total number of time intervals for the current data refill server to collect the perception results sent by the system under test within the set time period, the deviation of the time interval for the system under test to output the perception results is calculated, wherein the preset time interval deviation algorithm is expressed by the following formula:

[0028]

[0029] Among them, i represents the perception result received for the i-th time, t i represents the instantaneous error of the i-th output perception result of the system under test; τ0 represents the time interval for the current data feedback server to collect the perception results sent by the system under test; n represents the total number of time intervals;

[0030] Accordingly, the difference between the perception result and the standard data result is calculated to evaluate the performance of the system under test according to the difference, including:

[0031] The difference between the time deviation and the corresponding preset time threshold is calculated to evaluate the performance of the system under test according to the difference.

[0032] In a second aspect, an embodiment of the present invention further provides a test system for an edge computing system, including: a drive test collection device, a data re-injection server and a system to be tested, wherein:

[0033] The road test collection equipment is configured to collect road data and upload the collected road data as raw road data to the server, wherein the roadside collection equipment includes a camera and / or a radar;

[0034] The data re-injection server is configured to: store the road data uploaded by the roadside collection device as the original road data, and annotate the data information of each traffic participant in the original road data under the current measurement scene, and use the annotated result as the standard data result of each traffic participant, wherein the traffic participant includes any one or more of pedestrians, vehicles, traffic signs and traffic lights, and the data information includes type, location and / or quantity;

[0035] The system to be tested is configured to: identify data information about each traffic participant from the original road data, and send the data information as a perception result to a data feedback server;

[0036] The data re-injection server is configured to receive the perception result sent by the system to be tested, and calculate the difference value between the perception result and the standard data result, so as to evaluate the performance of the system to be tested according to the difference value.

[0037] Optionally, the data re-injection server and the system under test transmit video data via the real-time streaming protocol RTSP, and transmit radar data via the transmission control protocol TCP.

[0038] In a third aspect, an embodiment of the present invention further provides a test module for an edge computing system, including:

[0039] A data storage module, configured to store the original road data uploaded by the roadside collection device, wherein the roadside collection device includes a camera and / or a radar;

[0040] The data processing module is configured to annotate the data information of each traffic participant in the original road data under the current measurement scene, and use the annotated results as the standard data results of each traffic participant, wherein the traffic participants include any one or more of pedestrians, vehicles, traffic signs and traffic lights, and the data information includes type, location and / or quantity;

[0041] A data sending module, configured to send raw road data to the system to be tested;

[0042] A data receiving module is configured to receive a calculation result sent by the system to be tested, wherein the perception result is data information about each traffic participant identified by the system to be tested from the original road data;

[0043] The data comparison module is configured to calculate a difference value between the perception result and the standard data result, so as to evaluate the performance of the system to be tested according to the difference value.

[0044] Optionally, the data comparison module is specifically configured as follows:

[0045] Calculate the difference between the perception result and the standard data result. If the difference is within a preset value range, it is determined that the performance of the system under test meets the preset requirements.

[0046] If the difference value exceeds the preset value range, it is determined that the performance of the system under test does not meet the preset requirements.

[0047] Optionally, the current test scenario is to test the recognition accuracy of the system under test for the target traffic participant type;

[0048] Accordingly, the device provided by the embodiment of the present invention further includes:

[0049] A ratio value calculation module is configured to calculate the ratio of the number of target traffic participants to the total number of all traffic participants after receiving the perception result sent by the system to be tested;

[0050] Correspondingly, the data comparison module is specifically configured as follows:

[0051] A difference value between the ratio value and a standard ratio value is calculated to evaluate the recognition accuracy of the system to be tested for the target traffic participant type according to the difference value.

[0052] Optionally, the current test scenario is to test the calculation accuracy of the vehicle position by the system under test;

[0053] Accordingly, traffic participants are vehicles;

[0054] Correspondingly, the standard data result is the real position vector of the i-th vehicle in the two-dimensional plane;

[0055] Correspondingly, the data comparison module is specifically configured as follows:

[0056] The difference between the perception result and the standard data result is calculated according to the following formula to evaluate the calculation accuracy of the vehicle position by the system under test:

[0057]

[0058] Among them, x i represents the measured position vector of the i-th vehicle in the two-dimensional plane calculated by the system under test, represents the true position vector of the i-th vehicle in the two-dimensional plane, and N represents the total number of vehicles.

[0059] Optionally, the current test scenario is to test the deviation of the time interval for the system to be tested to output the perception result;

[0060] Accordingly, the device provided by the embodiment of the present invention further includes:

[0061] The instantaneous error determination module is configured to determine the instantaneous error of the sensing result sent by the system under test according to the magnitude relationship between the receiving frequency of the sensing result and the standard receiving frequency within a set time period;

[0062] The time interval deviation calculation module is configured to calculate the deviation of the time interval of the system to be tested outputting the perception results based on the preset time interval deviation algorithm and according to the instantaneous error and the time interval of the current data refill server collecting the perception results sent by the system to be tested within the set time period and the total number of the time intervals, wherein the preset time interval deviation algorithm is expressed by the following formula:

[0063]

[0064] Among them, i represents the perception result received for the i-th time, t i represents the instantaneous error of the i-th output of the perception result of the system under test; τ0 represents the time interval for the current data feedback server to collect the perception results sent by the system under test; n represents the total number of time intervals;

[0065] Correspondingly, the data comparison module is specifically configured as follows:

[0066] The difference between the time deviation and the corresponding preset time threshold is calculated to evaluate the deviation of the time interval of the system to be tested outputting the perception result according to the difference.

[0067] In a fourth aspect, an embodiment of the present invention further provides a data refill server, comprising:

[0068] A memory storing executable program code;

[0069] a processor coupled to the memory;

[0070] The processor calls the executable program code stored in the memory to execute the testing method for the edge computing system provided by any embodiment of the present invention.

[0071] In a fifth aspect, an embodiment of the present invention further provides a computer-readable storage medium on which a computer program is stored, and when the program is executed by a processor, the testing method for the edge computing system provided by any embodiment of the present invention is implemented.

[0072] According to the technical solution provided by the embodiment of the present invention, when testing the system to be tested, the current data re-injection server first stores the original road data collected by the roadside acquisition device, and sends the stored original road data to the system to be tested. The system to be tested will identify the original road data and send the perception result to the current re-injection server. The current re-injection server will compare the received perception result with the annotation result of the original road data, and evaluate the performance of the system to be tested based on the difference between the two. Compared with the method in the related art of directly sending the original road data to the system to be tested in real time through the roadside acquisition device for testing, the technical solution provided by this embodiment can realize the reuse of the original road data. Compared with the method in the related art of manually testing the system to be tested, the technical solution provided by this embodiment saves human resources and improves the efficiency and accuracy of the testing of the system to be tested. BRIEF DESCRIPTION OF THE DRAWINGS

[0073] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0074] Figure 1 A flowchart of a testing method for an edge computing system provided in Embodiment 1 of the present invention;

[0075] Figure 2 A structural block diagram of a test system for an edge computing system provided in Embodiment 2 of the present invention;

[0076] Figure 3 A structural block diagram of a testing device for an edge computing system provided in Embodiment 3 of the present invention;

[0077] Figure 4 A structural diagram of a data re-injection server provided in Embodiment 4 of the present invention. DETAILED DESCRIPTION

[0078] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0079] It should be noted that the terms "including" and "having" and any variations thereof in the embodiments of the present invention and the accompanying drawings are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device including a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products or devices.

[0080] The embodiments of the present invention disclose a test method, system, device, data re-injection server and medium for an edge computing system, which are described in detail below.

[0081] Embodiment 1

[0082] Figure 1 A flowchart of a testing method for an edge computing system provided in Embodiment 1 of the present invention, the method can be applied to a data re-injection server, which injects the collected actual vehicle data into the edge computing system to be tested for simulation and testing, that is, provides raw data for the edge computing system. In addition, the data re-injection server can compare the annotation results of the raw data with the measurement data obtained by the edge computing system, thereby realizing the evaluation and verification of the performance and algorithm of the edge computing system. The method provided in this embodiment can be executed by a testing device for an edge computing system, which can be implemented by software and / or hardware. Figure 1 As shown, the method provided in this embodiment specifically includes:

[0083] S110: storing the road data uploaded by the roadside collection device as original road data.

[0084] Among them, the roadside collection equipment includes cameras and / or radars and other equipment. The road data uploaded by the roadside collection equipment includes video data collected by cameras and / or point cloud data collected by radars, etc. The road data may include various traffic participants on the road, such as any one or more of pedestrians, vehicles, traffic signs, traffic lights and obstacles.

[0085] In this embodiment, after the roadside collection device uploads the collected road data to the current data re-injection server, the current data re-injection server will store the road data within a certain period of time to achieve information aggregation and unified scheduling.

[0086] In this embodiment, after the original road data is stored, the data re-injection server may pre-process the original road data, for example, the noise points in the camera or radar data may be removed by a filtering algorithm, wherein there are many filtering algorithms, such as Kalman filtering, Butterworth filtering, etc., and this embodiment does not specifically limit the filtering algorithm. In addition, for the case where the received camera or radar data is lost due to network problems, the pre-processing operation of the data re-injection server may include: data frame filling processing, for example, the missing data frame may be filled with the previous frame of valid data. In addition, since the sampling frequencies of the camera and the radar are inconsistent, in order to facilitate the system under test to effectively identify the camera data and the radar data, the data re-injection server pre-processes the original road data and also includes: adding a new timestamp to the camera data and the radar data to ensure the synchronization of the radar data and the camera data.

[0087] S120: label the data information of each traffic participant in the original road data under the current measurement scenario, and use the labeling result as the standard data result of each traffic participant.

[0088] As an optional implementation, the current data re-injection server can use an automatic labeling method to use a trained model or a pre-designed algorithm to identify and label the type, location, and quantity of each traffic participant in the original road data. After the labeling is completed, the accuracy of the labeling results can be verified by manual review. As another optional implementation, the current data re-injection server can use a manual labeling method to label the data information of each traffic participant in the current test scenario.

[0089] In this embodiment, after the labeling of the traffic participant data information is completed, the data information including the labeling results can be used as the standard data results of each traffic participant.

[0090] S130: Send the original road data to the system to be tested.

[0091] Among them, the system to be tested is an edge computing system. The edge computing system refers to an open platform that integrates network, computing, and application core capabilities on the side close to the source of objects or data. In this embodiment, the original road data is first stored and processed on the data re-injection server, and then sent to the system to be tested. Compared with the way that the roadside collection equipment directly sends the collected road data to the system to be tested, this embodiment is set up in this way to achieve the reuse of the original road data in different test scenarios, and by pre-processing the original road data on the data re-injection server, the test efficiency of the system to be tested can be improved.

[0092] In this embodiment, video data can be transmitted between the data feedback server and the system under test through the Real Time Streaming Protocol (RTSP), and radar data can be transmitted through the Transmission Control Protocol (TCP) or the User Datagram Protocol (UDP). This setting enables the data feedback server to be compatible with different systems under test, thereby meeting the testing requirements of different edge computing systems.

[0093] S140: Receive the perception result sent by the system to be tested.

[0094] In this embodiment, after receiving the original road data sent by the data feedback server, the system under test will use an internal model or algorithm to identify the original road data and obtain a perception result, which usually includes data information of each traffic participant.

[0095] S140, calculating a difference value between the perception result and the standard data result, so as to evaluate the performance of the system under test according to the difference value.

[0096] In this embodiment, the difference value between the perception result and the standard data result is calculated. If the difference value is within the preset numerical range, it is determined that the performance of the system to be tested meets the preset requirements. If the difference value exceeds the preset numerical range, it is determined that the performance of the system to be tested does not meet the preset requirements.

[0097] For example, the current test scenario is to test the recognition accuracy of the number of pedestrians on the road by the system under test. If the number of pedestrians in the annotation result of the original road data on the current data feedback server is 15, and the number of pedestrians recognized by the system under test from the original road data is 12, and the difference between the two exceeds the preset value range of 2, it means that the performance of the system under test does not meet the preset requirements.

[0098] For another example, the current test scenario is to test the recognition accuracy of the target traffic participant type by the system under test. In this embodiment, after receiving the perception result sent by the system under test, the current data re-injection server can calculate the proportion of the target traffic participant type to all traffic participant types, which can be specifically achieved through the following formula:

[0099]

[0100] Where R represents the ratio of target traffic participants to all traffic participants, TP represents the number of target traffic participants in the current test scenario, FN represents the number of non-target traffic participants in the current test field, and (TP+FN) represents the number of all traffic participant types in the current test field.

[0101] After obtaining the above-mentioned ratio value, by calculating the difference between the ratio value and the standard ratio value, the performance of the system under test can be evaluated according to the difference value. For example, if the ratio value is greater than or equal to the marked ratio value, it is determined that the recognition accuracy of the target traffic participant type by the system under test meets the preset requirements; if the ratio value is less than the marked ratio value, it is determined that the recognition accuracy of the target traffic participant type by the system under test meets the preset requirements.

[0102] For another example, in another test scenario, such as testing the calculation accuracy of the vehicle position of the system under test, the traffic participants are vehicles. Accordingly, the current data re-injection server can mark the real position vector of the i-th vehicle in the two-dimensional plane based on the original road data as the marking result. The system under test will obtain the measured position vector of the i-th vehicle in the two-dimensional plane based on the original road data as the perception result. The current data re-injection server will calculate the difference between the received perception result and the standard data result, so as to judge the performance of the system under test, which can be achieved through the following formula:

[0103]

[0104] Among them, x i represents the measured position vector of the i-th vehicle in the two-dimensional plane calculated by the system under test, represents the real position vector of the i-th vehicle in the two-dimensional plane, and N represents the total number of vehicles. If Ep≤E, it is determined that the performance of the system under test meets the preset requirements, and if Ep>E, it is determined that the performance of the system under test does not meet the preset requirements.

[0105] For another example, in the current test scenario of testing the deviation of the time interval of the output of the perception results of the system under test, the current data re-injection server can determine the instantaneous error of the data sent by the system under test according to the relationship between the receiving frequency of the perception results in the set time period and the standard receiving frequency. For example, the standard frequency of the system under test sending the perception results is 10Hz, that is, the perception results are sent every 100ms, that is, the current data re-injection server receives the perception results sent by the system under test every 100ms. However, due to system reasons, the system under test sends the perception results at intervals of 101ms, that is, the receiving frequency of the perception results in the set time period is (1 / 0.101)Hz. At this time, it can be determined that the instantaneous error of the data sent by the system under test is 1ms.

[0106] In this embodiment, the current data refill server can calculate the deviation of the time interval for the system to be tested to output the perception results based on the preset time interval deviation algorithm and according to the calculated instantaneous error and the time interval for the current data refill server to collect the perception results sent by the system to be tested within the set time period and the total number of the time intervals, wherein the preset time interval deviation algorithm is expressed by the following formula:

[0107]

[0108] Among them, D TDEV (nτ0) represents the deviation of the time interval of the output of the perception result of the system under test, i represents the perception result received for the i-th time, <.> represents the ensemble average, t i It represents the instantaneous error of the i-th output perception result of the system under test; τ0 represents the time interval for the current data feedback server to collect the perception results sent by the system under test; n represents the total number of time intervals.

[0109] After calculating the deviation of the time interval for the system to be tested to output the perception results, if the deviation is within the preset numerical range, it is determined that the deviation of the time interval for the system to be tested to output the perception results meets the preset requirements; if the deviation exceeds the preset numerical range, it is determined that the deviation of the time interval for the system to be tested to output the perception results does not meet the preset requirements.

[0110] In this embodiment, when testing the performance of the system to be tested, the current data re-injection server first stores the original road data collected by the roadside collection device, and sends the stored original road data to the system to be tested. The system to be tested will identify the original road data and send the perception results to the current re-injection server. The current re-injection server will compare the received perception results with the annotation results of the original road data, and evaluate the performance of the system to be tested based on the difference between the two. Compared with the method in the related art of directly sending the original road data to the system to be tested in real time through the roadside collection device for testing, the technical solution provided by this embodiment can realize the reuse of the original road data. Compared with the method in the related art of manually testing the system to be tested, the technical solution provided by this embodiment saves human resources and improves the efficiency and accuracy of the testing of the system to be tested.

[0111] Embodiment 2

[0112] Figure 2 A structural block diagram of a test system for an edge computing system provided in Embodiment 2 of the present invention, such as Figure 2 As shown, the test system includes: a drive test collection device 210, a data refill server 220 and a system to be tested 230, wherein:

[0113] The road test collection device 210 is configured to collect road data and upload the collected road data as raw road data to the server, wherein the roadside collection device includes a camera and / or a radar;

[0114] The data re-injection server 220 is configured to: store the road data uploaded by the roadside collection device as original road data, and mark the data information of each traffic participant in the original road data under the current test scenario, and use the marked results as the standard data results of each traffic participant, and send the original road data to the system to be tested.

[0115] Traffic participants include any one or more of pedestrians, vehicles, traffic signs and traffic lights, and data information includes type, location and / or quantity.

[0116] The system to be tested 230 is configured to: identify data information about each traffic participant from the original road data, and send the data information as a perception result to the data feedback server;

[0117] The data refill server 220 is configured to receive the sensing result sent by the system under test, and calculate the difference between the sensing result and the standard data result, so as to evaluate the performance of the system under test according to the difference.

[0118] The data re-injection server and the system to be tested communicate via the real-time streaming protocol RTSP, so that different systems to be tested can be compatible, so as to provide test data for different systems to be tested. In this embodiment, the processing process of the original road data by the data re-injection server and the device to be tested can refer to the description of the above embodiment, which will not be repeated here.

[0119] In this embodiment, when testing the system to be tested, the current data re-injection server first stores the original road data collected by the roadside collection device, and sends the stored original road data to the system to be tested. The system to be tested will identify the original road data and send the perception results to the current re-injection server. The current re-injection server will compare the received perception results with the annotation results of the original road data, and evaluate the performance of the system to be tested based on the difference between the two. Compared with the method in the related art of directly sending the original road data to the system to be tested in real time through the roadside collection device for testing, the technical solution provided by this embodiment can realize the reuse of the original road data. Compared with the method in the related art of manually testing the system to be tested, the technical solution provided by this embodiment saves human resources and improves the efficiency and accuracy of the testing of the system to be tested.

[0120] Embodiment 3

[0121] Figure 3 A structural block diagram of a test module for an edge computing system provided in Embodiment 3 of the present invention, the test module includes: a data storage module 310, a data processing module 320, a data sending module 330, a data receiving module 340 and a data comparison module 350, wherein:

[0122] The data storage module 310 is configured to store the original road data uploaded by the roadside collection device, wherein the roadside collection device includes a camera and / or a radar;

[0123] The data processing module 320 is configured to: annotate the data information of each traffic participant in the original road data under the current measurement scene, and use the annotated results as the standard data results of each traffic participant, wherein the traffic participants include any one or more of pedestrians, vehicles, traffic signs and traffic lights, and the data information includes type, location and / or quantity;

[0124] The data sending module 330 is configured to send the original road data to the system to be tested;

[0125] The data receiving module 340 is configured to receive the calculation result sent by the system to be tested, where the perception result is the data information about each traffic participant identified by the system to be tested from the original road data;

[0126] The data comparison module 350 is configured to calculate a difference value between the perception result and the standard data result, so as to evaluate the performance of the system to be tested according to the difference value.

[0127] Optionally, the data comparison module is specifically configured as follows:

[0128] Calculate the difference between the perception result and the standard data result. If the difference is within a preset value range, it is determined that the performance of the system under test meets the preset requirements.

[0129] If the difference value exceeds the preset value range, it is determined that the performance of the system under test does not meet the preset requirements.

[0130] Optionally, the current test scenario is to test the recognition accuracy of the system under test for the target traffic participant type;

[0131] Accordingly, the device provided by the embodiment of the present invention further includes:

[0132] A ratio value calculation module is configured to calculate the ratio of the number of target traffic participants to the total number of all traffic participants after receiving the perception result sent by the system to be tested;

[0133] Correspondingly, the data comparison module is specifically configured as follows:

[0134] A difference value between the ratio value and a standard ratio value is calculated to evaluate the recognition accuracy of the system to be tested for the target traffic participant type according to the difference value.

[0135] Optionally, the current test scenario is to test the calculation accuracy of the vehicle position by the system under test;

[0136] Accordingly, traffic participants are vehicles;

[0137] Correspondingly, the standard data result is the real position vector of the i-th vehicle in the two-dimensional plane;

[0138] Correspondingly, the data comparison module is specifically configured as follows:

[0139] The difference between the perception result and the standard data result is calculated according to the following formula to evaluate the calculation accuracy of the vehicle position by the system under test:

[0140]

[0141] Among them, x i represents the measured position vector of the i-th vehicle in the two-dimensional plane calculated by the system under test, represents the true position vector of the i-th vehicle in the two-dimensional plane, and N represents the total number of vehicles.

[0142] Optionally, the current test scenario is to test the deviation of the time interval for the system to be tested to output the perception result;

[0143] Accordingly, the device provided by the embodiment of the present invention further includes:

[0144] The instantaneous error determination module is configured to determine the instantaneous error of the sensing result sent by the system under test according to the magnitude relationship between the receiving frequency of the sensing result and the standard receiving frequency within a set time period;

[0145] The time interval deviation calculation module is configured to calculate the deviation of the time interval of the system to be tested outputting the perception results based on the preset time interval deviation algorithm and according to the instantaneous error and the time interval of the current data refill server collecting the perception results sent by the system to be tested within the set time period and the total number of the time intervals, wherein the preset time interval deviation algorithm is expressed by the following formula:

[0146]

[0147] Among them, i represents the perception result received for the i-th time, t i represents the instantaneous error of the i-th output of the perception result of the system under test; τ0 represents the time interval for the current data feedback server to collect the perception result sent by the system under test; n represents the total number of the time intervals;

[0148] Correspondingly, the data comparison module is specifically configured as follows:

[0149] The difference between the time deviation and the corresponding preset time threshold is calculated to evaluate the deviation of the time interval of the system to be tested outputting the perception result according to the difference.

[0150] Embodiment 4

[0151] See also Figure 4 , Figure 4 This is a schematic diagram of the structure of a data refill server provided by Embodiment 4 of the present invention. Figure 4 As shown, the data refill server may include:

[0152] A memory 701 storing executable program codes;

[0153] a processor 702 coupled to the memory 701;

[0154] The processor 702 calls the executable program code stored in the memory 701 to execute the testing method for the edge computing system provided by any embodiment of the present invention.

[0155] An embodiment of the present invention discloses a computer-readable storage medium storing a computer program, wherein the computer program enables a computer to execute a testing method for an edge computing system provided by any embodiment of the present invention.

[0156] In various embodiments of the present invention, it should be understood that the size of the serial numbers of the above-mentioned processes does not mean the necessary order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0157] In the embodiments provided by the present invention, it should be understood that "B corresponding to A" means that B is associated with A, and B can be determined according to A. However, it should also be understood that determining B according to A does not mean determining B only according to A, and B can also be determined according to A and / or other information.

[0158] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0159] If the above integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-accessible memory. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product, which is stored in a memory and includes several requests for a computer device (which can be a personal computer, a server or a network device, etc., specifically a processor in a computer device) to perform some or all of the steps of the above methods of various embodiments of the present invention.

[0160] Those skilled in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium, and the storage medium includes a read-only memory (ROM), a random access memory (RAM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), a one-time programmable read-only memory (OTPROM), an electronically erasable rewritable read-only memory (EEPROM), a compact disc (CD-ROM) or other optical disc storage, magnetic disk storage, magnetic tape storage, or any other computer-readable medium that can be used to carry or store data.

[0161] Those skilled in the art can understand that the accompanying drawings are only schematic diagrams of an embodiment, and the modules or processes in the accompanying drawings are not necessarily required to implement the present invention.

[0162] Those skilled in the art can understand that the modules in the device in the embodiment can be distributed in the device in the embodiment according to the description of the embodiment, or can be changed accordingly and located in one or more devices different from the embodiment. The modules in the above embodiment can be combined into one module, or can be further divided into multiple sub-modules.

[0163] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A testing method for an edge computing system, applied to a data re-injection server, characterized in that: include: storing the road data uploaded by the roadside collection device as the original road data, wherein the roadside collection device includes a camera and / or a radar; Annotate the data information of each traffic participant in the original road data under the current test scene, and use the annotation result as the standard data result of each traffic participant, wherein the traffic participant includes any one or more of pedestrians, vehicles, traffic signs and traffic lights, and the data information includes type, location and / or quantity; Sending the original road data to the system to be tested; Receiving a perception result sent by the system to be tested, wherein the perception result is data information about each traffic participant identified by the system to be tested from the original road data; A difference value between the perception result and the standard data result is calculated to evaluate the performance of the system under test according to the difference value.

2. The method according to claim 1, characterized in that: The calculating the difference between the perception result and the standard data result to evaluate the performance of the system under test according to the difference includes: Calculating a difference between the perception result and the standard data result, and if the difference is within a preset value range, determining that the performance of the system to be tested meets preset requirements; If the difference value exceeds a preset value range, it is determined that the performance of the system to be tested does not meet preset requirements.

3. The method according to claim 1 or 2, characterized in that: The current test scenario is to test the recognition accuracy of the system under test on the target traffic participant type; Accordingly, the method further includes: After receiving the perception result sent by the system to be tested, calculating the ratio of the number of target traffic participants to the total number of all traffic participants; Accordingly, calculating a difference value between the perception result and the standard data result to evaluate the performance of the system under test according to the difference value includes: A difference value between the ratio value and a standard ratio value is calculated to evaluate the recognition accuracy of the system to be tested for the target traffic participant type according to the difference value.

4. The method according to claim 1 or 2, characterized in that: The current test scenario is to test the calculation accuracy of the vehicle position of the system under test; accordingly, the traffic participant is a vehicle, and accordingly, the standard data result is the real position vector of the i-th vehicle in the two-dimensional plane; Accordingly, the calculating the difference value between the perception result and the standard data result to evaluate the performance of the system under test according to the difference value includes: The difference between the perception result and the standard data result is calculated according to the following formula, so as to evaluate the calculation accuracy of the vehicle position by the system under test according to the difference: Among them, x i represents the measured position vector of the i-th vehicle in the two-dimensional plane calculated by the system under test, represents the true position vector of the i-th vehicle in the two-dimensional plane, and N represents the total number of vehicles.

5. The method according to claim 1 or 2, characterized in that: The current test scenario is to test the deviation of the time interval for outputting the perception result of the system under test; Accordingly, the method further includes: Determine the instantaneous error of the system to be tested sending the perception result according to the magnitude relationship between the receiving frequency of the perception result and the standard receiving frequency within the set time period; Based on the preset time interval deviation algorithm, and according to the instantaneous error and the time interval for the current data refill server to collect the perception results sent by the system under test within the set time period and the total number of the time intervals, the deviation of the time interval for the system under test to output the perception results is calculated, wherein the preset time interval deviation algorithm is expressed by the following formula: Among them, i represents the perception result received for the i-th time, t i represents the instantaneous error of the i-th output of the perception result of the system under test; τ0 represents the time interval for the current data feedback server to collect the perception result sent by the system under test; n represents the total number of the time intervals; Accordingly, the calculating the difference value between the perception result and the standard data result to evaluate the performance of the system under test according to the difference value includes: A difference value between the time deviation and a corresponding preset time threshold is calculated to evaluate the deviation of the time interval at which the system to be tested outputs the perception result according to the difference value.

6. A testing system for an edge computing system, characterized in that: include: Drive test collection equipment, data re-injection server and system under test, including: The road test collection device is configured to collect road data and upload the collected road data as raw road data to the server, wherein the roadside collection device includes a camera and / or a radar; The data re-injection server is configured to: store the road data uploaded by the roadside collection device as the original road data, and annotate the data information of each traffic participant in the original road data under the current test scene, and use the annotated result as the standard data result of each traffic participant, and send the original road data to the system to be tested, wherein the traffic participants include any one or more of pedestrians, vehicles, traffic signs and traffic lights, and the data information includes type, location and / or quantity; The system to be tested is configured to: identify data information about each traffic participant from the original road data, and send the data information as a perception result to the data feedback server; The data re-injection server is configured to receive the perception result sent by the system under test, and calculate the difference value between the perception result and the standard data result, so as to evaluate the performance of the system under test according to the difference value.

7. The test system according to claim 6, characterized in that: The data re-injection server transmits video data to the system under test via the real-time streaming protocol RTSP, and transmits radar data via the transmission control protocol TCP.

8. A test module for an edge computing system, characterized in that: include: A data storage module, configured to store the original road data uploaded by the roadside collection device, wherein the roadside collection device includes a camera and / or a radar; A data processing module is configured to annotate data information of each traffic participant in the original road data under the current measurement scene, and use the annotated result as the standard data result of each traffic participant, wherein the traffic participant includes any one or more of pedestrians, vehicles, traffic signs and traffic lights, and the data information includes type, location and / or quantity; A data sending module, configured to send the original road data to the system to be tested; A data receiving module is configured to receive a calculation result sent by the system to be tested, wherein the perception result is data information about each traffic participant identified by the system to be tested from the original road data; The data comparison module is configured to calculate a difference value between the perception result and the standard data result, so as to evaluate the performance of the system to be tested according to the difference value.

9. A server, characterized in that: The server comprises: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the testing method for the edge computing system as described in any one of claims 1-5.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, it implements the testing method for the edge computing system as described in any one of claims 1-5.

Images