Real-time index testing system and method for target recognition device

Abstract

The invention discloses a real-time index test system for a target recognition device. The real-time index test system comprises a signal generation device A1, a signal interface A2, a data interface A4, a clock system A5 and a comparison system A6. Through clock synchronization and clock triggering of corresponding interfaces of an input end and an output end and a high-precision clock system, a real-time index of a target recognition device is accurately obtained, signal input can be dynamically adjusted according to a target recognition result based on a feedback network composed of a target signal generation device and a comparison system, and the target recognition accuracy is improved. And the performance boundary of the target recognition device is continuously approached.

Description

Technical Field

[0001] The present invention relates to a testing technology for artificial intelligence algorithms, and in particular to a testing method for real-time indicators of intelligent recognition algorithms used in weapon equipment, intelligent networked vehicles, and smart homes. Background Art

[0002] The rapid development of artificial intelligence technology has brought many conveniences to people's lives. In particular, with the widespread application of deep learning algorithms and machine learning algorithms, applications such as intelligent recommendation, intelligent recognition, and intelligent voice assistants have brought great convenience to people's lives. It is also important to note that in some special scenarios and fields, the application of artificial intelligence technology will also bring certain security risks, and even have adverse effects on national defense and people's life safety. For example, target recognition devices used in scenarios such as weapons and equipment, intelligent networked vehicles, and smart cities. In these scenarios, the recognition results of targets are often used for the guidance and detonation of weapons and equipment, the braking of self-driving cars, etc. Failure to take timely action at critical moments or making wrong operations will bring serious consequences. This places high demands on the efficiency of target recognition devices and recognition algorithms, and requires scientific and rigorous means to test and evaluate them.

[0003] The core of the target recognition device described in the above scenario is the training and application of classification recognition algorithms in various scenarios. The target recognition media include three-dimensional point cloud data, voice, microwave signals, etc. The efficiency of the target recognition device and the recognition algorithm is reflected by the real-time index. At present, the evaluation of the recognition algorithm is mainly carried out in a pure software way. The data of the public or private media library is called and handed over to the recognition algorithm for recognition, and the recognition result is given for comparison with the reference result. The real-time performance of the recognition algorithm is evaluated by calculating the time interval from the recognition algorithm reading the media library data to outputting the recognition result. This method has two disadvantages. First, the data reading time replaces the actual signal processing speed. In view of the development of current storage technology, the data reading speed can be very fast, but the actual signal processing process is often limited by the signal processing link; second, the real-time performance of the signal recognition algorithm is not only affected by the degree of optimization of its own algorithm, but also by the complexity of the media library data. Once the open source database or private database is established, the editability of its parameters is poor, and it is not possible to obtain the performance boundary of the target recognition device and the target recognition algorithm well, nor can it be well evaluated. The impact of parameter changes in target data on the real-time performance indicators of the target recognition algorithm. Summary of the invention

[0004] The object of the present invention is to provide a target recognition device real-time index test system, characterized in that it includes: a signal generating device A1, a signal interface A2, a data interface A4, a clock system A5, and a comparison system A6; wherein,

[0005] The signal generating device A1 is used to generate a target analog signal Y(t) for target identification;

[0006] The signal interface A2 is used to detect whether the target analog signal Y(t) generated by the signal generating device A1 reaches the determination threshold, and at the same time sends the first clock trigger signal to the clock system A5. At this time, the clock system A5 records the time as T0;

[0007] The data interface A4 transmits the recognition result R of the target analog signal Y(t) to the target recognition device A3. D The information converted into a specific format is forwarded to the comparison system A6; after receiving the recognition result R D At the same time, the second clock trigger signal is generated to the clock system A5. At this time, the clock system A5 records the time as T1. The definition of the real-time index of the target recognition device to be tested is shown in formula (1):

[0008] Index_T=T1-T0 (1)

[0009] The unit of Index_T is generally ms.

[0010] The clock system A5 provides a reference clock for the signal interface A2 and the data interface A4 to operate; receives the first and second trigger signals sent by the signal interface A2 and the data interface A4, and records the current time information; calculates the real-time index Index_T according to the time interval between the first and second trigger signals, and sends it to the comparison system A6;

[0011] The comparison system A6 controls the signal generating device A1 to generate a target analog signal Y(t) of a specified form by sending a control instruction; receives the real-time index Index_T sent by the clock system A5 and the recognition result R sent by the data interface A4; D , the real-time index Index_T and the recognition result R are combined by time marking points D Pairing; the recognition result R D By comparing with the theoretical model, the recognition accuracy index Index_E of the target recognition device to be tested is obtained. The real-time index Index_T and the recognition accuracy index Index_E constitute an index pair.

[0012] Preferably, the signal generating device A1 includes a three-dimensional point cloud data simulation device, a radar target signal source, and a radar interference signal source, which are respectively used to generate three-dimensional point cloud data for three-dimensional information recognition, radar guidance signals for radar target recognition, and interference signals; wherein, the target signal generated by the three-dimensional point cloud data simulation device is input in the form of a multi-channel three-dimensional lidar echo signal, and the target signals generated by the radar target signal source and the radar interference signal source are input in the form of radio frequency signals.

[0013] Preferably, the signal interface A2 comprises a switch array and tooling controlled by a microprocessor;

[0014] When the microprocessor detects that the signal output by the signal generating device A1 reaches the determination threshold, it controls the corresponding channel of the switch array to be turned on and sends a clock trigger signal to the clock system A5. The clock system A5 records the time at this time as T0';

[0015] The clock trigger signal of the signal interface A2 is anchored on the switch array on control signal.

[0016] Preferably, the clock trigger signal of the data interface A4 is anchored on the data reception signal.

[0017] Preferably, the comparison system A6 includes a digital communication interface, a central processing unit, a storage system, and a display system; wherein the digital communication interface is used to control A1 to generate the required target analog signal Y(t) and receive the recognition result R output by A4. D ;

[0018] The CPU recognizes the result R D The algorithm compares with the theoretical model to generate the recognition accuracy index Index_E, and the real-time index Index_T is obtained through the time mark point analysis algorithm;

[0019] The storage system stores the test data and test results in a structured manner, and the display system monitors and displays the test process and test results.

[0020] The present invention also provides a test method applicable to a real-time index test system of a target recognition device, characterized in that it comprises the steps of:

[0021] S1: The comparison system A6 controls the signal generating device A1 to generate a target analog signal Y(t) with parameters M(C1, C2...Cn) according to the test requirements, and outputs it to the target identification device A3 through the signal interface A2.

[0022] S2: The clock system A5 records the signal input time T0 and the result output time T1 according to the clock trigger signal of the signal interface A2 and the data interface A4;

[0023] S3: The comparison system A6 calculates the real-time index and the recognition accuracy index to obtain the index pair N(i,j);

[0024] S4: Comparison system A6 determines whether the parameters are completely traversed; if not, proceed to S5; if yes, proceed to S6;

[0025] S5: Comparison system A6 adjusts M(C1, C2...Cn) according to the trend of N(i, j)~M(C1, C2...Cn), and returns to S1;

[0026] S5 is based on N(i,j)~M(C1,C2…C n )Trend adjustment M(C1,C2...C n ), we should follow N(i,j) along with M(C1,C2…C n ) adjust the parameters in the direction of deterioration to obtain the performance boundary of the target recognition device and the target recognition algorithm to be tested;

[0027] S6: The comparison system A6 draws the performance boundary of the target recognition device A3 to be tested along with M(C1, C2...Cn).

[0028] Preferably, for a specific target identification device A3 to be tested, a typical number of target analog signals Y(t) of 100 to 100,000 are preset and successively input into the target identification device A3 to obtain an index pair N equal to the number of input target analog signals. m (i,j),m=1,2L M.

[0029] Preferably, the signal generating device A1, the comparison system A6, and the data interface A4 form a feedback network, which is led by the comparison system A6 and compares N according to the obtained indicators. m (i,j),m=1,2L M, continuously adjust the parameters of the target signal, and monitor the index for N' m (i,j),m=1,2L changes in M.

[0030] Preferably, the change of a parameter is adjusted until one or both of the real-time index or the recognition accuracy index exceeds the threshold. The curve of the index pair N(i,j) changing with the current parameter can obtain the influence trend and influence boundary of the current parameter on the target recognition device.

[0031] Preferably, the S2 includes:

[0032] S21: The signal interface A2 receives the target signal output by the signal generating device and opens the corresponding signal path as needed;

[0033] S22: The signal interface A2 opens the signal path and outputs the clock trigger signal at the same time. The clock system A5 records the current time, which is T0, and starts timing.

[0034] S23: the data interface A4 receives the output signal of the target identification device A3 and outputs a clock trigger signal, and the clock system A5 records the current time, which is T1, and the timing ends;

[0035] S24: The clock system A5 calculates the real-time index according to formula (2), and the comparison system A6 compares the target recognition result with the standard model to obtain the recognition accuracy index.

[0036] The beneficial effects of the present invention include: the present invention introduces a real-time index testing system for a target recognition device and a target recognition algorithm, which accurately obtains the real-time index of the target recognition device through clock synchronization and clock triggering of the corresponding interface at the input and output ends with the high-precision clock system, and is based on a target signal generating device and a feedback network composed of a comparison system, and can dynamically adjust the signal input according to the target recognition result, thereby continuously approaching the performance boundary of the target recognition device. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 .Description of real-time indicators of target recognition device;

[0038] Figure 2 . Composition diagram of the target recognition device real-time index test system;

[0039] Figure 3 . Target recognition device real-time index test flow chart. DETAILED DESCRIPTION

[0040] The technical solutions in the embodiments of the present invention are described clearly and completely below. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0041] The present invention provides a target recognition device real-time index test system, comprising: a signal generating device A1, a signal interface A2, a data interface A4, a clock system A5, and a comparison system A6; wherein:

[0042] The signal generating device A1 is used to generate a target analog signal Y(t) for target identification;

[0043] The signal interface A2 is used to detect whether the target analog signal Y(t) generated by the signal generating device A1 reaches the determination threshold, and at the same time sends the first clock trigger signal to the clock system A5. At this time, the clock system A5 records the time as T0;

[0044] The data interface A4 transmits the recognition result R of the target analog signal Y(t) to the target recognition device A3. D The information converted into a specific format is forwarded to the comparison system A6; after receiving the recognition result R D At the same time, the second clock trigger signal is generated to the clock system A5, and the clock system A5 records the time as T1;

[0045] The clock system A5 provides a reference clock for the signal interface A2 and the data interface A4 to operate; receives the first and second trigger signals sent by the signal interface A2 and the data interface A4, and records the current time information; calculates the real-time index Index_T according to the time interval between the first and second trigger signals, and sends it to the comparison system A6;

[0046] The comparison system A6 controls the signal generating device A1 to generate a target analog signal Y(t) of a specified form by sending a control instruction; receives the real-time index Index_T sent by the clock system A5 and the recognition result R sent by the data interface A4; D , the real-time index Index_T and the recognition result R are combined by time marking points D Pairing; the recognition result R D By comparing with the theoretical model, the recognition accuracy index Index_E of the target recognition device to be tested is obtained. The real-time index Index_T and the recognition accuracy index Index_E constitute an index pair.

[0047] According to one embodiment of the present invention, the signal generating device A1 includes a three-dimensional point cloud data simulation device, a radar target signal source, and a radar interference signal source, which are respectively used to generate three-dimensional point cloud data for three-dimensional information recognition, radar guidance signals for radar target recognition, and interference signals; wherein the target signal generated by the three-dimensional point cloud data simulation device is input in the form of a multi-channel three-dimensional lidar echo signal, and the target signals generated by the radar target signal source and the radar interference signal source are input in the form of radio frequency signals.

[0048] According to one embodiment of the present invention, the signal interface A2 comprises a switch array and tooling controlled by a microprocessor;

[0049] When the microprocessor detects that the signal output by the signal generating device A1 reaches the determination threshold, it controls the corresponding channel of the switch array to be turned on and sends a clock trigger signal to the clock system A5. The clock system A5 records the time at this time as T0';

[0050] The clock trigger signal of the signal interface A2 is anchored on the switch array on control signal.

[0051] According to an embodiment of the present invention, the clock trigger signal of the data interface A4 is anchored on the data receiving signal.

[0052] According to one embodiment of the present invention, the comparison system A6 includes a digital communication interface, a central processing unit, a storage system, and a display system; wherein the digital communication interface is used to control A1 to generate the required target analog signal Y(t) and receive the recognition result R output by A4.D ;

[0053] The CPU recognizes the result R D The algorithm compares with the theoretical model to generate the recognition accuracy index Index_E, and the real-time index Index_T is obtained through the time mark point analysis algorithm;

[0054] The storage system stores the test data and test results in a structured manner, and the display system monitors and displays the test process and test results.

[0055] The present invention also proposes a test method for a target recognition device real-time index test system, which is characterized by comprising the steps of:

[0056] S1: The comparison system A6 controls the signal generating device A1 to generate a target analog signal Y(t) with parameters M(C1, C2...Cn) according to the test requirements, and outputs it to the target identification device A3 through the signal interface A2.

[0057] S2: The clock system A5 records the signal input time T0 and the result output time T1 according to the clock trigger signal of the signal interface A2 and the data interface A4;

[0058] S3: The comparison system A6 calculates the real-time index and the recognition accuracy index to obtain the index pair N(i,j);

[0059] S4: Comparison system A6 determines whether the parameters are completely traversed; if not, proceed to S5; if yes, proceed to S6;

[0060] S5: Comparison system A6 adjusts M(C1, C2...Cn) according to the trend of N(i, j)~M(C1, C2...Cn), and returns to S1;

[0061] S5 is based on N(i,j)~M(C1,C2…C n )Trend adjustment M(C1,C2...C n ), we should follow N(i,j) along with M(C1,C2…C n ) adjust the parameters in the direction of deterioration to obtain the performance boundary of the target recognition device and the target recognition algorithm to be tested;

[0062] S6: The comparison system A6 draws the performance boundary of the target recognition device A3 to be tested along with M(C1, C2...Cn).

[0063] According to one embodiment of the present invention, for a specific target identification device A3 to be tested, a typical number of target analog signals Y(t) of 100 to 100,000 are preset and successively input into the target identification device A3 to obtain an index pair N equal to the number of input target analog signals. m(i,j),m=1,2L M.

[0064] According to an embodiment of the present invention, the signal generating device A1, the comparison system A6, and the data interface A4 form a feedback network, which is dominated by the comparison system A6. m (i,j),m=1,2L M, continuously adjust the parameters of the target signal, and monitor the index for N' m (i,j),m=1,2L changes in M.

[0065] According to one embodiment of the present invention, a parameter change is adjusted until one or both of the real-time index or the recognition accuracy index exceed a threshold value. The curve of the index pair N(i, j) changing with the current parameter can obtain the influence trend and influence boundary of the current parameter on the target recognition device.

[0066] According to one embodiment of the present invention, the S2 includes:

[0067] S21: The signal interface A2 receives the target signal output by the signal generating device and opens the corresponding signal path as needed;

[0068] S22: The signal interface A2 opens the signal path and outputs the clock trigger signal at the same time. The clock system A5 records the current time, which is T0, and starts timing.

[0069] S23: the data interface A4 receives the output signal of the target identification device A3 and outputs a clock trigger signal, and the clock system A5 records the current time, which is T1, and the timing ends;

[0070] S24: The clock system A5 calculates the real-time index according to formula (2), and the comparison system A6 compares the target recognition result with the standard model to obtain the recognition accuracy index.

[0071] In summary, the innovations of the present invention include:

[0072] (1) High-precision measurement of the target recognition real-time performance of the target recognition device not only considers the performance limitations of the target recognition algorithm, but also considers the performance limitations of the target recognition device hardware.

[0073] (2) The input target signal generated based on hardware has rich editability, which can comprehensively consider the impact of the change of target input signal parameters on the real-time index and target recognition accuracy index of the target recognition device to be tested, and obtain a more comprehensive performance boundary of the target recognition device.

[0074] Although the content of the present invention has been described in detail through the above preferred embodiments, it should be appreciated that the above description should not be considered as a limitation of the present invention. After reading the above content, it will be apparent to those skilled in the art that various modifications and substitutions of the present invention will occur. Therefore, the protection scope of the present invention should be limited by the appended claims.

Claims

1. A target recognition device real-time index testing system, characterized in that: include: Signal generating device A1, signal interface A2, data interface A4, clock system A5, comparison system A6; wherein, The signal generating device A1 is used to generate a target analog signal Y(t) for target identification; The signal interface A2 is used to detect whether the target analog signal Y(t) generated by the signal generating device A1 reaches the determination threshold, and at the same time sends the first clock trigger signal to the clock system A5. At this time, the clock system A5 records the time as T0; The data interface A4 transmits the recognition result R of the target analog signal Y(t) to the target recognition device A3. D The information converted into a specific format is forwarded to the comparison system A6; after receiving the recognition result R D At the same time, the second clock trigger signal is generated to the clock system A5, and the clock system A5 records the time as T1; The clock system A5 provides a reference clock for the signal interface A2 and the data interface A4 to operate; receives the first and second trigger signals sent by the signal interface A2 and the data interface A4, and records the current time information; calculates the real-time index Index_T according to the time interval between the first and second trigger signals, and sends it to the comparison system A6; The comparison system A6 controls the signal generating device A1 to generate a target analog signal Y(t) of a specified form by sending a control instruction; receives the real-time index Index_T sent by the clock system A5 and the recognition result R sent by the data interface A4; D , the real-time index Index_T and the recognition result R are combined by time marking points D Pairing; the recognition result R D By comparing with the theoretical model, the recognition accuracy index Index_E of the target recognition device to be tested is obtained. The real-time index Index_T and the recognition accuracy index Index_E constitute an index pair.

2. The target recognition device real-time index testing system according to claim 1, characterized in that: The signal generating device A1 includes a three-dimensional point cloud data simulation device, a radar target signal source, and a radar interference signal source, which are respectively used to generate three-dimensional point cloud data for three-dimensional information recognition, radar guidance signals for radar target recognition, and interference signals; wherein, the target signal generated by the three-dimensional point cloud data simulation device is input in the form of a multi-channel three-dimensional laser radar echo signal, and the target signals generated by the radar target signal source and the radar interference signal source are input in the form of radio frequency signals.

3. The target recognition device real-time index testing system according to claim 2, characterized in that: The signal interface A2 includes a switch array and tooling controlled by a microprocessor; When the microprocessor detects that the signal output by the signal generating device A1 reaches the determination threshold, it controls the corresponding channel of the switch array to be turned on and sends a clock trigger signal to the clock system A5. The clock system A5 records the time at this time as T0'; The clock trigger signal of the signal interface A2 is anchored on the switch array on control signal.

4. The target recognition device real-time index testing system according to claim 3, characterized in that: The clock trigger signal of the data interface A4 is anchored on the data receiving signal.

5. The target recognition device real-time index testing system according to claim 1, characterized in that: The comparison system A6 includes a digital communication interface, a central processing unit, a storage system, and a display system; wherein the digital communication interface is used to control A1 to generate the required target analog signal Y(t) and receive the recognition result R output by A4. D ; The CPU recognizes the result R D The algorithm compares with the theoretical model to generate the recognition accuracy index Index_E, and the real-time index Index_T is obtained through the time mark point analysis algorithm; The storage system stores the test data and test results in a structured manner, and the display system monitors and displays the test process and test results.

6. A test method for a real-time index test system of a target recognition device applicable to any one of items 1 to 5, characterized in that: Includes steps: S1: The comparison system A6 controls the signal generating device A1 to generate a target analog signal Y(t) with parameters M(C1, C2...Cn) according to the test requirements, and outputs it to the target identification device A3 through the signal interface A2. S2: The clock system A5 records the signal input time T0 and the result output time T1 according to the clock trigger signal of the signal interface A2 and the data interface A4; S3: The comparison system A6 calculates the real-time index and the recognition accuracy index to obtain the index pair N(i,j); S4: Comparison system A6 determines whether the parameters are completely traversed; if not, proceed to S5; if yes, proceed to S6; S5: Comparison system A6 adjusts M(C1, C2...Cn) according to the trend of N(i, j)~M(C1, C2...Cn), and returns to S1; S5 is based on N(i,j)~M(C1,C2…C n )Trend adjustment M(C1,C2...C n ), we should follow N(i,j) along with M(C1,C2…C n ) adjust the parameters in the direction of deterioration to obtain the performance boundary of the target recognition device and the target recognition algorithm to be tested; S6: The comparison system A6 draws the performance boundary of the target recognition device A3 to be tested along with M(C1, C2...Cn).

7. The target recognition device real-time index testing method according to claim 6 is characterized in that: For a specific target recognition device A3 to be tested, a typical number of target analog signals Y(t) of 100 to 100,000 are preset and successively input into the target recognition device A3 to obtain an index pair N equal to the number of input target analog signals. m (i,j),m=1,2L M.

8. The target recognition device real-time index testing method according to claim 7 is characterized in that: The signal generating device A1, the comparison system A6, and the data interface A4 form a feedback network, which is led by the comparison system A6 and compares N according to the obtained indicators. m (i,j),m=1,2L M, continuously adjust the parameters of the target signal, and monitor the index for N' m (i,j),m=1,2L changes in M.

9. The target recognition device real-time index testing method according to claim 8 is characterized in that: By adjusting the change of a parameter until one or both of the real-time index or the recognition accuracy index exceeds the threshold, the curve of the index pair N(i,j) changing with the current parameter can obtain the influence trend and influence boundary of the current parameter on the target recognition device.

10. The target recognition device real-time index testing method according to claim 9 is characterized in that: The S2 includes: S21: The signal interface A2 receives the target signal output by the signal generating device and opens the corresponding signal path as needed; S22: The signal interface A2 opens the signal path and outputs the clock trigger signal at the same time. The clock system A5 records the current time, which is T0, and starts timing. S23: the data interface A4 receives the output signal of the target identification device A3 and outputs a clock trigger signal, and the clock system A5 records the current time, which is T1, and the timing ends; S24: The clock system A5 calculates the real-time index according to formula (2), and the comparison system A6 compares the target recognition result with the standard model to obtain the recognition accuracy index.

Images