Processor computing power resource measurement method, device, electronic device and storage medium

Abstract

The present application relates to a method, device, electronic device, and storage medium for measuring the computing power resources of a processor. The method includes: obtaining a plurality of preset test scenarios, each test scenario including the processor model to be tested, the number of video stream channels, the base library data corresponding to the number of test base libraries, and the number of processors allocated; obtaining a plurality of video files, and based on the number of video stream channels, inputting the plurality of video files into the test algorithms under each test scenario for algorithm operation testing, recording the corresponding test data obtained by running the test algorithms under each test scenario, and further obtaining the measurement result corresponding to the processor model to be tested. The measurement result includes the maximum number of channels supported by the processor model to be tested under different numbers of test base libraries and different numbers of processor allocations. Through the present application, the problem of how to balance the low cost and high performance of the computing power resources occupied by the AI algorithm is solved, and the balance between cost and performance is achieved on the premise of meeting the business requirements.

Description

Technical Field

[0001] This application relates to the field of video AI intelligent monitoring, and particularly to a method, device, electronic device, and storage medium for measuring the computing power resources of a processor. Background Art

[0002] In the actual application scenarios of an AI analysis platform, especially when it is necessary to dock and apply the AI algorithms of manufacturers, the selection and procurement of GPU cards become a crucial link. This decision not only affects the operating efficiency and performance of the platform, but also directly impacts the overall cost investment and business benefits. Therefore, the platform side needs to comprehensively consider various factors such as its own business needs, usage scenarios, budget constraints, as well as the cost and performance of GPUs, in order to formulate the optimal GPU procurement plan.

[0003] However, in the current technical context, there is a significant problem: the lack of a general and rigorous measurement model to support the formulation of GPU procurement plans. Specifically, for face recognition algorithms, how to balance the two key factors of cost and performance in the process of purchasing and selecting GPU computing power resources according to the actual business demand of the platform, in order to produce the optimal procurement plan, has become an urgent problem to be solved.

[0004] Currently, no effective solution has been proposed for the problem of how to balance the low cost and high performance of the computing power resources occupied by AI algorithms in related technologies. Summary of the Invention

[0005] Embodiments of this application provide a method, device, electronic device, and storage medium for measuring the computing power resources of a processor, so as to at least solve the problem of how to balance the low cost and high performance of the computing power resources occupied by AI algorithms in related technologies.

[0006] In a first aspect, embodiments of this application provide a method for measuring the computing power resources of a processor, including:

[0007] Obtain a plurality of preset test scenarios, each of the test scenarios including the model of the processor to be tested, the number of video stream channels, the database data corresponding to the number of test databases, and the number of processors allocated;

[0008] Obtain a plurality of video files, and based on the number of video stream channels, input the plurality of video files into the test algorithms in each of the test scenarios for algorithm operation testing, and record the corresponding test data obtained by running the test algorithms in each of the test scenarios;

[0009] Based on the test data, obtain the measurement result corresponding to the processor model to be tested, where the measurement result includes the maximum number of channels supported by the processor model to be tested under different numbers of the test bottom libraries and different numbers of processor allocations.

[0010] In some embodiments, the obtaining the measurement result corresponding to the processor model to be tested based on the test data includes:

[0011] Verify one by one whether the test data in each test scenario meets the preset validity condition, and record the test data that meets the validity condition as valid data;

[0012] Based on the valid data, obtain the measurement result corresponding to the processor model to be tested.

[0013] In some embodiments, the validity condition includes that the difference between the multi-channel test data and the reference data is within the threshold range of the corresponding data; the reference data is determined according to the test data corresponding to the single-channel video stream; the multi-channel test data is determined according to the test data corresponding to the video stream with two or more channels.

[0014] In some embodiments, that the difference between the multi-channel test data and the reference data is within the threshold range of the corresponding data includes:

[0015] The difference in the number of captured images between the multi-channel test data and the reference data does not exceed the captured image valid threshold;

[0016] The difference in the number of alarms between the multi-channel test data and the reference data does not exceed the alarm valid threshold;

[0017] The difference in the number of false alarms between the multi-channel test data and the reference data does not exceed the false alarm valid threshold, and the false alarm rate corresponding to the number of false alarms does not exceed the false alarm rate threshold;

[0018] The difference in the number of missed alarms between the multi-channel test data and the reference data does not exceed the missed alarm valid threshold, and the missed alarm rate corresponding to the number of missed alarms does not exceed the missed alarm rate threshold;

[0019] The time interval between the capture and the alarm does not exceed the delay determination threshold.

[0020] In some embodiments, the validity condition further includes determining that there is no delay problem in the capture and alarm functions of the test scenario under multi-channel concurrency.

[0021] In some embodiments, the determining that there is no delay problem in the capture and alarm functions of the test scenario under multi-channel concurrency includes:

[0022] Add the real video stream address captured by a real camera to the test scenario to construct a test environment containing N video streams;

[0023] Select real people as test objects, and calculate the capture interval duration between the actual passing time recorded by the real camera based on the real people, the capture time recorded during the algorithm running test, and the alarm time triggered after capturing the real people, as well as the alarm interval duration between the capture time and the alarm time;

[0024] If the capture interval duration does not exceed the preset capture delay duration threshold and the alarm interval duration does not exceed the preset alarm delay duration threshold, it is determined that there is no delay problem in the capture and alarm functions in the test scenario under multi-channel concurrency.

[0025] In some embodiments, the obtaining of multiple video files and the inputting of the multiple video files into the test algorithms in each of the test scenarios based on the number of video streams for algorithm running tests include:

[0026] Deploy the test algorithm on the physical machine of the processor model to be tested, and the test algorithm is used to perform algorithm running tests in each of the test scenarios;

[0027] Utilize the deployed multi-channel real-time video distribution server to obtain multiple video files and input the video files into the test algorithms in each of the test scenarios deployed on the physical machine.

[0028] In a second aspect, an embodiment of the present application provides a processor computing power resource measurement device, including:

[0029] An input module, configured to obtain a preset plurality of test scenarios, each of the test scenarios including the processor model to be tested, the number of video streams, the bottom database data corresponding to the number of test bottom databases, and the number of processor allocations;

[0030] A measurement module, configured to obtain multiple video files, and based on the number of video streams, input the multiple video files into the test algorithms in each of the test scenarios to perform algorithm running tests, and record the corresponding test data obtained by running the test algorithms in each of the test scenarios;

[0031] A result analysis module, configured to obtain a measurement result corresponding to the processor model to be tested based on the test data, and the measurement result includes the maximum number of channels supported by the processor model to be tested under different numbers of test bottom databases and different numbers of processor allocations.

[0032] In a third aspect, an embodiment of the present application provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, the method for calculating the computing power resource of the processor as described in the first aspect above is implemented.

[0033] In a fourth aspect, an embodiment of the present application provides a storage medium, on which a computer program is stored. When the program is executed by a processor, the method for calculating the computing power resource of the processor as described in the first aspect above is implemented.

[0034] Compared with the related art, the method, device, electronic device, and storage medium for calculating the computing power resource of the processor provided by the embodiments of the present application obtain a plurality of preset test scenarios, including the processor model, the number of video stream channels, the bottom library data, and the allocation number. The video file is input into the algorithms under each test scenario for running tests, and the test data is recorded. Furthermore, the maximum supported number of channels of the processor model under different bottom library quantities and allocation numbers is obtained, solving the problem of how to balance the low cost and high performance of the computing power resources occupied by the AI algorithm in the related art, and achieving a balance between cost and performance on the premise of meeting the business requirements.

[0035] The details of one or more embodiments of the present application are set forth in the following drawings and description to make the other features, objects, and advantages of the present application more comprehensible. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments and descriptions thereof of the present application are used to explain the present application and do not constitute an improper limitation of the present application. In the drawings:

[0037] Figure 1 is a block diagram of the hardware structure of the terminal of the method for calculating the computing power resource of the processor according to an embodiment of the present invention;

[0038] Figure 2 is a flowchart of the method for calculating the computing power resource of the processor according to an embodiment of the present application;

[0039] Figure 3 is a schematic diagram of the modules of the method for calculating the computing power resource of the processor according to a preferred embodiment of the present application;

[0040] Figure 4 is a block diagram of the structure of the device for calculating the computing power resource of the processor according to an embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0041] In order to make the objectives, technical solutions and advantages of the present application clearer and more understandable, the present application will be described and explained below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application. All other embodiments obtained by those of ordinary skill in the art based on the embodiments provided in the present application without making creative efforts belong to the scope of protection of the present application. In addition, it can also be understood that although the efforts made in such a development process may be complex and lengthy, for those of ordinary skill in the art related to the content disclosed in the present application, some design, manufacturing or production changes based on the technical content disclosed in the present application are only conventional technical means and should not be understood as the content disclosed in the present application being insufficient.

[0042] In the present application, the mention of "embodiment" means that the specific features, structures or characteristics described in connection with the embodiment may be included in at least one embodiment of the present application. The phrase appears in various positions in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment mutually exclusive with other embodiments. Those of ordinary skill in the art explicitly and implicitly understand that the embodiments described in the present application may be combined with other embodiments without conflict.

[0043] Unless otherwise defined, the technical terms or scientific terms involved in the present application should be the ordinary meanings understood by those with ordinary skills in the technical field to which the present application belongs. The terms "a", "one", "kind", "the" and other similar words involved in the present application do not indicate a quantity limitation and may represent a single or plural number. The terms "including", "comprising", "having" and any variations thereof involved in the present application are intended to cover non-exclusive inclusion; for example, a process, method, system, product or device including a series of steps or modules (units) is not limited to the listed steps or units, but may further include unlisted steps or units, or may further include other steps or units inherent to these processes, methods, products or devices. The terms "connected", "coupled" and other similar words involved in the present application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. The "plurality" involved in the present application means greater than or equal to two. "And / or" describes the association relationship of associated objects and indicates that three relationships may exist. For example, "A and / or B" may represent: A exists alone, A and B exist simultaneously, and B exists alone. The terms "first", "second", "third", etc. involved in the present application are only used to distinguish similar objects and do not represent a specific order for the objects.

[0044] The method embodiment provided in this embodiment can be executed on a terminal, a computer or a similar computing device. Taking running on a terminal as an example,Figure 1 It is a hardware structure block diagram of a terminal for the method of measuring the computing power resources of a processor according to an embodiment of the present invention. As Figure 1 shown, the terminal may include one or more ( Figure 1 only one is shown in the figure) processors 102 (the processor 102 may include, but is not limited to, a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data. Optionally, the above terminal may further include a transmission device 106 for communication functions and an input / output device 108. Those of ordinary skill in the art can understand that Figure 1 the structure shown is only illustrative and does not limit the structure of the above terminal. For example, the terminal may further include more or fewer components than Figure 1 shown in the figure, or have a different configuration from Figure 1 shown.

[0045] The memory 104 can be used to store computer programs. For example, software programs and modules of application software, such as the computer program corresponding to the method of measuring the computing power resources of a processor according to an embodiment of the present invention. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, the above method is implemented. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memories, or other non-volatile solid-state memories. In some instances, the memory 104 may further include a memory remotely set relative to the processor 102, and these remote memories can be connected to the terminal through a network. Examples of the above network include, but are not limited to, the Internet, an enterprise intranet, a local area network, a mobile communication network, and combinations thereof.

[0046] The transmission device 106 is used to receive or send data via a network. Specific examples of the above network may include a wireless network provided by a communication provider of the terminal. In one instance, the transmission device 106 includes a network adapter (abbreviated as NIC), which can be connected to other network devices through a base station and thus communicate with the Internet. In one instance, the transmission device 106 may be a radio frequency (abbreviated as RF) module, which is used to communicate with the Internet wirelessly.

[0047] This embodiment provides a method for measuring the computing power resources of a processor, Figure 2 which is a flowchart of the method for measuring the computing power resources of a processor according to an embodiment of the present application. As Figure 2 shown, the process includes the following steps:

[0048] Step S201: Obtain a plurality of preset test scenarios. Each test scenario includes the processor model to be tested, the number of video stream channels, the database data corresponding to the number of test databases, and the number of processors allocated.

[0049] Among them, each test scenario includes multiple key parameters: the processor model to be tested, the number of video stream channels, the database data corresponding to the number of test databases, and the number of processors allocated. The processor model to be tested refers to the model of the GPU card that needs to be tested. In practical applications, different GPU cards have different performances and costs, so it is necessary to select a suitable model for testing according to the actual business requirements. The number of video stream channels refers to the number of video streams processed simultaneously. In the AI analysis platform, the number of video stream channels directly affects the load and performance of the GPU card, so it is necessary to test the performance under different numbers of channels to determine the optimal channel configuration. The database data is the basic data for face recognition, including the feature information of known faces, etc. The number of test databases directly affects the recognition speed and accuracy of the algorithm, so it is necessary to prepare different numbers of database data for testing to determine the optimal database configuration. The number of processors allocated refers to the number of GPU cards allocated to each test scenario. In practical applications, the processing speed and performance of the algorithm can be improved by increasing the number of GPU cards, so it is necessary to test the performance under different numbers of processors allocated to determine the optimal processor configuration. In this step, by setting test scenarios that include different processor models, numbers of video stream channels, numbers of test databases, and numbers of processors allocated, multiple test scenarios can comprehensively cover various situations that may be encountered, thereby improving the comprehensiveness and accuracy of the test, ensuring that the performance of the algorithm in practical applications meets expectations, and by using these multiple test scenarios to test the performance under different processor models, numbers of video stream channels, numbers of test databases, and numbers of processors allocated, the optimal resource configuration plan can be determined.

[0050] Step S202: Obtain multiple video files, and based on the number of video stream channels, input the multiple video files into the test algorithms under each test scenario for algorithm operation testing, and record the corresponding test data obtained by running the test algorithms under each test scenario.

[0051] Among them, diverse video files can be obtained from the actual monitoring system. The video files should cover different scenarios, lighting conditions, face angles, occlusion situations, etc., to ensure the comprehensiveness and authenticity of the test. Then, preprocess the video files, such as cropping, format conversion, etc. According to the preset test scenario configuration, combine different GPU models, the number of video stream channels, the number of test bottom libraries, and the number of allocated processors. Under each test scenario, configure the corresponding test parameters, and input the prepared video files into the corresponding test scenarios according to the preset number of video stream channels through a multi-channel real-time video distribution server. Start the algorithm running test, and record in real time the test data obtained by running the test algorithm in each test scenario. The test data includes the number of captures, the number of alarms, the number of false alarms, the number of missed alarms, the time of capture and alarm, the algorithm running time, and the resource occupancy (such as the usage rates of CPU, memory, and GPU), etc., to ensure the accuracy and integrity of the data for subsequent data analysis and performance evaluation. In this step, by using multiple video files and different test scenario combinations, the performance of the algorithm under different conditions can be comprehensively evaluated, and the recorded test data can provide strong data support for subsequent performance optimization and resource allocation.

[0052] Step S203: Based on the test data, obtain the measurement results corresponding to the processor model to be tested. The measurement results include the maximum number of channels supported by the processor model to be tested under different numbers of test bottom libraries and different numbers of allocated processors.

[0053] Among them, compare and count the number of captures, the number of alarms, the number of false alarms, and the number of missed alarms in the test data, and further analyze the maximum number of video stream channels that different processor models can support under different numbers of bottom libraries and different numbers of allocated processors. Present the analysis results in an intuitive way, such as a table or a chart, for easy understanding and application. The output results should clearly show the maximum supported number of channels of the processor model to be tested under different test conditions, providing strong support for subsequent procurement and selection decisions. In this step, by analyzing the maximum number of channels supported by the processor model to be tested under different numbers of test bottom libraries and different numbers of allocated processors, the performance of the processor model to be tested under different test conditions can be more accurately understood, thus providing a scientific basis for procurement and selection decisions and avoiding resource waste caused by blind procurement and selection; this measurement result can ensure that the selected processor model can support a sufficient number of video stream channels, thus meeting the requirements of the face recognition algorithm for real-time performance and accuracy, and according to the measurement result, the GPU computing power resources can be reasonably configured to ensure the balance of cost and performance while meeting the business requirements, helping to reduce the operation cost and improve the resource utilization efficiency.

[0054] Through the above steps, by presetting multiple test scenarios and comprehensively considering multiple key factors, the performance of the processor can be evaluated more comprehensively and meticulously, avoiding potential performance biases in a single test scenario and improving the accuracy and reliability of the measurement results. Secondly, by obtaining multiple video files and inputting these video files into the test algorithms under each test scenario for running tests according to the preset number of video stream channels, various test data obtained from the algorithm running under each test scenario are recorded in detail, including key indicators such as the number of captures, the number of alarms, the number of false alarms, the number of missed alarms, and the time delay of the algorithm running. By recording and analyzing in detail the various test data obtained from the algorithm running, the performance bottlenecks and potential problems of the processor can be understood more deeply, which provides an important reference basis for subsequent performance optimization and algorithm improvement. Finally, based on the collected test data, the measurement results corresponding to the processor model to be tested are further analyzed and obtained, specifically including the maximum number of video stream channels supported by the processor model to be tested under different test database quantities and different numbers of processor allocations, which helps to intuitively understand the performance of the processor under different configurations and provides a basis for subsequent resource allocation and performance optimization, solving the problem in related technologies of how to balance the low cost and high performance of the computing power resources occupied by the AI algorithm, and achieving a balance between cost and performance on the premise of meeting business requirements.

[0055] In some embodiments, based on the test data, the measurement results corresponding to the processor model to be tested are obtained, including:

[0056] Verify one by one whether the test data under each test scenario meet the preset validity conditions, and record the test data that meet the validity conditions as valid data;

[0057] Based on the valid data, the measurement results corresponding to the processor model to be tested are obtained.

[0058] Among them, for each test scenario, check whether the corresponding test data meets the preset validity conditions, which include but are not limited to the difference value ranges of the capture quantity, alarm quantity, false alarm number, and missed alarm number between multi-channel and single-channel, as well as whether there is a phenomenon of capture delay or alarm delay in the algorithm operation. Mark the test data that meets all validity conditions as valid data for subsequent analysis and processing. If a set of test data does not meet the validity conditions, mark it as invalid data and exclude it from subsequent analysis. By statistically analyzing the valid data, calculate the maximum number of video stream channels supported by the processor model to be tested under different base library quantities and different numbers of allocated processors, that is, the measurement result. Further specific conclusions can be drawn, such as how many real-time video stream channels a certain model of GPU graphics card can support at a specific base library quantity and the corresponding hardware configuration suggestions. In this embodiment, by verifying the validity of the test data one by one, the accuracy and reliability of the analysis result are ensured, and the interference of invalid data on the measurement result is avoided.

[0059] In some of these embodiments, the validity conditions include that the difference between the multi-channel test data and the reference data is within the threshold range of the corresponding data; the reference data is determined according to the test data corresponding to the single-channel video stream; the multi-channel test data is determined according to the test data corresponding to two or more video stream channels.

[0060] It can be explained that for each test scenario, collect the test data and compare it with the preset validity conditions one by one to check whether the capture quantity, alarm quantity, false alarm number, missed alarm number, and the delay conditions of capture and alarm in the test data meet the threshold requirements. In this embodiment, by setting clear validity conditions and verifying the test data one by one, it can be ensured that the test data used is accurate and reliable, which helps to avoid the problem of inaccurate measurement results caused by data errors.

[0061] In some of these embodiments, the difference between the multi-channel test data and the reference data being within the threshold range of the corresponding data includes:

[0062] The difference in the capture quantity between the multi-channel test data and the reference data does not exceed the capture validity threshold;

[0063] The difference in the alarm quantity between the multi-channel test data and the reference data does not exceed the alarm validity threshold;

[0064] The difference in the false alarm quantity between the multi-channel test data and the reference data does not exceed the false alarm validity threshold, and the false alarm rate corresponding to the false alarm quantity does not exceed the false alarm rate threshold;

[0065] The difference in the missed alarm quantity between the multi-channel test data and the reference data does not exceed the missed alarm validity threshold, and the missed alarm rate corresponding to the missed alarm quantity does not exceed the missed alarm rate threshold;

[0066] The time interval between capture and alarm does not exceed the delay determination threshold.

[0067] For example, during multi-channel testing, first record the capture count of each video stream and compare it with the reference data (i.e., the capture count of a single video stream) to ensure that the difference in the capture count between the multi-channel test data and the reference data does not exceed the preset capture validity threshold (e.g., not exceeding 15%). This is achieved by comparing and analyzing the test data to ensure that the capture count remains stable under multi-channel concurrent pressure without significant fluctuations. Similarly, record the alarm count of each video stream and compare it with the reference data to ensure that the difference in the alarm count between the multi-channel test data and the reference data does not exceed the preset alarm validity threshold (also not exceeding 15%), which helps to verify the alarm accuracy of the algorithm under multi-channel concurrent conditions and ensure that the system can respond to abnormal situations in a timely manner. Record the false alarm count in the multi-channel test and calculate the false alarm rate to ensure that the difference in the false alarm count between the multi-channel test data and the reference data does not exceed the preset false alarm validity threshold (not exceeding 10% of the number of people who should be reported), and the false alarm rate corresponding to the false alarm count does not exceed the false alarm rate threshold (not exceeding 15%), which helps to reduce the false alarm rate of the system and improve the accuracy of the algorithm. Record the missed alarm count in the multi-channel test and calculate the missed alarm rate to ensure that the difference in the missed alarm count between the multi-channel test data and the reference data does not exceed the preset missed alarm validity threshold (not exceeding 10% of the number of people who should be reported), and the missed alarm rate corresponding to the missed alarm count does not exceed the missed alarm rate threshold (also not exceeding 15%), which helps to reduce the missed alarm situation of the system and improve the coverage and reliability of the algorithm. During the test process, record the time interval between capture and alarm to ensure that this time interval does not exceed the preset delay determination threshold (e.g., within 3 seconds), which helps to verify the performance of the algorithm in terms of real-time and ensure that the system can respond to and process abnormal situations in a timely manner, enhancing the user experience. Through the above steps in this embodiment, the stability of the algorithm under multi-channel concurrent pressure can be ensured, the fluctuations in the capture count and alarm count can be reduced, and the reliability of the algorithm can be improved; controlling the false alarm count and missed alarm count helps to reduce the false alarm rate and missed alarm rate of the system and improve the accuracy of the algorithm; ensuring that the time interval between capture and alarm is within a reasonable range can enhance the real-time performance of the system, reduce the user waiting time, and improve the user experience.

[0068] In some of these embodiments, the validity condition further includes determining that there is no delay problem in the capture and alarm functions of the test scenario under multi-channel concurrent conditions.

[0069] In some of these embodiments, determining that there is no delay problem in the capture and alarm functions of the test scenario under multi-channel concurrent conditions includes:

[0070] Add the real video stream address captured by a real camera to the test scenario to construct a test environment containing N video streams;

[0071] Select real people as test subjects, and calculate the capture interval duration between the actual passing time recorded by the real camera based on real people, the capture time recorded during the algorithm running test, and the alarm time triggered after capturing real people. Also calculate the alarm interval duration between the capture time and the alarm time;

[0072] If the capture interval duration does not exceed the preset capture delay duration threshold, and the alarm interval duration does not exceed the preset alarm delay duration threshold, it is determined that there is no delay problem in the capture and alarm functions in the multi-channel concurrent scenario.

[0073] Among them, use a multi-channel real-time video distribution server to simulate the RTSP addresses (Real Time Streaming Protocol, real-time streaming protocol, the RTSP addresses of the required number of channels simulated by the multi-channel real-time video distribution server, used to test the performance of the algorithm in the multi-channel concurrent scenario) of the required number of channels (such as N - 1 channels), and add the RTSP address of 1 real camera to construct a multi-channel concurrent test scenario (N channels). Select real person A as the test subject and add it to the bottom database of the surveillance. When real person A passes by the camera, let real person A record the time point at the scene immediately as the actual passing time of the capture; obtain the time point when real person A is captured through the algorithm background, that is, the capture time, and compare the capture time with the actual passing time to calculate the capture interval duration, which represents the delay duration of the capture function. Similarly, view the alarm interval duration between the capture time and the alarm time through the algorithm background, which represents the alarm delay duration. Set the delay determination threshold according to business requirements (such as within 3 seconds). After multiple test verifications, if the capture interval duration does not exceed the preset capture delay duration threshold (such as within 3 seconds), and the alarm interval duration does not exceed the preset alarm delay duration threshold (such as within 3 seconds), it is determined that there is no delay problem in the capture and alarm functions in the multi-channel concurrent scenario (N channels). This embodiment can accurately evaluate the real-time performance of the algorithm in the multi-channel concurrent scenario through simulating the multi-channel concurrent scenario and conducting real-person cooperative tests, ensure the timeliness of the capture and alarm functions, ensure that there is no delay problem in the capture and alarm functions in the multi-channel concurrent scenario, help enhance the overall stability of the system, and improve the reliability and accuracy of the system.

[0074] In some of these embodiments, obtain multiple video files, and based on the number of video stream channels, input the multiple video files into the test algorithms in each test scenario for algorithm running tests, including:

[0075] Deploy the test algorithm on the physical machine of the processor model to be tested, and the test algorithm is used to conduct algorithm running tests in each test scenario;

[0076] Using the deployed multi-channel real-time video distribution server, obtain multiple video files and input the video files into the test algorithms under various test scenarios deployed on the physical machines.

[0077] Among them, after determining the GPU model to be tested (such as Nvidia Tesla T4), on the physical machine with this model of GPU, deploy the host instance of the specified face recognition algorithm. This instance will be the main body of the test, used to run the algorithm under various test scenarios and collect relevant performance data. Deploy the multi-channel real-time video distribution server (RTSP server), which can simulate multi-channel concurrent scenarios, provide multiple RTSP addresses, and prepare test materials, including the base library (i.e., the face database) and video files. These video files will be used as input data to test the performance of the algorithm under different scenarios. Distribute these video files to the test algorithms deployed on the physical machines through the RTSP server. The test algorithms will receive these video files, perform face recognition processing, and output relevant result data. In this embodiment, deploying the test algorithm on the physical machine of the processor model to be tested ensures the consistency and accuracy of the test environment, can truly reflect the performance of the GPU model to be tested during the operation of the face recognition algorithm, is convenient for collecting and analyzing the operation data of the algorithm under different scenarios, and provides strong support for subsequent performance optimization and algorithm improvement; using the deployed multi-channel real-time video distribution server can simulate multi-channel concurrent scenarios, comprehensively test the performance of the algorithm in complex environments, and improve the efficiency and accuracy of the test.

[0078] The embodiments of the present application will be described and illustrated below through preferred embodiments.

[0079] Figure 3 It is a schematic diagram of the module of the processor computing power resource measurement method according to the preferred embodiment of the present application. As Figure 3 shown, the processor computing power resource measurement method includes the following steps:

[0080] (1) Deploy the RTSP multi-channel real-time video distribution server (press machine) on the specified server.

[0081] (2) Deploy the host instance of the specified algorithm on the GPU physical machine.

[0082] (3) Prepare test materials, such as the base library and video files.

[0083] (4) Prerequisites for valid test results (including the following five conditions a, b, c, d, and e):

[0084] a. The difference in the number of captures between multi-channel and single-channel does not exceed 15%.

[0085] b. The difference in the number of alarms between multi-channel and single-channel does not exceed 15%.

[0086] c. The difference in the number of false alarms between multi-channel and single-channel does not exceed 10% of the number of alarms that should be reported, and the false alarm rate should not exceed 15%.

[0087] d. The difference in the number of missed alarms between multi-channel and single-channel does not exceed 10% of the number of alarms that should be reported, and the missed alarm rate should not exceed 15%.

[0088] e. The algorithm operation of each channel should not have the phenomenon of capture delay or alarm delay. Specifically, in the case of multi-channel and single-channel, each time a person appears in each video of each channel, the face needs to be captured within 3 seconds, and if the captured face matches the face in the database, an alarm needs to be issued within 3 seconds after the capture.

[0089] Specifically, in order to meet the "prerequisite e for valid test results", we adopted a workaround method to verify to a certain extent whether the capture and alarm of the camera are delayed due to too many channels and result in a poor experience. The specific approach is as follows. For example, if we want to stress test whether the algorithm can support N channels, we first use a multi-channel real-time video distribution server to simulate the RTSP addresses of N - 1 channels, then add the RTSP address of 1 real camera, and then let real person A cooperate with the test. That is, first add A to the controlled database, then let A pass by this camera and record the time point at the scene by himself immediately. After that, compare it with the time point when the algorithm background captures this person A to see how many seconds are in between. If after multiple tests and verifications, this interval duration can be within the range we specified (currently set within 3 seconds), it can be shown to a certain extent that the algorithm basically does not have the phenomenon of capture delay under the concurrent pressure of N channels. Finally, check through the algorithm background how many seconds are in between the capture and alarm of this person A. If after multiple tests and verifications, this interval duration can be within the range we specified (currently set within 3 seconds), it can be shown to a certain extent that the algorithm basically does not have the phenomenon of alarm delay under the concurrent pressure of N channels.

[0090] (5)Taking the GPU model, number of channels, number of databases, and number of allocated GPUs as input conditions, conduct combined tests, collect the 5 test data listed in the fourth point, and calculate the maximum number of channels supported by the corresponding model of GPU card in scenarios with different numbers of databases and different numbers of allocated GPUs.

[0091] Compared with the measurement methods of hardware resources required by other AI algorithms, the preferred embodiment of the present application has more test scenarios composed of input conditions, and more diverse and rich results as data support for subsequent test analysis and conclusions. Taking a single channel as the benchmark, a fluctuation range of the difference from the benchmark is specified. This range is based on the impact on the business. If it exceeds this range, it is determined that the impact on the business is large and the result is invalid. The difference from the benchmark is 10%-15% because it is determined that the impact on the business within 20% of the difference is acceptable. The response time of 3 seconds refers to the 358 principle in the industry (if a page can be opened within three seconds, the user experience is relatively good; if it takes more than five seconds to open a page, the user experience will be affected; if the page has not been opened after more than eight seconds, the user will abandon using the product), which reflects that the present application focuses on considering the user experience and business performance under pressure during the implementation process.

[0092] The following are two test cases:

[0093] Case 1:

[0094] This is an extract of the valid data for Case 1 (not reflecting all the test data collected during the test process), which intuitively reflects the conditions and data described above. As one of the references, the following conclusions can be drawn from this data:

[0095] 1) The dynamic face recognition algorithm 1 on a single Nvidia Tesla T4 model GPU card can support a maximum of 22 live video streams in the case of a 5W face database. The detailed test result data is shown in Table 1 below.

[0096] 2) If applying for the host resources of a single-card GPU for this algorithm, the specific hardware configuration is: 1 T4 card, 24-thread CPU (at least 24 threads, 32 threads if possible), and 80G of memory.

[0097] Table 1

[0098]

[0099] Case 2:

[0100] Tables 2 to 6 show the data of the entire test process for Case 2. Based on this data, the following test conclusions are drawn:

[0101] 1) The more the number of stress test channels, the more stable the capture number and alarm number, with no obvious fluctuation and a deviation of less than 10%.

[0102] 2) The more the number of stress test channels, the more the false alarms and missed alarms show an increasing trend. The deviation of 12 channels and 15 channels is relatively large compared to other multi-channel scenarios.

[0103] Comprehensive optimal performance: 10 channels.

[0104] Table 2

[0105]

[0106] Table 3

[0107]

[0108] Table 4

[0109]

[0110] Table 5

[0111]

[0112] Table 6

[0113]

[0114] This embodiment also provides a device for measuring the computing power resources of a processor. This device is used to implement the above-mentioned embodiments and preferred implementation manners, and those that have been described will not be repeated here. As used below, terms such as "module", "unit", "sub-unit", etc. can be a combination of software and / or hardware that can achieve a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, implementation in hardware, or a combination of software and hardware is also possible and contemplated.

[0115] Figure 4 is a structural block diagram of the device for measuring the computing power resources of a processor according to an embodiment of the present application. As Figure 4 shown, the device includes:

[0116] An input module 10, configured to obtain a plurality of preset test scenarios, where each test scenario includes the model of the processor to be tested, the number of video stream channels, the base library data corresponding to the number of test base libraries, and the number of processors allocated;

[0117] A measurement module 20, configured to obtain a plurality of video files, and based on the number of video stream channels, input the plurality of video files into the test algorithms in each test scenario for algorithm operation testing, and record the corresponding test data obtained by running the test algorithms in each test scenario;

[0118] A result analysis module 30, configured to obtain a measurement result corresponding to the model of the processor to be tested based on the test data, where the measurement result includes the maximum number of channels supported by the model of the processor to be tested under different numbers of test base libraries and different numbers of processors allocated.

[0119] It should be noted that each of the above modules can be a functional module or a program module, and can be implemented either by software or by hardware. For the modules implemented by hardware, each of the above modules can be located in the same processor; or each of the above modules can also be located in different processors in any combination.

[0120] This embodiment also provides an electronic device, including a memory and a processor. A computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0121] Optionally, the above electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the above processor, and the input / output device is connected to the above processor.

[0122] Optionally, in this embodiment, the above processor may be configured to execute the following steps through a computer program:

[0123] S1, obtain a plurality of preset test scenarios, each test scenario including the model of the processor to be tested, the number of video stream channels, the base library data corresponding to the number of test base libraries, and the number of processor allocations;

[0124] S2, obtain a plurality of video files, and based on the number of video stream channels, input the plurality of video files into the test algorithms under each test scenario for algorithm running test, and record the corresponding test data obtained by running the test algorithms under each test scenario;

[0125] S3, based on the test data, obtain the measurement result corresponding to the model of the processor to be tested, and the measurement result includes the maximum number of channels supported by the model of the processor to be tested under different numbers of test base libraries and different numbers of processor allocations.

[0126] It should be noted that the specific examples in this embodiment may refer to the examples described in the above embodiments and optional implementation manners, and will not be elaborated herein.

[0127] In addition, in combination with the processor computing power resource measurement method in the above embodiments, an embodiment of the present application can be implemented by providing a storage medium. A computer program is stored on the storage medium; when the computer program is executed by a processor, it implements any one of the processor computing power resource measurement methods in the above embodiments.

[0128] Those skilled in the art should understand that the technical features of the above embodiments can be combined arbitrarily. For the sake of brevity of description, all possible combinations of the technical features in the above embodiments are not described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope described in this specification.

[0129] The above-described embodiments merely represent several implementation manners of the present application. The description thereof is relatively specific and detailed, but it should not be construed as a limitation on the scope of the invention patent. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several modifications and improvements can still be made, and these all fall within the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the appended claims.

Claims

1. A method for calculating processor computing power resources, characterized in that: include: Acquire multiple preset test scenarios, each of which includes the processor model to be tested, the number of video streams, the base library data corresponding to the number of test base libraries, and the number of processors allocated; Acquire multiple video files, and based on the number of video streams, input the multiple video files into the test algorithm under each of the test scenarios, perform algorithm operation test, and record corresponding test data obtained by running the test algorithm under each of the test scenarios; Based on the test data, obtaining a calculation result corresponding to the processor model to be tested includes: Verifying whether the test data in each of the test scenarios meets a preset validity condition one by one, and recording the test data that meets the validity condition as valid data; Based on the valid data, the calculation result corresponding to the processor model to be tested is obtained; the calculation result includes the maximum number of paths supported by the processor model to be tested under different numbers of the test base libraries and different numbers of processor allocations; the validity condition also includes determining that the capture and alarm functions of the test scenario have no delay problems under multi-path concurrency, including: Add a real video stream address captured by a real camera to the test scene to build a test environment containing N video streams; A real person is selected as a test subject, and based on the actual elapsed time recorded by the real camera when the real person passes by, the capture time recorded in the algorithm running test, and the alarm time when the alarm is triggered after the real person is captured, the capture interval between the actual elapsed time and the capture time, and the alarm interval between the capture time and the alarm time are calculated; If the capture interval does not exceed the preset capture delay threshold, and the alarm interval does not exceed the preset alarm delay threshold, it is determined that there is no delay problem in the capture and alarm functions of the test scenario under multi-channel concurrency.

2. The method for calculating processor computing power resources according to claim 1, characterized in that: The validity condition includes that the difference between the multi-channel test data and the reference data is within the threshold range of the corresponding data; The reference data is determined based on the test data corresponding to when the number of video streams is single; the multi-channel test data is determined based on the test data corresponding to when the number of video streams is two or more.

3. The method for calculating processor computing power resources according to claim 2, characterized in that: The difference between the multi-channel test data and the benchmark data is within the threshold range of the corresponding data, including: The difference between the number of snapshots of the multi-channel test data and the number of snapshots of the reference data does not exceed the snapshot validity threshold; The difference in the number of alarms between the multi-channel test data and the reference data does not exceed the alarm validity threshold; The difference in the number of false alarms between the multi-channel test data and the reference data does not exceed a false alarm effective threshold, and the false alarm rate corresponding to the number of false alarms does not exceed a false alarm rate threshold; The difference between the number of missed reports of the multi-channel test data and the reference data does not exceed a missed report effective threshold, and the missed report rate corresponding to the number of missed reports does not exceed a missed report rate threshold; The time interval between the snapshot and the alarm does not exceed the delay determination threshold.

4. The method for calculating processor computing power resources according to claim 1, characterized in that: The acquiring of multiple video files and, based on the number of video streams, inputting the multiple video files into the test algorithms in each of the test scenarios to perform algorithm operation tests include: Deploy the test algorithm on a physical machine of the processor model to be tested, wherein the test algorithm is used to perform algorithm operation tests in each of the test scenarios; Using the deployed multi-channel real-time video distribution server, a plurality of the video files are acquired, and the video files are input into the test algorithm in each of the test scenarios deployed on the physical machine.

5. A processor computing power resource measurement device, characterized in that: include: An input module is used to obtain a plurality of preset test scenarios, each of which includes the processor model to be tested, the number of video streams, the base library data corresponding to the number of test base libraries, and the number of processors allocated; A calculation module, used for acquiring a plurality of video files, and based on the number of video streams, inputting the plurality of video files into a test algorithm under each of the test scenarios, performing an algorithm operation test, and recording corresponding test data obtained by running the test algorithm under each of the test scenarios; A result analysis module, used to obtain the measurement result corresponding to the processor model to be tested based on the test data, including: Verifying whether the test data in each of the test scenarios meets a preset validity condition one by one, and recording the test data that meets the validity condition as valid data; Based on the valid data, the calculation result corresponding to the processor model to be tested is obtained; the calculation result includes the maximum number of paths supported by the processor model to be tested under different numbers of the test base libraries and different numbers of processor allocations; the validity condition also includes determining that the capture and alarm functions of the test scenario have no delay problems under multi-path concurrency, including: Add a real video stream address captured by a real camera to the test scene to build a test environment containing N video streams; A real person is selected as a test subject, and based on the actual elapsed time recorded by the real camera when the real person passes by, the capture time recorded in the algorithm running test, and the alarm time when the alarm is triggered after the real person is captured, the capture interval between the actual elapsed time and the capture time, and the alarm interval between the capture time and the alarm time are calculated; If the capture interval does not exceed the preset capture delay threshold, and the alarm interval does not exceed the preset alarm delay threshold, it is determined that there is no delay problem in the capture and alarm functions of the test scenario under multi-channel concurrency.

6. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to run the computer program to execute the processor computing power resource measurement method according to any one of claims 1 to 4.

7. A storage medium, characterized in that: The storage medium stores a computer program, wherein the computer program is configured to execute the processor computing power resource estimation method according to any one of claims 1 to 4 when running.

Images