An automated testing method, device, computer equipment and storage medium

Abstract

The application provides an automatic test method and device, computer equipment and a storage medium, comprising: driving an IPC device to execute an algorithm application to make the IPC device output a log; analyzing the log, extracting a unique identifier of each detection object, and extracting feature point data and corresponding attribute information of the detection object; archiving and storing the feature point data and the attribute information to the database with the unique identifier as the association key; reading the feature point data and the attribute information associated with the unique identifier from the database, generating a labeling file from the coordinate sequence in the feature point data and the attribute information; sending the labeling file to an NVR device to drive the NVR device to perform recognition and obtain a recognition result; comparing the recognition result with the attribute information to obtain a test evaluation report. The above method effectively improves the test efficiency and data accuracy.

Description

Technical Field

[0001] This invention relates to the field of algorithm testing technology for intelligent video surveillance equipment, and more specifically, to an automated testing method, apparatus, computer equipment, and storage medium. Background Technology

[0002] With the rapid development of the intelligent security industry, two-level linkage monitoring systems, consisting of front-end network cameras for data acquisition and analysis and back-end network video recorders for storage and recognition, have become the mainstream deployment form for various security scenarios. The performance of algorithms such as face detection and attribute recognition directly determines the business effect and operational stability of the entire monitoring system. Therefore, algorithm testing has become a core and necessary process in the equipment research and development, debugging and delivery stages.

[0003] Currently, the industry standard for algorithm testing of such linkage systems generally adopts an implementation plan of independent testing for the front-end and back-end. For the front-end camera equipment, testers manually prepare test image materials and trigger the device's algorithm to run, verifying the accuracy of the feature coordinates and attribute labels extracted by the device through manual comparison. For the back-end recording equipment, testers pre-annotate and generate simulated feature point data, inputting the annotated data into the device to verify its recognition algorithm performance. The test data for the two types of equipment are stored and managed independently, without establishing a data link.

[0004] The implementation plan has the drawback of insufficient end-to-end test coverage. The independent test mode can only verify the independent algorithm performance of a single device, and cannot cover the complete business link of "front-end feature extraction - data transmission and interaction - back-end recognition and comparison". Linkage issues such as incompatibility of front-end and back-end data formats and mismatch of attribute fields cannot be detected in advance during the testing phase. They usually only emerge during the on-site equipment integration and debugging phase, which greatly increases the integration and debugging costs and lengthens the overall project delivery cycle. Summary of the Invention

[0005] In view of this, the purpose of the present invention is to provide an automated testing method, apparatus, computer equipment and storage medium, which effectively improves testing efficiency and data accuracy.

[0006] In a first aspect, embodiments of this application provide an automated testing method, the method comprising: Drive the IPC device to execute the algorithm application so that the IPC device outputs logs; The logs are parsed to extract the unique identifier of each detected object, as well as the feature point data and corresponding attribute information of the detected object. The feature point data and the attribute information are archived and stored in the database using the unique identifier as the association key; Read the feature point data and attribute information associated with the unique identifier from the database, and generate a labeling file using the coordinate sequence in the feature point data and the attribute information; The annotation file is sent to the NVR device, which then performs recognition and obtains the recognition result. The identification results are compared with the attribute information to obtain a test evaluation report.

[0007] Optionally, the feature point data includes a coordinate sequence of the bounding rectangle of the target, the coordinate sequence consisting of four values, used to select the target region of the detected object in the image; the attribute information includes the category label and attribute fields of the detected object; The step of generating a labeling file using the coordinate sequence and attribute information from the feature point data includes: The coordinate sequence is used as the bounding box field value of the annotation file, the category label is used as the label field value of the annotation file, and the attribute field is used as the attribute field value of the annotation file. An annotation file containing bounding box field, label field and attribute field is generated and output as text format or JSON format.

[0008] Optionally, before driving the NVR device to perform identification, the method further includes: Determine the test mode; When the test mode is the linkage mode, the step of reading the feature point data and attribute information associated with the unique identifier from the database and generating a label file is executed, and the label file is sent to the NVR device to drive the NVR device to perform recognition based on the label file; When the test mode is standalone mode, the NVR device is driven to directly extract feature points from the original image to perform recognition, without reading the feature point data and attribute information in the database.

[0009] Optionally, parsing the logs to extract the unique identifier of each detected object, and extracting the feature point data and corresponding attribute information of the detected object, includes: The unique identifier is extracted from the target field by matching the target field in the log using regular expressions. The feature point data is extracted by parsing the feature point field in the log. The attribute information is extracted by parsing the attribute fields in the log.

[0010] Optionally, the step of archiving and storing the feature point data and the attribute information in the database using the unique identifier as the association key includes: Extract the image prefix names from the logs; The feature point data is stored as a feature point file using a file naming format that includes the unique identifier; The unique identifier, the image prefix name, and the attribute information are associated and stored in a data table of the database, wherein the unique identifier is used as the primary key or index field of the data table; The step of reading the feature point data and attribute information associated with the unique identifier from the database includes: Based on the unique identifier, query the data table of the database for records that match the unique identifier, and read the attribute information from the matching records; Based on the image prefix name and the unique identifier, determine the storage path of the feature point file, and read the coordinate sequence in the feature point file.

[0011] Optionally, the algorithm application that drives the IPC device includes: Connect to the IPC device via serial port, mount the shared directory, and start the algorithm process of the IPC device. After parsing the log, the method further includes: Based on the unique identifier, scan the shared directory to obtain the affine transformation diagram file; The storage path of the affine transformation diagram file is embedded into the data structure of the corresponding detection object using the unique identifier; Sending the labeled file to the NVR device and driving the NVR device to perform recognition includes: The annotation file and the affine transformation diagram file are sent together to the NVR device; The NVR device is driven to load the annotation file and perform feature point recognition based on the affine transformation image file.

[0012] Optionally, comparing the identification result with the attribute information to obtain a test evaluation report includes: Each time, determine whether the NVR device's recognition result for each detected object is consistent with the attribute information; If the recognition result matches the category label and attribute field in the attribute information, it is marked as a true example; If the identification result contains a detection object that is not present in the attribute information, it is marked as a false positive. If the detected object in the attribute information is not recognized by the NVR device, or if the recognized attribute field is inconsistent with the attribute information, it is marked as a false negative example; If the NVR device correctly excludes non-target objects, it is marked as a true negative example; Based on the statistical results of the true positives, false positives, false negatives and true negatives, the evaluation metrics are calculated, including accuracy, recall and F1 score. The test evaluation report is obtained by integrating the evaluation indicators.

[0013] Secondly, embodiments of this application provide an automated testing apparatus, the apparatus comprising: The log acquisition module is used to drive the IPC device to execute algorithm applications so that the IPC device outputs logs. The log parsing module is used to parse the logs, extract the unique identifier of each detected object, and extract the feature point data and corresponding attribute information of the detected object; The storage module is used to archive and store the feature point data and the attribute information in the database using the unique identifier as the association key; The annotation generation module is used to read the feature point data and attribute information associated with the unique identifier from the database, and generate an annotation file using the coordinate sequence in the feature point data and the attribute information; The sending driver module is used to send the annotation file to the NVR device, drive the NVR device to perform recognition, and obtain the recognition result; The comparison output module is used to compare the recognition result with the attribute information to obtain a test evaluation report.

[0014] Optionally, the feature point data includes a coordinate sequence of the bounding rectangle of the target, the coordinate sequence consisting of four values, used to select the target region of the detected object in the image; the attribute information includes the category label and attribute fields of the detected object; The annotation generation module is specifically used for: The coordinate sequence is used as the bounding box field value of the annotation file, the category label is used as the label field value of the annotation file, and the attribute field is used as the attribute field value of the annotation file. An annotation file containing bounding box field, label field and attribute field is generated and output as text format or JSON format.

[0015] Optionally, the device further includes a mode determination module for determining a test mode before driving the NVR device to perform recognition; When the test mode is the linkage mode, the annotation generation module performs the step of reading the feature point data and attribute information associated with the unique identifier from the database and generating an annotation file, and the sending driver module sends the annotation file to the NVR device to drive the NVR device to perform recognition based on the annotation file; When the test mode is standalone mode, the sending driver module drives the NVR device to directly extract feature points from the original image to perform recognition, without reading the feature point data and attribute information in the database.

[0016] Optionally, the log parsing module is specifically used for: The unique identifier is extracted from the target field by matching the target field in the log using regular expressions. The feature point data is extracted by parsing the feature point field in the log. The attribute information is extracted by parsing the attribute fields in the log.

[0017] Optionally, the storage module is specifically used for: Extract the image prefix names from the logs; The feature point data is stored as a feature point file using a file naming format that includes the unique identifier; The unique identifier, the image prefix name, and the attribute information are associated and stored in a data table of the database, wherein the unique identifier is used as the primary key or index field of the data table; The annotation generation module is specifically used for: Based on the unique identifier, query the data table of the database for records that match the unique identifier, and read the attribute information from the matching records; Based on the image prefix name and the unique identifier, determine the storage path of the feature point file, and read the coordinate sequence in the feature point file.

[0018] Optionally, the log acquisition module is specifically used for: Connect to the IPC device via serial port, mount the shared directory, and start the algorithm process of the IPC device. The storage module is further configured to: after parsing the log, scan the shared directory to obtain the affine transformation graph file based on the unique identifier; and embed the storage path of the affine transformation graph file into the data structure of the corresponding detection object using the unique identifier; The sending driver module is specifically used for: The annotation file and the affine transformation diagram file are sent together to the NVR device; The NVR device is driven to load the annotation file and perform feature point recognition based on the affine transformation image file.

[0019] Optionally, the comparison output module is specifically used for: Each time, determine whether the NVR device's recognition result for each detected object is consistent with the attribute information; If the recognition result matches the category label and attribute field in the attribute information, it is marked as a true example; If the identification result contains a detection object that is not present in the attribute information, it is marked as a false positive. If the detected object in the attribute information is not recognized by the NVR device, or if the recognized attribute field is inconsistent with the attribute information, it is marked as a false negative example; If the NVR device correctly excludes non-target objects, it is marked as a true negative example; Based on the statistical results of the true positives, false positives, false negatives and true negatives, the evaluation metrics are calculated, including accuracy, recall and F1 score. The test evaluation report is obtained by integrating the evaluation indicators.

[0020] Thirdly, embodiments of this application provide a computer device, including: a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the computer device is running, the processor communicates with the memory via the bus. When the machine-readable instructions are executed by the processor, the steps of the automated testing method described in any of the optional embodiments of the first aspect are performed.

[0021] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor, performs the steps of the automated testing method described in any of the optional embodiments of the first aspect.

[0022] The technical solution provided in this application includes, but is not limited to, the following beneficial effects: The algorithm application is driven by the IPC device to output logs, which can automatically trigger the device algorithm to run without the need for testers to manually upload test materials and start the algorithm process, thus reducing the cost of manual operation. At the same time, it ensures that the algorithm start conditions and running conditions are consistent for each round of testing, thereby improving the repeatability of the test process and the comparability of results.

[0023] Parsing the logs and extracting the unique identifier of each detected object, as well as the feature point data and corresponding attribute information of the detected object, can automatically complete the structured extraction of core test data, replacing the manual method of reading logs line by line and manually recording coordinates and attribute parameters, greatly improving data extraction efficiency, while avoiding problems such as numerical input errors and field omissions caused by manual recording, ensuring the accuracy of basic test data, and the extracted unique identifier also provides a core basis for subsequent full-link data association.

[0024] By archiving and storing the feature point data and attribute information in the database with the unique identifier as the association key, standardized and persistent storage of test data can be achieved, a binding association between feature point data and attribute information can be established, the problem of scattered data storage and inability to manage data in traditional solutions can be solved, a reusable standardized test dataset can be accumulated, and a structured data source can be provided for subsequent cross-end data calls and problem tracing.

[0025] The system reads the feature point data and attribute information associated with the unique identifier from the database, and generates a labeling file using the coordinate sequence in the feature point data and the attribute information. This system can automatically generate the labeling file required for backend testing based on the ground value data archived by the front end, eliminating the need for manual labeling data compilation, reducing the time cost and human error of manual labeling, ensuring the consistency and standardization of the labeling data with the data extracted by the front end, and effectively improving the preparation efficiency and data reliability of backend testing.

[0026] Sending the labeled file to the NVR device drives the NVR device to perform recognition and obtain recognition results. This can establish a transmission link from the front-end data to the back-end recognition, automatically drive the back-end device to perform recognition, reproduce the operational logic of front-end and back-end linkage in real business, realize end-to-end complete link testing and verification, replace the manual operation of inputting test data into the NVR, and improve the automation level and link coverage integrity of testing.

[0027] The identification results are compared with the attribute information to obtain a test evaluation report. The system can automatically compare the identification results with the true attributes one by one, and output a standardized test evaluation report. This eliminates the need for manual verification and statistics, greatly improving the efficiency of result analysis. At the same time, the system ensures the objectivity and accuracy of the evaluation results based on a unified true value benchmark, and can intuitively present the overall performance of the algorithm.

[0028] In summary, this invention achieves complete link testing of IPC and NVR front-end and back-end linkage through the fully automated design of front-end device driving, data extraction, data archiving to back-end annotation generation, recognition driving, and result evaluation. It effectively improves testing efficiency and data accuracy, ensures the completeness of test coverage, and can support efficient testing and algorithm iteration of intelligent monitoring linkage systems.

[0029] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0030] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0031] Figure 1 This diagram illustrates an automated testing platform system architecture provided in Embodiment 1 of the present invention. Figure 2 A flowchart of an automated testing method provided in Embodiment 1 of the present invention is shown; Figure 3 This diagram illustrates an IPC front-end testing flowchart provided in Embodiment 1 of the present invention. Figure 4 The flowchart of a data processing method under different test modes provided in Embodiment 1 of the present invention is shown; Figure 5 This invention provides a flowchart of a test process for an NVR backend linkage mode according to Embodiment 1. Figure 6 The flowchart of a method for extracting detection object data provided in Embodiment 1 of the present invention is shown; Figure 7 A flowchart of a database archiving and storage method provided in Embodiment 1 of the present invention is shown; Figure 8 A flowchart of a database reading method provided in Embodiment 1 of the present invention is shown; Figure 9 A flowchart of a storage path embedding method provided in Embodiment 1 of the present invention is shown; Figure 10 The flowchart of a method for driving an NVR device to perform identification, as provided in Embodiment 1 of the present invention, is shown. Figure 11 A flowchart of a method for obtaining a test evaluation report provided in Embodiment 1 of the present invention is shown; Figure 12 A schematic diagram of the structure of an automated testing device provided in Embodiment 2 of the present invention is shown; Figure 13 A schematic diagram of the structure of a computer device provided in Embodiment 3 of the present invention is shown. Detailed Implementation

[0032] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0033] Example 1 See Figure 1 As shown, Figure 1 The diagram illustrates an automated testing platform system architecture provided in Embodiment 1 of the present invention. This architecture forms the overall execution framework of the present invention. The top-level unified automated testing platform is divided into two parallel branches: an IPC testing tool and an NVR testing tool. Both tools integrate serial port drivers, log parsing, and feature point extraction capabilities. After completing the front-end data processing, the IPC testing tool outputs a result output module. The NVR testing tool can communicate bidirectionally with the IPC side result output through the IPCTool cross-database query module. The entire chain relies on the objId association mechanism to achieve front-end and back-end data binding and traceability, solving the shortcomings of traditional solutions where IPC and NVR data are independent and unrelated, and end-to-end testing cannot be carried out.

[0034] This solution establishes a unified automated testing platform as the core of the overall architecture. The platform is divided into two parallel execution branches: an IPC testing tool and an NVR testing tool. Both tools have serial port driver, log parsing, and feature point extraction capabilities. The front-end test data output by the IPC testing tool can achieve front-end and back-end data communication through the IPCTool cross-database query module. The entire chain relies on the objId association mechanism to achieve data binding and traceability.

[0035] This invention addresses five inherent shortcomings of existing independent testing solutions: First, the testing coverage is insufficient. Testing IPCs and NVRs separately cannot complete the entire business chain of "IPC feature point extraction → feature point transmission → NVR recognition," and compatibility faults such as inconsistent data formats and missing attribute fields are only exposed during the device integration and debugging phase. Second, manual annotation is costly and error-prone. Traditional NVR testing requires manual annotation of coordinates and facial attribute labels, which is time-consuming to prepare test materials and prone to human error in parameter writing. Third, fault location efficiency is extremely low. There is no unified data association identifier, and it is impossible to quickly distinguish whether the fault originates from abnormal IPC feature extraction or a defect in the NVR recognition algorithm after an NVR recognition error. The average time for locating a single problem is 2 to 4 hours. Fourth, test data is not reusable. There is no unified archiving standard for IPC raw feature point data, and test materials need to be remade for each round of regression testing, resulting in a large amount of repetitive and ineffective work. Fifth, there is a lack of visual debugging support. Affine transformation debugging diagrams cannot be bound to the corresponding detection objects, making it difficult for R&D personnel to intuitively analyze the underlying causes of recognition failures.

[0036] The entire solution includes two core innovations with the highest priority: Innovation 1 is a front-end and back-end data association mechanism based on objId, and Innovation 2 is an automatic annotation generation method for cross-database queries using IPCTool. It also sets up five equivalent replacement technical implementations to broaden the scope of protection: HTTP (HyperText Transfer Protocol) / REST API (Representational State Transfer Application Programming Interface) and SSH (Secure Shell) remote network interfaces to replace RS232 / RS485 serial communication; MySQL (open-source relational database) and SQLite (lightweight embedded relational database) relational databases to replace TinyDB (lightweight embedded NoSQL database) lightweight NoSQL database; and MQTT (Message Queuing Telemetry Transport) and Kafka (distributed streaming message queue) message queue real-time data streams to replace NFS (Network File System). System (network file sharing directory) shared file transfer; cloud-based distributed test scheduling platform replaces local single-machine test host; deep learning log intelligent recognition model replaces regular expression rule matching and parsing logs. The various alternatives differ only in their applicable scenarios, but all can fully achieve the core objective of automated linkage testing of this invention.

[0037] To facilitate understanding of this application, the following is combined with... Figure 2The flowchart of the automated testing method provided in Embodiment 1 of the present invention will be described in detail for Embodiment 1 of this application.

[0038] See Figure 2 As shown, Figure 2 A flowchart of an automated testing method provided in Embodiment 1 of the present invention is shown, wherein the method includes steps S201 to S206: S201: Drive the IPC device to execute the algorithm application so that the IPC device outputs logs.

[0039] Specifically, this step relies on device_algorithm_plus (a dedicated algorithm testing tool for IPC front-end) to automatically execute the device startup process. By default, it uses RS232 / RS485 serial ports to establish a communication channel between the test host and the IPC hardware device. After the communication is established, an NFS shared directory is mounted. This directory is used to uniformly store the original test image, the affine transformation image generated by the IPC, and the standardized archived feature files. After the tool issues the startup command, it runs the IPC's built-in face detection and feature extraction algorithm process.

[0040] During continuous operation, the algorithm automatically outputs device operation logs and business logs carrying structured information of all detected objects. The entire process eliminates the need for testers to manually upload images or trigger the algorithm, completely solving the pain point of low efficiency in traditional manual pre-operation. There are equivalent alternatives to serial communication, which can be replaced with HTTP / REST API or SSH remote interface to achieve cross-network remote device calls. The serial port has stronger stability in offline laboratory environments and better network interface adaptability in online batch remote testing scenarios.

[0041] S202: Parse the log, extract the unique identifier of each detected object, and extract the feature point data and corresponding attribute information of the detected object.

[0042] Specifically, this step uses regular expressions to match key fields in the logs to complete data extraction by default. Alternatively, you can choose to train a deep learning recognition model to automatically identify target fields in the logs and adapt to non-standard custom log formats from various vendors. Both parsing methods can completely extract all core business fields.

[0043] objId is a globally unique identifier for the detected object, with example values ​​objId1 and objId2. It is the core key for data association throughout the entire IPC and NVR chain. nvr_point (NVR-side feature point storage field) is a dedicated field in the log that stores the coordinates of the face region. It internally encapsulates a standard four-element coordinate sequence format [xmin (horizontal pixel coordinates of the upper left corner of the face's outer rectangle), ymin (vertical pixel coordinates of the upper left corner of the face's outer rectangle), xmax (horizontal pixel coordinates of the lower right corner of the face's outer rectangle), ymax (vertical pixel coordinates of the lower right corner of the face's outer rectangle)].

[0044] According to the accompanying technical Q&A, these four sets of values ​​are only used to select the overall face area, which is different from the fine facial feature key points such as the corners of the eyes, the tip of the nose, and the lips, which are far more than four. The actual business logic is to first lock the face area with a four-element rectangle, and then extract fine feature points within the box for final comparison. attributes (complete set of attributes of the detected object): built-in subfields include custom_name (custom ID name of the target), side_face (face side profile identifier, value 0 represents frontal face), face_race (ethnicity classification label), and face_decorations (face decoration labels, such as glasses, hats).

[0045] The testing tool iterates through the IPC output logs line by line, simultaneously extracting the unique objId identifier, nvr_point quad rectangle coordinates, and all attribute labels for a single face object. This completely replaces the traditional method of manually viewing and recording parameters, avoiding the problems of time-consuming manual annotation and incorrect parameter recording.

[0046] S203: Archive and store the feature point data and the attribute information in the database using the unique identifier as the association key.

[0047] Specifically, this step sets up two sets of standardized file naming rules to complete local file archiving, and the two naming formulas are fully reused throughout the entire process: The first set of naming rules for feature point files is: pre_name (prefix name of the original test image)_objId.txt, with a unified storage path of output / points / ; The second set is the naming convention for affine transformation diagram files: pre_name_objId.jpg, with a unified storage path of output / affine / ; Where pre_name is the prefix name of the original test image extracted from the associated field of the IPC log material, used to match the original test image; the original affine image file output by the IPC algorithm is named chn0_.yuv (YUV, luminance and chrominance color encoding format)_objId_affine_after.jpg, and the tool automatically renames it and saves it to the archive directory.

[0048] Data persistence uses TinyDB database by default, with storage files in db.json (JSON, JavaScriptObject Notation, a lightweight structured data exchange format). Equivalent alternatives exist, such as MySQL or SQLite relational databases, and standardized data tables and retrieval indexes can be created. The database tables use objId as the primary key / index field, and each data table record synchronously stores objId, pre_name, quaternion coordinate sequence, complete attributes, and the complete local storage path of the affine transformation graph.

[0049] The standardized and archived feature dataset can be repeatedly called to perform multiple rounds of regression testing, solving the shortcomings of traditional solutions where test data cannot be reused and the workload of repetitive labor is large. At the same time, the affine graph storage path is bound to the corresponding detection object and stored in the database, making up for the shortcomings of traditional solutions where affine graphs have no associated labels and cannot intuitively locate the cause of identification failure. This is also the first core innovation of this invention, which is the front-end implementation of the front-end and back-end data association mechanism based on objId.

[0050] See Figure 3 As shown, Figure 3 This document illustrates an IPC front-end testing flowchart provided in Embodiment 1 of the present invention. This flowchart corresponds to the first stage of the IPC front-end feature point extraction and archiving process of the present invention. It is executed fully automatically using the device_algorithm_plus tool. The process starts from "Start IPC Test" and sequentially completes the following steps: serial port connection and algorithm startup, log parsing to extract objId, coordinates, nvr_point, scanning the shared directory and binding the affine graph with objId, standardizing and archiving feature point files and affine graph files, and finally ending the process. The process outputs a standardized dataset stored in TinyDB to provide ground truth for NVR linkage testing. At the same time, it binds the affine graph with the detection object, making up for the shortcoming of traditional solutions that lack visual debugging associated materials.

[0051] S204: Read the feature point data and attribute information associated with the unique identifier from the database, and generate a labeling file using the coordinate sequence in the feature point data and the attribute information.

[0052] Specifically, this step relies on the IPCTool cross-database query tool to automatically read the front-end true data. The fixed condition formula for database retrieval is resp_object(detection object response data body).affine.objId == objId AND resp_object.affine.filename == pre_name.jpg; resp_object.affine.objId represents the unique identifier field of the object bound to the affine diagram in the data table, and resp_object.affine.filename represents the file name field of the affine transformation diagram in the data table.

[0053] The tool first traverses all archived feature point files under the path testcases / {app_uuid (application Unique Identifier, unique ID of the test application)} / {chip (device chip model)}_{resolution (image resolution)} / points / , and parses the filenames to extract two key parameters: pre_name and objId. Parameter explanation: app_uuid is the globally unique ID of the test application, chip is the IPC device chip model, and resolution is the test image resolution.

[0054] Then, the IPCTool cross-database query interface is called, using the split objId and pre_name as joint search conditions to retrieve the IPC test database, extract the complete attribute information and the [xmin,ymin,xmax,ymax] quaternary coordinate sequence from the matching records, and automatically assemble and generate a standardized alarms annotation file, supporting both plain text (txt) and structured JSON output formats.

[0055] The complete template for a standard JSON annotation file is as follows: [{ "bbox(bounding box, target bounding rectangle)": {"xmin": 210, "ymin": 51, "xmax": 224, "ymax": 70}, "label(target category label)": "person_001", "attributes": { "custom_name": "person_001", "side_face": 0, "face_race": "yellow" } }] Where bbox is the bounding box field of the annotation file, and label is the target category label field.

[0056] The entire annotation generation process eliminates the need for manual input of coordinates and attribute parameters, completely eliminating the time cost of manual annotation. At the same time, it ensures that the annotation data is uniform and standardized, avoiding test deviations caused by manually written parameters. This corresponds to the second core innovation of this invention: the IPCTool cross-database query automatic annotation generation method.

[0057] S205: Send the labeled file to the NVR device, drive the NVR device to perform recognition, and obtain the recognition result.

[0058] Specifically, this step involves distributing test materials and starting the NVR recognition algorithm through device_algorithm_backend (a dedicated algorithm testing tool for NVR backend). In addition to the labeled files, the affine transformation image file matching the current objId is simultaneously pushed to the NVR backend recognition component. According to the accompanying technical Q&A, the affine transformation image is not just archived material, but will directly participate in the backend automatic feature comparison and verification process.

[0059] The entire data distribution and recognition process fully replicates the logic of real security business: In real offline business, the NVR directly receives the real-time video stream from the IPC to complete face recognition. In automated testing scenarios, the coordinates, affine graphs, and ground truth attributes extracted from the IPC front end are uniformly injected into the NVR device, fully replicating the end-to-end linkage link of "IPC feature point extraction - data transmission - NVR recognition". This solves the problem that traditional independent testing can only verify a single device and cannot detect linkage failures such as incompatibility of front-end and back-end data formats and missing attribute fields in advance.

[0060] S206: Compare the identification result with the attribute information to obtain a test evaluation report.

[0061] Specifically, this step uses the complete attributes archived and stored in the database at the IPC end as the standard truth value, and uses objId as an independent comparison unit to match the NVR output recognition results item by item, defining four types of recognition judgment labels and assigning them unique judgment criteria: TP (True Positive): The NVR outputs a label and all attribute fields that perfectly match the IPC ground truth. FP (False Positive): The NVR identifies a detected object that does not exist in the ground truth output, representing a false detection by the algorithm. FN (False Negative): A face object that exists in the ground truth is not detected by the NVR, or the attribute fields identified after detection are inconsistent with the ground truth, representing a missed detection or an attribute identification error. TN (True Negative): When there is no face target to be detected in the frame, the NVR has no recognition output, representing that the algorithm correctly filters out irrelevant objects.

[0062] Based on the statistical data of four types of samples, three quantitative algorithm performance indicators—Precision, Recall, and F1-Score—are calculated respectively. Finally, the performance indicators, details of single object recognition correctness and error, and statistical content of various error cases are integrated to generate a structured test evaluation report.

[0063] Based on the complete traceability link of "feature point file - IPC database record - NVR recognition result" built by objId, it can quickly distinguish whether the identification failure is caused by abnormal feature extraction of IPC front end or defect of NVR back end recognition algorithm, which greatly shortens the fault location time of 2 to 4 hours for a single case in the traditional solution; it achieves 100% end-to-end test coverage of IPC and NVR, greatly reduces the overall test cost of equipment integration and adjustment, and supports rapid iteration and high-quality delivery of intelligent security IPC and NVR equipment.

[0064] In one feasible implementation, the feature point data includes a coordinate sequence of the bounding rectangle of the target, the coordinate sequence consisting of four values, used to select the target region of the detected object in the image; the attribute information includes the category label and attribute fields of the detected object.

[0065] Specifically, this section contains technical Q&A for this solution. The four sets of values ​​[xmin, ymin, xmax, ymax] only represent the bounding rectangle detection box of the face, which is used to mark the overall range of the face. The key points of the fine facial features refer to points such as the corners of the eyes, nose, and lips. There are far more than four of them, and they are the core comparison points for face recognition.

[0066] The execution logic of the two processes has a specific order: first, the face region is locked by the coordinates of the four-element rectangle, and then multiple fine facial feature key points are extracted within the region; the attribute information fully covers the label category labels in the JSON annotation file, as well as the multi-dimensional face description fields such as custom_name, side_face, face_race, and face_decorations, which serve as a unified benchmark for judging the correctness of the recognition results.

[0067] The step of generating a labeling file using the coordinate sequence and attribute information from the feature point data includes: The coordinate sequence is used as the bounding box field value of the annotation file, the category label is used as the label field value of the annotation file, and the attribute field is used as the attribute field value of the annotation file. An annotation file containing bounding box field, label field and attribute field is generated and output as text format or JSON format.

[0068] Specifically, this step executes a fixed field one-to-one mapping rule: the nvr_point quadrilateral coordinate sequence is directly assigned to the bbox bounding box field, the custom_name parameter is assigned the value from the label category label field, and all attribute subfields are completely assigned to the attribute object inside the annotation file.

[0069] The tool simultaneously outputs two annotation file formats: plain text (txt) and structured JSON. It is compatible with the test reading interfaces of different NVR devices and requires no additional format conversion.

[0070] In one feasible implementation plan, see Figure 4 As shown, Figure 4 The flowchart illustrates a data processing method under different testing modes provided in Embodiment 1 of the present invention. Before driving the NVR device to perform recognition, the method further includes steps S401-S403: Specifically, this section contains technical Q&A content for this solution. The entire automated testing tool is fully compatible with two types of real business workflows of NVR. It distinguishes different test modes by relying on test_type (test mode control parameter). A unified testing platform covers two scenarios: single device independent testing and front-end and back-end linkage link testing.

[0071] Compared to the traditional approach of building two separate testing tools for IPC and NVR, this solution can significantly reduce the development and maintenance costs of testing tools. Both testing modes fully cover all real-world business scenarios of the NVR, with no omissions in testing scenarios.

[0072] S401: Determine the test mode.

[0073] Specifically, test_type is the only control parameter that distinguishes the NVR test mode. Different parameter values ​​correspond to completely independent test execution logic, and testers can flexibly switch the corresponding mode according to verification needs.

[0074] S402: When the test mode is the linkage mode, the step of reading the feature point data and attribute information associated with the unique identifier from the database and generating a label file is executed, and the label file is sent to the NVR device to drive the NVR device to perform recognition based on the label file.

[0075] Specifically, the linkage mode corresponds to test_type=1, which is the core innovative application scenario of this solution. It simulates the real business process where the IPC extracts facial features and sends the coordinates to the NVR in the front end, and the NVR only completes the final feature comparison and recognition work logic. It is specifically used to verify the front-end and back-end linkage compatibility of IPC and NVR and the stability of the end-to-end link.

[0076] This model fully relies on two core innovations: a front-end and back-end data association mechanism based on objId and an IPCTool cross-database query method to automatically generate annotations, thereby achieving fully automated closed-loop end-to-end testing.

[0077] S403: When the test mode is standalone mode, the NVR device is driven to directly extract feature points from the original image to perform recognition, without reading the feature point data and attribute information in the database.

[0078] Specifically, the independent mode corresponds to test_type=0, which is the traditional independent test mode. The process does not depend on any output data from the IPC front end. The NVR device independently completes the entire process of target detection, feature extraction, comparison and recognition.

[0079] This mode is only used to verify the performance of the NVR backend's own recognition algorithm. The two modes can be flexibly switched and executed according to the testing requirements.

[0080] See Figure 5 As shown, Figure 5 This document illustrates a test flowchart for an NVR backend linkage mode provided in Embodiment 1 of the present invention. This flowchart corresponds to the complete test logic of the NVR end under the test_type=1 linkage mode, which is the core innovative application scenario of the present invention. The process starts from "Start NVR Test", selects the linkage mode, reads the feature point file of the IPC archive and parses the objId, calls IPCTool to query the IPC database across databases to obtain the complete attribute truth values, automatically generates a JSON format annotation file, drives the NVR to perform recognition and comparison and output an evaluation report, and the process ends. It fully realizes fully automatic end-to-end verification, eliminates manual annotation operations, and quickly locates front-end and back-end faults by relying on the objId association mechanism.

[0081] In one feasible implementation plan, see Figure 6 As shown, Figure 6The flowchart of a detection object data extraction method provided in Embodiment 1 of the present invention is shown. The steps of parsing the log, extracting the unique identifier of each detection object, and extracting the feature point data and corresponding attribute information of the detection object include steps S601-S603: Specifically, the entire log data extraction process is the core prerequisite for standardized archiving in the IPC front end. All the structured data output by the process serves as the truth benchmark for subsequent NVR linkage testing, and the accuracy of data extraction directly determines the accuracy of subsequent identification, comparison, and fault location.

[0082] S601: Extract the unique identifier from the target field by matching the target field in the log using a regular expression.

[0083] Specifically, the basic implementation relies on preset regular expression rules to match log text line by line, accurately locate log lines carrying the keyword objId, and quickly extract globally unique identifiers; alternatively, a deep learning intelligent log parsing model can be trained to automatically identify key log fields and adapt to vendor-customized non-standard log formats.

[0084] Both parsing methods can extract the objId parameter completely and accurately; the only difference lies in the log format scenarios they are compatible with.

[0085] S602: Extract the feature point data by parsing the feature point field in the log.

[0086] Specifically, the nvr_point field in the log stores a standardized four-element rectangle coordinate string. The tool parses the string and splits it into four independent values: xmin, ymin, xmax, and ymax, which are then encapsulated into structured bounding box data.

[0087] This structured coordinate data is the sole data source for the automatically generated bbox field of the annotation file.

[0088] S603: Extract the attribute information by parsing the attribute fields in the log.

[0089] Specifically, the testing tool parses all subfields under the root field of attributes in the log, and fully collects face dimension attribute parameters such as custom_name, side_face, face_race, and face_decorations.

[0090] All extracted attribute content is used as the standard truth value for comparing the correctness of the NVR recognition results.

[0091] In one feasible implementation plan, see Figure 7 As shown, Figure 7The flowchart illustrates a database archiving and storage method according to Embodiment 1 of the present invention, wherein the step of archiving and storing the feature point data and the attribute information in the database using the unique identifier as the association key includes steps S701 to S703: Specifically, this process completes the standardized binding and persistent storage of IPC front-end test data, and establishes the objId data traceability foundation that runs through the entire link of IPC and NVR testing. This is the key implementation process of the first core innovation of this invention.

[0092] S701: Extract the image prefix name from the log.

[0093] Specifically, the testing tool parses the pre_name parameter from the material association field in the IPC operation log. pre_name is the unified prefix name of the original test image and is a core basic parameter for splicing feature point files, affine map file storage paths, and cross-database query conditions.

[0094] S702: Store the feature point data as a feature point file using a file naming format that includes the unique identifier.

[0095] Specifically, strictly following the fixed naming rules of pre_name_objId.txt, the nvr_point quadrilateral coordinate sequence is written to a text file and uniformly archived in the output / points directory.

[0096] The file name contains a globally unique objId, enabling a one-to-one binding between local feature files and individual face detection objects.

[0097] S703: The unique identifier, the image prefix name, and the attribute information are associated and stored in a data table of the database, wherein the unique identifier is used as the primary key or index field of the data table.

[0098] Specifically, the default data storage medium is TinyDB, and the storage file is db.json; it can also be replaced with MySQL or SQLite relational databases, and dedicated search indexes can be created for the data tables.

[0099] Each database record synchronously stores the objId (primary key), pre_name, quadrilateral coordinates, complete attributes, and the complete storage path of the affine transformation graph, ultimately forming a complete data traceability link that binds the affine graph file, feature point file, and database record.

[0100] See Figure 8 As shown, Figure 8The flowchart illustrates a database reading method according to Embodiment 1 of the present invention, wherein reading the feature point data and attribute information associated with the unique identifier from the database includes steps S801-S802: Specifically, this process is the pre-data reading stage for automatically generating annotation files in the NVR linkage test mode. It reuses the front-end true data by relying on the cross-database query capability of IPCTool, which corresponds to the second core innovation of this invention.

[0101] S801: Based on the unique identifier, query the data table of the database for records that match the unique identifier, and read the attribute information from the matching records.

[0102] Specifically, during the NVR testing phase, the feature point file name is parsed to obtain the objId parameter. The IPCTool cross-database query interface is called to execute the retrieval formula resp_object.affine.objId == objId AND resp_object.affine.filename == pre_name.jpg to retrieve the database and extract the complete attributes inside the matching record, which serves as the ground value data source for automatically assembling the attribute content of the labeled file.

[0103] S802: Determine the storage path of the feature point file based on the image prefix name and the unique identifier, and read the coordinate sequence in the feature point file.

[0104] Specifically, by combining the two parameters pre_name and objId obtained from parsing and concatenating the complete file path output / points / pre_name_objId.txt, the [xmin,ymin,xmax,ymax] quadrilateral coordinate values ​​stored inside the file are read and used to fill the bounding box field in the annotation file.

[0105] In one feasible implementation, the process of driving the IPC device to execute the algorithm application includes: connecting to the IPC device via a serial port, mounting a shared directory, and starting the algorithm process of the IPC device.

[0106] Specifically, there are equivalent alternatives for serial communication links. RS232 / RS485 serial ports can be replaced with HTTP / RESTAPI or SSH remote network interfaces to enable remote device calls; NFS shared directory file transfer can be replaced with MQTT or Kafka message queues for real-time data stream transmission; and local single-machine test host deployment can be replaced with a cloud-based distributed test platform to enable parallel scheduling testing of multiple devices.

[0107] The entire process of serial port connection, shared directory mounting, and algorithm process startup can be automated in one stop using the device_algorithm_plus tool, without the need for manual step-by-step operations.

[0108] See Figure 9 As shown, Figure 9 The flowchart illustrates a storage path embedding method provided in Embodiment 1 of the present invention, wherein, after parsing the log, the method further includes steps S901-S902: Specifically, this process enables the binding of visual debugging materials with structured detection data, making up for the shortcomings of traditional solutions where affine diagrams cannot be associated with corresponding detection objects and are difficult to intuitively analyze and identify the causes of failure.

[0109] S901: Based on the unique identifier, scan the shared directory to obtain the affine transformation diagram file.

[0110] Specifically, after the IPC algorithm completes its operation, it will generate an original affine graph file in the NFS shared directory. The original file is named chn0_.yuv_objId_affine_after.jpg.

[0111] The testing tool traverses the shared directory based on a globally unique objId identifier to accurately match the affine transformation image material specific to each individual face detection object.

[0112] S902: The storage path of the affine transformation diagram file is embedded into the data structure of the corresponding detection object through the unique identifier.

[0113] Specifically, the testing tool writes the complete local storage path of the affine transformation graph into the data structure of the detection object with the corresponding objId, and synchronously persists it into the TinyDB database, realizing the one-to-one binding and association between the visual debugging image material and the structured feature coordinates and attribute data.

[0114] See Figure 10 As shown, Figure 10 The flowchart illustrates a method for driving an NVR device to perform identification according to Embodiment 1 of the present invention. The step of sending the annotation file to the NVR device and driving the NVR device to perform identification includes steps S1001-S1002: Specifically, this process fully replicates the linkage logic of IPC transmitting image and feature data to NVR in real security business, ensuring that the automated test results are consistent with the actual online business operation performance, and provides the corresponding implementation process for the technical Q&A explanation of this solution.

[0115] S1001: Send the annotation file and the affine transformation diagram file together to the NVR device.

[0116] Specifically, the affine transformation diagram is not just manually archived material. The testing tool synchronously pushes the labeled file and the affine diagram file matching the current objId to the NVR backend recognition component, fully reproducing the data transmission link of real business.

[0117] S1002: Drive the NVR device to load the annotation file and perform feature point recognition based on the affine transformation map file.

[0118] Specifically, the NVR device loads the bounding box coordinates inside the annotation file to lock the target area of ​​the face. It then extracts fine facial feature key points such as the corners of the eyes, nose, and lips within the corresponding area of ​​the accompanying affine transformation map to complete the comparison and recognition. After the recognition process is completed, it automatically outputs the complete recognition results for subsequent unified comparison and evaluation.

[0119] In one feasible implementation plan, see Figure 11 As shown, Figure 11 The flowchart illustrates a method for obtaining a test evaluation report according to Embodiment 1 of the present invention, wherein comparing the identification result with the attribute information to obtain a test evaluation report includes steps S1101 to S1107: This process is the core final step in NVR linkage testing. Using standardized attribute data archived and stored at the IPC front end as the sole benchmark, it classifies the recognition results as correct or incorrect by comparing each detected object one by one. Then, based on the classification statistics, it quantifies and calculates the algorithm's performance indicators, ultimately outputting a structured test evaluation report. The output report not only directly supports the iterative optimization of the NVR recognition algorithm but also leverages the objId end-to-end traceability capability to locate front-end and back-end compatibility issues, providing precise data for the linkage adjustment and modification of IPC and NVR devices.

[0120] S1101: Determine one by one whether the recognition result of the NVR device for each detected object is consistent with the attribute information.

[0121] Specifically, the testing tool uses `objId` as an independent comparison unit and verifies the recognition results output by the NVR against the ground truth attributes archived by the IPC front end according to a one-to-one matching rule. The verification dimensions cover category labels and all attribute fields, including matching verification of the label category label, as well as value verification of all attribute subfields such as `custom_name`, `side_face`, `face_race`, and `face_decorations`. Any mismatch in any item results in a completely consistent judgment, ensuring a rigorous comparison process and reliable results.

[0122] S1102: If the recognition result is consistent with the category label and attribute field in the attribute information, it is marked as a true example.

[0123] Specifically, a true positive (TP) is defined as a sample that simultaneously meets two core conditions: first, the label output by the NVR must completely match the true IPC value; and second, all attribute fields corresponding to the detected object must have values ​​that are completely consistent with the true values. Only samples that meet both conditions are marked as true positives, representing that the algorithm's identification of the detected object is completely accurate and serving as the core valid sample for measuring the algorithm's correct identification ability.

[0124] 1103: If the identification result contains a detection object that does not exist in the attribute information, it is marked as a false positive.

[0125] Specifically, false positives (FP) are abbreviated as False Positives, corresponding to scenarios where the algorithm misdetects: the corresponding detection object does not exist in the ground truth attribute, or the target of that category does not exist in the ground truth value of a specified objId, but the NVR outputs the corresponding recognition result. The number of false positives directly reflects the degree of false detection by the algorithm. Too many false positive samples will increase the frequency of invalid alarms in the security system and affect the actual operational efficiency of business.

[0126] S1104: If the detected object in the attribute information is not recognized by the NVR device, or the recognized attribute field is inconsistent with the attribute information, it is marked as a false negative example.

[0127] Specifically, false negatives, abbreviated as FN, cover two types of algorithmic anomalies: the first is target miss detection, where a face clearly exists in the ground truth, but the NVR does not output any corresponding recognition result; the second is attribute recognition bias, where the NVR successfully detects the target object, but the values ​​of attribute fields such as side_face and face_race in the recognition output differ from the ground truth values. Both types of scenarios are classified as false negatives, which can simultaneously reflect the algorithm's missed detection defects and insufficient attribute recognition accuracy.

[0128] S1105: If the NVR device correctly excludes non-target objects, it is marked as a true negative example.

[0129] Specifically, True Negative (TN) refers to scenarios where the algorithm correctly filters out irrelevant objects: areas of the screen where there is no corresponding target in the test cases, or target categories that do not exist in the ground truth, and the NVR does not output corresponding recognition results. The number of True Negative (TN) examples reflects the algorithm's ability to resist interference from background and irrelevant objects, indicating that the algorithm can accurately distinguish between targets and non-targets, reducing unnecessary feature extraction calculations.

[0130] S1106: Based on the statistical results of the true positives, false positives, false negatives and true negatives, calculate the evaluation metrics, which include accuracy, recall and F1 score.

[0131] Specifically, based on the statistical values ​​of four types of samples (TP, FP, FN, and TN), three core quantitative indicators are calculated: Precision, which is the percentage of true positives among all recognition results, measuring the accuracy of the algorithm's recognition results; Recall, which is the percentage of correct recognitions among all true positive targets, measuring the algorithm's ability to detect targets; and F1-Score, which is the harmonic mean of precision and recall, balancing the performance of the two indicators and more comprehensively reflecting the algorithm's overall recognition performance. These three indicators together constitute a complete quantitative evaluation system for algorithm performance.

[0132] S1107: Integrate the evaluation indicators to obtain the test evaluation report.

[0133] Specifically, the final structured evaluation report includes multi-dimensional content: First, core overall performance indicators, showcasing the quantitative results of precision, recall, and F1-Score; second, detailed single-object recognition, listing the correct and incorrect classifications, specific matching items, and error items for each detected object by objId; third, error type statistics, classifying and statistically analyzing the number and percentage of various problems such as missed detections, attribute recognition errors, and false positives; and fourth, visualized correlation information, synchronously linking the affine transformation graph storage path of the corresponding detected object. The report can intuitively and completely present the end-to-end test results, assisting R&D personnel in quickly locating the root cause of problems and carrying out algorithm optimization and front-end / back-end collaborative troubleshooting.

[0134] Example 2 See Figure 12 As shown, Figure 12 A schematic diagram of an automated testing device provided in Embodiment 2 of the present invention is shown, wherein the device includes: The log acquisition module 1201 is used to drive the IPC device to execute the algorithm application so that the IPC device outputs logs. The log parsing module 1202 is used to parse the log, extract the unique identifier of each detected object, and extract the feature point data and corresponding attribute information of the detected object; Storage module 1203 is used to archive and store the feature point data and the attribute information in a database using the unique identifier as the association key; The annotation generation module 1204 is used to read the feature point data and attribute information associated with the unique identifier from the database, and generate an annotation file using the coordinate sequence in the feature point data and the attribute information; The sending driver module 1205 is used to send the annotation file to the NVR device, drive the NVR device to perform recognition, and obtain the recognition result; The comparison output module 1206 is used to compare the recognition result with the attribute information to obtain a test evaluation report.

[0135] In one feasible implementation, the feature point data includes a coordinate sequence of the bounding rectangle of the target, the coordinate sequence consisting of four values, used to select the target region of the detected object in the image; the attribute information includes the category label and attribute fields of the detected object; The annotation generation module is specifically used for: The coordinate sequence is used as the bounding box field value of the annotation file, the category label is used as the label field value of the annotation file, and the attribute field is used as the attribute field value of the annotation file. An annotation file containing bounding box field, label field and attribute field is generated and output as text format or JSON format.

[0136] In one feasible implementation, the apparatus further includes a pattern determination module for determining a test pattern before driving the NVR device to perform identification; When the test mode is the linkage mode, the annotation generation module performs the step of reading the feature point data and attribute information associated with the unique identifier from the database and generating an annotation file, and the sending driver module sends the annotation file to the NVR device to drive the NVR device to perform recognition based on the annotation file; When the test mode is standalone mode, the sending driver module drives the NVR device to directly extract feature points from the original image to perform recognition, without reading the feature point data and attribute information in the database.

[0137] In one feasible implementation, the log parsing module is specifically used for: The unique identifier is extracted from the target field by matching the target field in the log using regular expressions. The feature point data is extracted by parsing the feature point field in the log. The attribute information is extracted by parsing the attribute fields in the log.

[0138] In one feasible implementation, the storage module is specifically used for: Extract the image prefix names from the logs; The feature point data is stored as a feature point file using a file naming format that includes the unique identifier; The unique identifier, the image prefix name, and the attribute information are associated and stored in a data table of the database, wherein the unique identifier is used as the primary key or index field of the data table; The annotation generation module is specifically used for: Based on the unique identifier, query the data table of the database for records that match the unique identifier, and read the attribute information from the matching records; Based on the image prefix name and the unique identifier, determine the storage path of the feature point file, and read the coordinate sequence in the feature point file.

[0139] In one feasible implementation, the log acquisition module is specifically used for: Connect to the IPC device via serial port, mount the shared directory, and start the algorithm process of the IPC device. The storage module is further configured to: after parsing the log, scan the shared directory to obtain the affine transformation graph file based on the unique identifier; and embed the storage path of the affine transformation graph file into the data structure of the corresponding detection object using the unique identifier; The sending driver module is specifically used for: The annotation file and the affine transformation diagram file are sent together to the NVR device; The NVR device is driven to load the annotation file and perform feature point recognition based on the affine transformation image file.

[0140] In one feasible implementation, the comparison output module is specifically used for: Each time, determine whether the NVR device's recognition result for each detected object is consistent with the attribute information; If the recognition result matches the category label and attribute field in the attribute information, it is marked as a true example; If the identification result contains a detection object that is not present in the attribute information, it is marked as a false positive. If the detected object in the attribute information is not recognized by the NVR device, or if the recognized attribute field is inconsistent with the attribute information, it is marked as a false negative example; If the NVR device correctly excludes non-target objects, it is marked as a true negative example; Based on the statistical results of the true positives, false positives, false negatives and true negatives, the evaluation metrics are calculated, including accuracy, recall and F1 score. The test evaluation report is obtained by integrating the evaluation indicators.

[0141] Example 3 Based on the same application concept, see [link / reference] Figure 13As shown, Figure 13 A schematic diagram of the structure of a computer device provided in Embodiment 3 of the present invention is shown, wherein, as Figure 13 As shown, the computer device 1300 provided in Embodiment 3 of this application includes: The computer device 1300 includes a processor 1301, a memory 1302, and a bus 1303. The memory 1302 stores machine-readable instructions that can be executed by the processor 1301. When the computer device 1300 is running, the processor 1301 and the memory 1302 communicate through the bus 1303. The machine-readable instructions are executed by the processor 1301 to perform the steps of the automated testing method shown in Embodiment 1 above.

[0142] Example 4 Based on the same application concept, embodiments of the present invention also provide a computer-readable storage medium storing a computer program, wherein the computer program, when run by a processor, performs the steps of the automated testing method described in any of the above embodiments.

[0143] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the system and apparatus described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0144] The computer program product for automated testing provided in this embodiment of the invention includes a computer-readable storage medium storing program code. The instructions included in the program code can be used to execute the methods described in the preceding method embodiments. For specific implementation details, please refer to the method embodiments, which will not be repeated here.

[0145] The automated testing device provided in this embodiment of the invention can be specific hardware on a device or software or firmware installed on the device. The implementation principle and technical effects of the device provided in this embodiment of the invention are the same as those in the foregoing method embodiments. For the sake of brevity, any parts not mentioned in the device embodiments can be referred to the corresponding content in the foregoing method embodiments. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the devices, apparatuses, and units described above can all be referred to the corresponding processes in the above method embodiments, and will not be repeated here.

[0146] In the embodiments provided by this invention, it should be understood that the disclosed apparatus and method can be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Additionally, the displayed or discussed mutual couplings, direct couplings, or communication connections may be through some communication interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms.

[0147] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0148] In addition, the functional units in the embodiments provided by the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0149] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, essentially, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0150] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. In addition, the terms "first", "second", "third", etc. are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0151] Finally, it should be noted that the above-described embodiments are merely specific implementations of the present invention, used to illustrate the technical solutions of the present invention, and not to limit it. The scope of protection of the present invention is not limited thereto. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments within the technical scope disclosed in the present invention, or make equivalent substitutions for some of the technical features; and these modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention. All should be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. An automated testing method, characterized in that, The method includes: Drive the IPC device to execute the algorithm application so that the IPC device outputs logs; The logs are parsed to extract the unique identifier of each detected object, as well as the feature point data and corresponding attribute information of the detected object. The feature point data and the attribute information are archived and stored in the database using the unique identifier as the association key; Read the feature point data and attribute information associated with the unique identifier from the database, and generate a labeling file using the coordinate sequence in the feature point data and the attribute information; The annotation file is sent to the NVR device, which then performs recognition and obtains the recognition result. The identification results are compared with the attribute information to obtain a test evaluation report.

2. The method according to claim 1, characterized in that, The feature point data includes a coordinate sequence of the bounding rectangle of the target, which consists of four values ​​and is used to select the target area of ​​the detected object in the image. The attribute information includes the category label and attribute fields of the detected object; The step of generating a labeling file using the coordinate sequence and attribute information from the feature point data includes: The coordinate sequence is used as the bounding box field value of the annotation file, the category label is used as the label field value of the annotation file, and the attribute field is used as the attribute field value of the annotation file. An annotation file containing bounding box field, label field and attribute field is generated and output as text format or JSON format.

3. The method according to claim 1, characterized in that, Before driving the NVR device to perform identification, the method further includes: Determine the test mode; When the test mode is the linkage mode, the step of reading the feature point data and attribute information associated with the unique identifier from the database and generating a label file is executed, and the label file is sent to the NVR device to drive the NVR device to perform recognition based on the label file; When the test mode is standalone mode, the NVR device is driven to directly extract feature points from the original image to perform recognition, without reading the feature point data and attribute information in the database.

4. The method according to claim 1, characterized in that, The process of parsing the logs, extracting the unique identifier of each detected object, and extracting the feature point data and corresponding attribute information of the detected object includes: The unique identifier is extracted from the target field by matching the target field in the log using regular expressions. The feature point data is extracted by parsing the feature point field in the log. The attribute information is extracted by parsing the attribute fields in the log.

5. The method according to claim 1, characterized in that, The step of archiving and storing the feature point data and the attribute information in the database using the unique identifier as the association key includes: Extract the image prefix names from the logs; The feature point data is stored as a feature point file using a file naming format that includes the unique identifier; The unique identifier, the image prefix name, and the attribute information are associated and stored in a data table of the database, wherein the unique identifier is used as the primary key or index field of the data table; The step of reading the feature point data and attribute information associated with the unique identifier from the database includes: Based on the unique identifier, query the data table of the database for records that match the unique identifier, and read the attribute information from the matching records; Based on the image prefix name and the unique identifier, determine the storage path of the feature point file, and read the coordinate sequence in the feature point file.

6. The method according to claim 1, characterized in that, The algorithm application that drives the IPC device to execute includes: Connect to the IPC device via serial port, mount the shared directory, and start the algorithm process of the IPC device. After parsing the log, the method further includes: Based on the unique identifier, scan the shared directory to obtain the affine transformation diagram file; The storage path of the affine transformation diagram file is embedded into the data structure of the corresponding detection object using the unique identifier; Sending the labeled file to the NVR device and driving the NVR device to perform recognition includes: The annotation file and the affine transformation diagram file are sent together to the NVR device; The NVR device is driven to load the annotation file and perform feature point recognition based on the affine transformation image file.

7. The method according to claim 1, characterized in that, The step of comparing the identification result with the attribute information to obtain a test evaluation report includes: Each time, determine whether the NVR device's recognition result for each detected object is consistent with the attribute information; If the recognition result matches the category label and attribute field in the attribute information, it is marked as a true example; If the identification result contains a detection object that is not present in the attribute information, it is marked as a false positive. If the detected object in the attribute information is not recognized by the NVR device, or if the recognized attribute field is inconsistent with the attribute information, it is marked as a false negative example; If the NVR device correctly excludes non-target objects, it is marked as a true negative example; Based on the statistical results of the true positives, false positives, false negatives and true negatives, the evaluation metrics are calculated, including accuracy, recall and F1 score. The test evaluation report is obtained by integrating the evaluation indicators.

8. An automated testing device, characterized in that, The device includes: The log acquisition module is used to drive the IPC device to execute algorithm applications so that the IPC device outputs logs. The log parsing module is used to parse the logs, extract the unique identifier of each detected object, and extract the feature point data and corresponding attribute information of the detected object; The storage module is used to archive and store the feature point data and the attribute information in the database using the unique identifier as the association key; The annotation generation module is used to read the feature point data and attribute information associated with the unique identifier from the database, and generate an annotation file using the coordinate sequence in the feature point data and the attribute information; The sending driver module is used to send the annotation file to the NVR device, drive the NVR device to perform recognition, and obtain the recognition result; The comparison output module is used to compare the recognition result with the attribute information to obtain a test evaluation report.

9. A computer device, characterized in that, include: The computer device includes a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the computer device is running, the processor communicates with the memory via the bus. When the machine-readable instructions are executed by the processor, they perform the steps of the automated testing method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, performs the steps of the automated testing method as described in any one of claims 1 to 7.

Images