Camera AI function test method and device and storage medium

Abstract

The embodiment of the invention provides a camera AI function test method and device and a storage medium, and belongs to the technical field of data processing. The method comprises the steps that each piece of test sample video data in a test sample set is acquired, and the test sample video data comprises sample labeling information; performing a camera AI function simulation test based on each piece of test sample video data to obtain detection data corresponding to each piece of test sample video data; according to the detection data and the sample labeling information, obtaining a corresponding MIOU index, and determining a test result corresponding to each piece of test sample video data; and generating a test report of the camera AI function according to a test result corresponding to each test sample video data. According to the technical scheme provided by the embodiment of the invention, the efficiency and accuracy of camera AI function testing are considered.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a camera AI function testing method, device and storage medium. Background Art

[0002] With the development of science and technology, some AI (Artificial Intelligence) cameras have AI functions such as human detection and electric vehicle recognition, which can be applied to various corresponding scenarios. At present, the AI ​​function of cameras is generally tested manually, which is inefficient. There are also automated tests by directing the video source to a TV or display screen, but playing the video source through the display screen will introduce the light of the display screen itself, making the test scene different from the actual use scene, resulting in inaccurate test results.

[0003] Therefore, how to achieve both efficiency and accuracy in camera AI function testing has become an urgent problem to be solved. Summary of the invention

[0004] The main purpose of the embodiments of the present application is to provide a camera AI function testing method, device and storage medium, aiming to achieve both efficiency and accuracy of camera AI function testing.

[0005] In a first aspect, an embodiment of the present application provides a camera AI function testing method, the method comprising:

[0006] Acquire each test sample video data in the test sample set, wherein the test sample video data includes sample annotation information;

[0007] Perform a camera AI function simulation test based on each of the test sample video data to obtain detection data corresponding to each of the test sample video data;

[0008] According to the detection data and the sample annotation information, a corresponding mean intersection over union (MIOU) index is obtained, and a test result corresponding to each of the test sample video data is determined;

[0009] Generate a test report of the camera AI function based on the test results corresponding to each of the test sample video data.

[0010] In a second aspect, an embodiment of the present application provides a computer device, the computer device comprising:

[0011] A processor, a memory, a computer program stored in the memory and executable by the processor, and a data bus for realizing connection and communication between the processor and the memory, wherein when the computer program is executed by the processor, the steps of the camera AI function testing method as described above are realized.

[0012] In a third aspect, an embodiment of the present application provides a storage medium for computer-readable storage, wherein the storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps of the camera AI function testing method as described above.

[0013] The embodiments of the present application provide a camera AI function testing method, device and storage medium. Based on a test sample set, by obtaining each test sample video data in the test sample set, wherein the test sample video data includes sample annotation information, a camera AI function simulation test is performed based on each test sample video data, and the detection data corresponding to each test sample video data is obtained. Then, according to the detection data and the sample annotation information, the corresponding Mean Intersection Union (MIOU) index is obtained, and the test result corresponding to each test sample video data is determined. According to the test results corresponding to each test sample video data, a test report of the camera AI function is generated. Compared with manual testing or directing the video source to a display screen for automated testing, not only the efficiency of the camera AI function testing is improved, but also the accuracy of the camera AI function testing is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0015] Figure 1 A flowchart of a camera AI function testing method provided by an embodiment of the present invention;

[0016] Figure 2 A schematic diagram of a process for obtaining video data of each test sample in a test sample set provided by an embodiment of the present invention;

[0017] Figure 3 A schematic diagram of a process for performing a camera AI function simulation test based on each of the test sample video data to obtain detection data corresponding to each of the test sample video data provided by an embodiment of the present invention;

[0018] Figure 4A schematic diagram of a simulation test system for a camera AI function provided by an embodiment of the present invention;

[0019] Figure 5 A schematic diagram of a process for obtaining a corresponding mean intersection union (MIOU) indicator provided by an embodiment of the present invention;

[0020] Figure 6 A schematic diagram of the position of a person marked for a certain video frame of a test sample video data;

[0021] Figure 7 for Figure 6 Schematic diagram of the position of the detected person corresponding to the video frame;

[0022] Figure 8 A schematic diagram of the overlapping area corresponding to the position of the marked person and the position of the detected person;

[0023] Fig. 9 A schematic diagram of the merged area corresponding to the position of the marked person and the position of the detected person;

[0024] Fig.10 A schematic block diagram of the structure of a computer device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0025] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0026] The flowcharts shown in the accompanying drawings are only examples and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may also be decomposed, combined or partially merged, so the actual execution order may change according to actual conditions.

[0027] It should be understood that the terms used in this application specification are only for the purpose of describing specific embodiments and are not intended to limit the application. As used in this application specification and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0028] At present, the AI ​​function of cameras is generally tested manually, which is inefficient. There is also an automated test by directing the video source to a TV or display. However, playing the video source through the display will introduce the light of the display screen itself, making the test scene very different from the actual usage scene, resulting in inaccurate test results.

[0029] In order to solve the above problems, the embodiments of the present invention provide a camera AI function testing method, device and storage medium, aiming to achieve both efficiency and accuracy of camera AI function testing.

[0030] Some embodiments of the present invention are described in detail below in conjunction with the accompanying drawings. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0031] Please refer to Figure 1 , Figure 1 A flowchart of a camera AI function test method provided by an embodiment of the present invention. The camera AI function test method can be applied in a computer device, or in other devices such as a camera test device and a server, to achieve both efficiency and accuracy of the camera AI function test.

[0032] like Figure 1 As shown, the camera AI function testing method includes steps S101 to S104.

[0033] S101. Obtain each test sample video data in a test sample set, wherein the test sample video data includes sample annotation information.

[0034] Among them, the camera AI function includes but is not limited to the camera human detection function, the camera vehicle recognition function, etc. The sample annotation information of the test sample video data includes but is not limited to the annotation position of the target object, the annotation time when the target object enters the screen, etc. For the camera human detection function test, the target object is a person; for the camera vehicle recognition function test, the target object is a vehicle.

[0035] Taking the human figure detection function test of the camera as an example, first, multiple test sample video data required for the human figure detection function test of the camera are obtained. Exemplarily, a test sample set corresponding to the human figure detection function test of the camera is generated in advance, and the test sample set includes multiple test sample video data for the human figure detection function test of the camera. Among them, the sample annotation information of each test sample video data in the test sample set includes the time when the person in the sample enters the screen, and the person's position coordinates and other information.

[0036] In some embodiments, Figure 2As shown, step S101 may include sub-step S1011 and sub-step S1012.

[0037] S1011, reading the test sample set from a test database;

[0038] S1012: Perform data format conversion on each test sample video data in the test sample set based on a preset resolution to obtain the test sample video data after format conversion.

[0039] Still taking the camera human detection function test as an example, in order to generate a test sample set, a video duration threshold is pre-set. Based on the preset video duration threshold, videos of various human scenes can be captured from the local video of the prototype. The duration of each captured video is the video duration threshold, and the videos of various human scenes with a duration of the video duration threshold are used as original test samples. Among them, the local video source of the prototype can be a video of a real person walking in front of the prototype lens according to the AI ​​special test scenario, recorded and saved. For example, a video is recorded and automatically saved in a TF (Trans-flash Card) card.

[0040] For example, the video duration threshold is preset to 15 seconds, and 15 seconds of videos of various human-shaped scenes are captured from the local video of the prototype, and each 15-second video is used as an original test sample.

[0041] It should be noted that the specific value of the video length threshold can be flexibly set according to actual conditions and is not specifically limited in this application.

[0042] Afterwards, each original test sample is annotated, including but not limited to annotating the time when the person in the original test sample enters the screen, annotating the position coordinates of the person in the original test sample, etc., to generate the test sample video data corresponding to each original test sample. After obtaining each test sample video data, the test sample video data is aggregated to generate a test sample set, and the test sample set is placed in the test database directory for storage.

[0043] When the human detection function test of the camera is required, it is only necessary to call the test database and read the saved test sample set from the test database. In order to use the test sample video data in the test sample set smoothly, the test sample video data is converted into a data format from the original format to an internal data stream format of the camera to be tested with a preset resolution, and the format-converted test sample video data is obtained and saved, for example, the format-converted test sample video data is saved in the TF card directory of the camera to be tested.

[0044] For example, the preset resolution is set to 1280*720, and the test sample video data is converted from the original format to a 1280*720 YUV420SP (NV12) frame file to obtain the test sample video data after the format conversion.

[0045] It should be noted that the specific value of the preset resolution can be flexibly set according to actual conditions and is not specifically limited in this application.

[0046] S102: Perform a camera AI function simulation test based on each of the test sample video data to obtain detection data corresponding to each of the test sample video data.

[0047] After obtaining the test sample video data, inject the test sample video data into the service channel of the device with the injection port opened, perform simulation test on the camera AI function, perform simulation test based on each test sample video data, and obtain the detection data corresponding to the test sample video data. Taking the camera human detection function test as an example, turn on the human detection switch of the device with the injection port opened through the script and restart the device to implement the camera human detection function test.

[0048] Exemplarily, the detection data includes, but is not limited to, the detection position of the target object corresponding to the test sample video data, the alarm information, and the time when the alarm information is triggered. In the human detection function test scenario of the camera, the alarm information refers to the human detection alarm information, and the detection position of the target object refers to the position coordinates of the detected person.

[0049] In some embodiments, Figure 3 As shown, step S102 may include sub-steps S1021 to S1023.

[0050] S1021. In the camera simulation mode, control sample switching to sequentially read each video frame of each test sample video data, inject each video frame of each test sample video data based on a preset channel, and perform a camera AI function simulation test;

[0051] S1022. Read the log corresponding to the camera AI function simulation test;

[0052] S1023. Obtain the detection data based on the log.

[0053] For example, Figure 4As shown, the simulation test system of the camera AI function includes the camera product side and the device platform side, wherein the camera product side is responsible for the interface timing call of the chip chip, completing the creation of video, audio and other channels; and copying the audio and video frame data in the user-state memory to the system kernel-state MMA (Memory Management App) memory, and placing it in the queue of the corresponding underlying module to complete the video frame injection. The device platform side is responsible for the logical part of the simulation management module, realizing the normal function and simulation function of the camera product, the function of dynamic switching of samples during the simulation process, and the function of reading the corresponding video frame data. Specifically, mode switching mainly manages the switching between simulation mode and real-time mode, and channel establishment; sample switching mainly manages the switching of test sample video data in simulation mode; reading frame data refers to reading the test sample video data from the test sample file.

[0054] During the camera AI function simulation test, a log corresponding to the camera AI function simulation test is generated. The detection data corresponding to each test sample video data is obtained by reading the log. Exemplarily, the log can be recorded through a serial port or Telnet, and the detection data corresponding to each test sample video data can be read from the log.

[0055] S103: Obtain a corresponding Mean Intersection Over Union (MIOU) index based on the detection data and the sample annotation information, and determine a test result corresponding to each of the test sample video data.

[0056] The detection data obtained from the log is matched with the sample annotation information of the corresponding test sample video data by algorithm to obtain the corresponding MIOU (Mean Intersection over Union) index, and the test results corresponding to the test sample video data are determined based on the MIOU index.

[0057] In some embodiments, Figure 5 As shown, step S103 may include sub-step S1031 and sub-step S1032.

[0058] S1031, matching the marked position of the target object corresponding to each video frame of the test sample video data with the detection position to obtain an intersection-over-union (IOU) index corresponding to each video frame;

[0059] S1032. Obtain the MIOU indicator according to each of the IOU indicators.

[0060] For each video frame of the test sample video data, the marked position of the target object corresponding to each video frame is matched and calculated with the detection position read from the log to obtain the IOU (Intersection over Union) index corresponding to each video frame. For example, still taking the camera human detection function test as an example, for each video frame of the test sample video data, the position coordinates of the marked person corresponding to each video frame are matched and calculated with the position coordinates of the person read from the log to obtain the IOU index corresponding to the video frame.

[0061] In some embodiments, matching the marked position of the target object corresponding to each video frame of the test sample video data with the detection position to obtain an intersection-over-union (IOU) index corresponding to each video frame includes:

[0062] Determine the overlapping area of ​​the marked position and the detected position of the target object corresponding to each of the video frames, and the combined area of ​​the marked position and the detected position of the target object corresponding to each of the video frames;

[0063] The ratio of the overlapping area to the merged area is calculated, and the ratio is determined as the IOU index corresponding to each of the video frames.

[0064] For a certain video frame of the test sample video data, according to the marked position of the target object and the detected position of the target object obtained by the simulation test, the overlapping area S1 of the marked position and the detected position of the target object and the combined area S2 of the marked position and the detected position of the target object are determined. Then, the IOU index corresponding to the video frame is calculated according to the following formula:

[0065] IOU=S1 / S2

[0066] For example, still taking the camera human detection function test as an example, for a certain video frame of the test sample video data, for example, Figure 6 As shown, the position coordinates of the marked person are at the position marked by the quadrilateral box in the figure. When performing simulation tests, for the video frame, for example, Figure 7 As shown, the position of the detected person is the position marked by the quadrilateral box in the figure. According to the position of the marked person and the position of the detected person, the overlapping area of ​​the two positions is determined, for example, Figure 8 As shown, and determine the overlapping area S1. And determine the merged area of ​​the two positions, for example, as Fig. 9 As shown, the merged area S2 is determined. Then, the formula IOU=S1 / S2 is substituted to calculate the IOU index corresponding to the video frame.

[0067] For each video frame, the corresponding IOU index can be calculated in the above manner, thereby obtaining multiple IOU indexes corresponding to multiple video frames of each test sample video data, and based on the multiple IOU indexes, the corresponding MIOU index is obtained.

[0068] In some embodiments, obtaining the MIOU indicator according to each of the IOU indicators includes:

[0069] Accumulate and sum each of the IOU indicators to obtain a corresponding accumulated value;

[0070] Determine the number of target object categories corresponding to each video frame of the test sample video data;

[0071] The accumulated value is divided by the number of target object categories to obtain the MIOU index.

[0072] After obtaining multiple IOU indicators corresponding to each test sample video data, the multiple IOU indicators are accumulated and summed to obtain the corresponding accumulated value sum(IOU). In addition, the number of target object categories corresponding to all video frames of each test sample video data is determined. For example, if there are 3 people in a video frame, the number of human categories corresponding to the video frame is 3. The number of target object categories corresponding to all video frames is accumulated and summed to obtain the total number of target object categories. Then sum(IOU) is divided by the total number of target object categories to calculate the MIOU indicator.

[0073] The MIOU index corresponding to each test sample video data can be obtained by the above method. For example, an MIOU index table can be generated according to the MIOU index corresponding to each test sample video data. For example, the generated MIOU index table is shown in Table 1 below:

[0074] Table 1

[0075]

[0076]

[0077] In some embodiments, determining the test result corresponding to each of the test sample video data includes:

[0078] If the detection data and the sample annotation information meet the preset matching condition, determining that the test result corresponding to the test sample video data is passed;

[0079] If the detection data and the sample annotation information do not meet the preset matching condition, it is determined that the test result corresponding to the test sample video data is failed.

[0080] Exemplarily, the detection data and the sample annotation information meet the preset matching condition including at least one of the following:

[0081] The time difference between the time when the alarm information is triggered and the marked time is less than or equal to the preset time length;

[0082] The MIOU indicator is greater than or equal to a preset indicator threshold.

[0083] The preset time difference corresponds to the preset reference duration, and the MIOU indicator corresponds to the preset reference indicator threshold. For example, the preset duration is preset to 1 second, and the preset indicator threshold is preset to 0.5. It should be noted that the specific values ​​of the preset duration and the preset indicator threshold can be flexibly set according to actual conditions, and are not specifically limited in this application.

[0084] For each test sample video data, the time of triggering the alarm information obtained by the simulation test is compared with the marked time of the target object entering the screen corresponding to the test sample video data, and the time difference between the time of triggering the alarm information and the marked time of the target object entering the screen is determined. The time difference between the time of triggering the alarm information and the marked time of the target object entering the screen is compared with the preset duration. In addition, the obtained MIOU index is compared with the preset index threshold. If the time difference between the time of triggering the alarm information and the marked time of the target object entering the screen is less than or equal to the preset duration, and / or the MIOU index is greater than or equal to the preset index threshold, then the test result corresponding to the test sample video data is determined to be passed; otherwise, the test result corresponding to the test sample video data is determined to be failed.

[0085] For example, taking the video data stream in Table 1 as an example, for video data stream 1, the corresponding MIOU index is 0.672, which is greater than the preset index threshold of 0.5, and the test result corresponding to video data stream 1 is determined to be passed. For video data stream 2, the corresponding MIOU index is 0.638, which is greater than the preset index threshold of 0.5, and the test result corresponding to video data stream 2 is determined to be passed. For video data stream 3, the corresponding MIOU index is 0.481, which is less than the preset index threshold of 0.5, and the test result corresponding to video data stream 3 is determined to be failed.

[0086] S104: Generate a test report of the camera AI function according to the test results corresponding to each of the test sample video data.

[0087] Exemplarily, the test report of the camera AI function includes, but is not limited to, the test results corresponding to each test sample video data, the camera AI function test accuracy, and other information. For example, still taking the camera human detection function test as an example, a camera human detection function test report is generated, and the test report includes the test results corresponding to each test sample video data, and the camera human detection function test accuracy, and other information.

[0088] Exemplarily, a test report of the camera AI function is output. For example, the test report of the camera AI function is displayed on a display interface, or the test report of the camera AI function is sent to a related terminal device.

[0089] In the above embodiment, by obtaining each test sample video data in the test sample set, wherein the test sample video data includes sample annotation information, a camera AI function simulation test is performed based on each test sample video data, and the detection data corresponding to each test sample video data is obtained. Then, according to the detection data and the sample annotation information, the corresponding Mean Intersection Union (MIOU) indicator is obtained, and the test result corresponding to each test sample video data is determined. According to the test results corresponding to each test sample video data, a test report of the camera AI function is generated. Compared with performing manual testing or directing the video source to a display screen for automated testing, not only the efficiency of the camera AI function test is improved, but also the accuracy of the camera AI function test is improved.

[0090] The embodiment of the present invention also provides a computer device, see Fig.10 , Fig.10 It is a schematic block diagram of a computer device provided in one embodiment of the present application.

[0091] like Fig.10 As shown, the computer device 200 may include a processor 210 and a memory 220, wherein the processor 210 and the memory 220 are connected via a bus, such as an I2C (Inter-integrated Circuit) bus.

[0092] Specifically, the processor 210 may be a micro-controller unit (MCU), a central processing unit (CPU), or a digital signal processor (DSP).

[0093] Specifically, the memory 220 may be a Flash chip, a read-only memory (ROM) disk, an optical disk, a USB disk, or a mobile hard disk, etc. The memory 220 stores various computer programs for the processor 210 to execute.

[0094] The processor 210 is used to run a computer program stored in the memory, and implements the following steps when executing the computer program:

[0095] Acquire each test sample video data in the test sample set, wherein the test sample video data includes sample annotation information;

[0096] Perform a camera AI function simulation test based on each of the test sample video data to obtain detection data corresponding to each of the test sample video data;

[0097] According to the detection data and the sample annotation information, a corresponding mean intersection over union (MIOU) index is obtained, and a test result corresponding to each of the test sample video data is determined;

[0098] Generate a test report of the camera AI function based on the test results corresponding to each of the test sample video data.

[0099] In some embodiments, the sample annotation information includes the annotation position of the target object, and the detection data includes the detection position of the target object. When the processor 210 obtains the corresponding mean intersection and union (MIOU) indicator according to the detection data and the sample annotation information, it is used to implement:

[0100] Matching the marked position of the target object corresponding to each video frame of the test sample video data with the detection position to obtain an intersection-over-union (IOU) index corresponding to each video frame;

[0101] According to each of the IOU indicators, the MIOU indicator is obtained.

[0102] In some embodiments, when the processor 210 matches the marked position of the target object corresponding to each video frame of the test sample video data with the detection position to obtain the intersection-and-union (IOU) index corresponding to each video frame, it is used to implement:

[0103] Determine the overlapping area of ​​the marked position and the detected position of the target object corresponding to each of the video frames, and the combined area of ​​the marked position and the detected position of the target object corresponding to each of the video frames;

[0104] The ratio of the overlapping area to the merged area is calculated, and the ratio is determined as the IOU index corresponding to each of the video frames.

[0105] In some embodiments, when the processor 210 implements the obtaining of the MIOU indicator according to each of the IOU indicators, it is configured to implement:

[0106] Accumulate and sum each of the IOU indicators to obtain a corresponding accumulated value;

[0107] Determine the number of target object categories corresponding to each video frame of the test sample video data;

[0108] The accumulated value is divided by the number of target object categories to obtain the MIOU index.

[0109] In some embodiments, when the processor 210 performs the camera AI function simulation test based on each of the test sample video data and obtains the detection data corresponding to each of the test sample video data, it is used to implement:

[0110] In the camera simulation mode, the sample switching is controlled to sequentially read each video frame of the test sample video data, and each video frame of the test sample video data is injected based on the preset channel to perform a camera AI function simulation test;

[0111] Read the log corresponding to the camera AI function simulation test;

[0112] The detection data is acquired based on the log.

[0113] In some embodiments, when determining the test result corresponding to each of the test sample video data, the processor 210 is used to implement:

[0114] If the detection data and the sample annotation information meet the preset matching condition, determining that the test result corresponding to the test sample video data is passed;

[0115] If the detection data and the sample annotation information do not meet the preset matching condition, it is determined that the test result corresponding to the test sample video data is failed.

[0116] In some embodiments, the detection data includes alarm information and the time when the alarm information is triggered, the sample annotation information includes the annotation time when the target object enters the screen, and the detection data and the sample annotation information meet the preset matching conditions including at least one of the following:

[0117] The time difference between the time when the alarm information is triggered and the marked time is less than or equal to the preset time length;

[0118] The MIOU indicator is greater than or equal to a preset indicator threshold.

[0119] In some embodiments, when the processor 210 implements the acquisition of each test sample video data in the test sample set, it is configured to implement:

[0120] Read the test sample set from a test database;

[0121] The data format of each of the test sample video data in the test sample set is converted based on a preset resolution to obtain the test sample video data after the format conversion.

[0122] An embodiment of the present application also provides a storage medium for computer-readable storage, wherein the storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps of any camera AI function testing method provided in the description of the embodiment of the present application.

[0123] The storage medium may be an internal storage unit of the computer device described in the above embodiment, such as a hard disk or memory of the computer device. The storage medium may also be an external storage device of the computer device, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the computer device.

[0124] It will be appreciated by those skilled in the art that all or some of the steps, systems, and functional modules / units in the methods disclosed above may be implemented as software, firmware, hardware, and appropriate combinations thereof. In a hardware embodiment, the division between the functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed by several physical components in cooperation. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or implemented as hardware, or implemented as an integrated circuit, such as an application-specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include a computer storage medium (or non-transitory medium) and a communication medium (or temporary medium). As known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and can be accessed by a computer. In addition, it is well known to those of ordinary skill in the art that communication media typically contain computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.

[0125] It should be understood that the term "and / or" used in the present specification and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, including these combinations. It should be noted that, in this article, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "including a..." does not exclude the presence of other identical elements in the process, method, article or system including the element.

[0126] The serial numbers of the embodiments of the present application are only for description and do not represent the advantages and disadvantages of the embodiments. The above are only specific embodiments of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present application, and these modifications or replacements should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A camera artificial intelligence (AI) function testing method, the method comprising: Acquire each test sample video data in the test sample set, wherein the test sample video data includes sample annotation information; Perform a camera AI function simulation test based on each of the test sample video data to obtain detection data corresponding to each of the test sample video data; According to the detection data and the sample annotation information, a corresponding mean intersection over union (MIOU) index is obtained, and a test result corresponding to each of the test sample video data is determined; Generate a test report of the camera AI function based on the test results corresponding to each of the test sample video data.

2. The method according to claim 1, characterized in that The sample annotation information includes the annotation position of the target object, the detection data includes the detection position of the target object, and obtaining the corresponding Mean Intersection Over Union (MIOU) index according to the detection data and the sample annotation information includes: Matching the marked position of the target object corresponding to each video frame of the test sample video data with the detection position to obtain an intersection-over-union (IOU) index corresponding to each video frame; According to each of the IOU indicators, the MIOU indicator is obtained.

3. The method according to claim 2, characterized in that The matching of the marked position of the target object corresponding to each video frame of the test sample video data with the detection position to obtain an intersection-over-union (IOU) index corresponding to each video frame includes: Determine the overlapping area of ​​the marked position and the detected position of the target object corresponding to each of the video frames, and the combined area of ​​the marked position and the detected position of the target object corresponding to each of the video frames; The ratio of the overlapping area to the merged area is calculated, and the ratio is determined as the IOU index corresponding to each of the video frames.

4. The method according to claim 2, characterized in that: The step of obtaining the MIOU indicator according to each of the IOU indicators includes: Accumulate and sum each of the IOU indicators to obtain a corresponding accumulated value; Determine the number of target object categories corresponding to each video frame of the test sample video data; The accumulated value is divided by the number of target object categories to obtain the MIOU index.

5. The method according to claim 1, characterized in that The performing of a camera AI function simulation test based on each of the test sample video data to obtain detection data corresponding to each of the test sample video data includes: In the camera simulation mode, the sample switching is controlled to sequentially read each video frame of the test sample video data, and each video frame of the test sample video data is injected based on the preset channel to perform a camera AI function simulation test; Read the log corresponding to the camera AI function simulation test; The detection data is acquired based on the log.

6. The method according to claim 1, characterized in that The determining of the test result corresponding to each of the test sample video data comprises: If the detection data and the sample annotation information meet the preset matching condition, determining that the test result corresponding to the test sample video data is passed; If the detection data and the sample annotation information do not meet the preset matching condition, it is determined that the test result corresponding to the test sample video data is failed.

7. The method according to claim 6, characterized in that The detection data includes alarm information and the time when the alarm information is triggered, the sample annotation information includes the annotation time when the target object enters the screen, and the detection data and the sample annotation information meet the preset matching condition including at least one of the following: The time difference between the time when the alarm information is triggered and the marked time is less than or equal to the preset time length; The MIOU indicator is greater than or equal to a preset indicator threshold.

8. The method according to any one of claims 1 to 7, characterized in that: The step of obtaining video data of each test sample in the test sample set includes: Read the test sample set from a test database; The data format of each of the test sample video data in the test sample set is converted based on a preset resolution to obtain the test sample video data after the format conversion.

9. A computer device, characterized in that: The computer device comprises: A processor, a memory, a computer program stored in the memory and executable by the processor, and a data bus for realizing connection and communication between the processor and the memory, wherein when the computer program is executed by the processor, the steps of the camera AI function testing method as described in any one of claims 1 to 8 are implemented.

10. A storage medium for computer-readable storage, characterized in that: The storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps of the camera AI function testing method as described in any one of claims 1 to 8.

Images