3C equipment AI detection effect evaluation method and system and medium

Abstract

The invention provides a 3C equipment AI detection effect evaluation method and system and a medium. The method comprises the steps of obtaining AI detection data of 3C equipment, processing the AI detection data to obtain a visual detection effect evaluation index and a performance detection effect evaluation index, performing AI detection effect testing under multiple concurrent tasks, processing the AI detection data to obtain a multi-task response time evaluation factor and a resource utilization efficiency evaluation factor, and testing the performance of the 3C equipment according to the multi-task response time evaluation factor and the resource utilization efficiency evaluation factor. Obtaining a visual detection effect evaluation index and a performance detection effect evaluation index under multiple concurrent tasks, performing processing to obtain a multi-task detection effect evaluation factor, and performing processing according to a multi-task response time evaluation factor, the multi-task detection effect evaluation factor and a resource utilization efficiency evaluation factor to obtain an expandability evaluation index. Performing AI detection effect judgment according to the visual detection effect evaluation index, the performance detection effect evaluation index and the expandability evaluation index; therefore, the purpose of evaluating the AI detection effect of the 3C equipment is achieved.

Description

Technical Field

[0001] This application relates to the technical field of AI detection effect evaluation. Specifically, it relates to a method, system, and medium for evaluating the AI detection effect of 3C devices. Background Art

[0002] With the rapid development of artificial intelligence (AI) technology, its application in the field of 3C device detection is becoming more and more extensive. 3C products refer to computers (Computer), communications (Communication), and consumer electronics (Consumer Electronic). Due to their high technical content and rapid product updates, the requirements for product quality and performance detection are becoming increasingly strict. Traditional 3C device detection methods often rely on manual operations, which are not only inefficient but also easily affected by human factors, making it difficult to ensure the accuracy and reliability of detection results. Existing evaluation technologies often only focus on single-dimensional detection data, such as only emphasizing visual detection results or only considering performance detection indicators, and cannot comprehensively and integrally reflect the overall effect of AI detection. On the other hand, 3C devices often need to handle multiple tasks simultaneously in actual usage scenarios, while existing evaluation methods rarely consider the performance of AI detection under multi-concurrent tasks. Therefore, it is urgent to develop a method that can comprehensively, accurately, and effectively evaluate the AI detection effect of 3C devices to provide strong support for the quality control and performance improvement of 3C devices. Summary of the Invention

[0003] The purpose of this application is to provide a method, system, and medium for evaluating the AI detection effect of 3C devices, which can comprehensively evaluate the AI detection effect by evaluating the AI visual detection effect, performance detection effect, and scalability detection effect of 3C devices.

[0004] This application also provides a method for evaluating the AI detection effect of 3C devices, including the following steps: Obtain the AI detection data of 3C devices, including AI visual detection data and AI performance detection data; Process the AI visual detection data to obtain a visual detection effect evaluation index, and process the AI performance detection data to obtain a performance detection effect evaluation index; Conduct an AI detection effect test under multi-concurrent tasks to obtain scalability test data, and process the scalability test data to obtain a multi-task response time evaluation factor; Process the scalability test data to obtain a resource utilization efficiency evaluation factor, obtain the visual detection effect evaluation index and performance detection effect evaluation index under multi-concurrent tasks, and process them to obtain a multi-task detection effect evaluation factor; Process to obtain a scalability evaluation index according to the multi-task response time evaluation factor, multi-task detection effect evaluation factor, and resource utilization efficiency evaluation factor; Determine the AI detection effect according to the visual detection effect evaluation index, performance detection effect evaluation index, and scalability evaluation index.

[0005] Optionally, in the 3C device AI detection effect evaluation method described in this application, the acquisition of AI detection data for the 3C device, including AI visual detection data and AI performance detection data, includes: The AI visual detection data includes the surface defect detection rate, shape abnormality detection rate, color abnormality detection rate, and assembly integrity detection rate; The AI performance detection data includes the hardware performance abnormality detection rate, screen performance abnormality detection rate, battery performance abnormality detection rate, and communication performance abnormality detection rate.

[0006] Optionally, in the 3C device AI detection effect evaluation method described in this application, the process of obtaining a visual detection effect evaluation index according to the AI visual detection data and obtaining a performance detection effect evaluation index according to the AI performance detection data includes: Process according to the surface defect detection rate, shape abnormality detection rate, color abnormality detection rate, and assembly integrity detection rate to obtain a visual detection effect evaluation index; Process according to the hardware performance abnormality detection rate, screen performance abnormality detection rate, battery performance abnormality detection rate, and communication performance abnormality detection rate to obtain a performance detection effect evaluation index.

[0007] Optionally, in the 3C device AI detection effect evaluation method described in this application, the process of performing an AI detection effect test under multi-concurrent tasks to obtain scalability test data and processing the scalability test data to obtain a multi-task response time evaluation factor includes: The scalability test data includes the single-task average response duration, multi-task average response duration, response time standard deviation, throughput, CPU occupancy rate, and memory occupancy rate; Process according to the single-task average response duration, multi-task average response duration, and response time standard deviation to obtain a multi-task response time evaluation factor.

[0008] Optionally, in the 3C device AI detection effect evaluation method described in this application, the process of obtaining a resource utilization efficiency evaluation factor according to the scalability test data, obtaining a visual detection effect evaluation index and a performance detection effect evaluation index under multi-concurrent tasks, and processing to obtain a multi-task detection effect evaluation factor includes: Process according to the throughput, CPU occupancy rate, and memory occupancy rate to obtain a resource utilization efficiency evaluation factor; Obtain the visual detection effect evaluation index and the performance detection effect evaluation index under multi-concurrent tasks, and process them in combination with the visual detection effect evaluation index and the performance detection effect evaluation index under single-task to obtain the multi-task detection effect evaluation factor.

[0009] Optionally, in the 3C device AI detection effect evaluation method described in this application, the obtaining of the scalability evaluation index by processing according to the multi-task response time evaluation factor, the multi-task detection effect evaluation factor, and the resource utilization efficiency evaluation factor includes: Input the multi-task response time evaluation factor, the multi-task detection effect evaluation factor, and the resource utilization efficiency evaluation factor into a preset scalability evaluation model for processing to obtain the scalability evaluation index.

[0010] Optionally, in the 3C device AI detection effect evaluation method described in this application, the determination of the AI detection effect according to the visual detection effect evaluation index, the performance detection effect evaluation index, and the scalability evaluation index includes: Process the visual detection effect evaluation index, the performance detection effect evaluation index, and the scalability evaluation index to obtain the comprehensive AI detection effect evaluation index; Compare the comprehensive AI detection effect evaluation index with a preset comprehensive AI detection effect evaluation index threshold. If the threshold comparison result does not meet the requirements of the preset threshold comparison result, determine that the AI detection effect is poor.

[0011] In a second aspect, this application provides a 3C device AI detection effect evaluation system, which includes: a memory and a processor. A program for the 3C device AI detection effect evaluation method is stored in the memory. When the program for the 3C device AI detection effect evaluation method is executed by the processor, the following steps are implemented: Obtain the AI detection data of the 3C device, including AI visual detection data and AI performance detection data; Process the AI visual detection data to obtain the visual detection effect evaluation index, and process the AI performance detection data to obtain the performance detection effect evaluation index; Conduct an AI detection effect test under multi-concurrent tasks to obtain scalability test data, and process the scalability test data to obtain the multi-task response time evaluation factor; Process the scalability test data to obtain the resource utilization efficiency evaluation factor, obtain the visual detection effect evaluation index and the performance detection effect evaluation index under multi-concurrent tasks, and process to obtain the multi-task detection effect evaluation factor; Process the multi-task response time evaluation factor, the multi-task detection effect evaluation factor, and the resource utilization efficiency evaluation factor to obtain the scalability evaluation index; Determine the AI detection effect according to the visual detection effect evaluation index, the performance detection effect evaluation index, and the scalability evaluation index.

[0012] Optionally, in the 3C device AI detection effect evaluation system described in this application, the obtaining of the AI detection data of the 3C device, including AI visual detection data and AI performance detection data, includes: The AI visual detection data includes the surface defect detection rate, the shape abnormality detection rate, the color abnormality detection rate, and the assembly integrity detection rate; The AI performance detection data includes the hardware performance abnormality detection rate, the screen performance abnormality detection rate, the battery performance abnormality detection rate, and the communication performance abnormality detection rate.

[0013] In a third aspect, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a program for the 3C device AI detection effect evaluation method. When the program for the 3C device AI detection effect evaluation method is executed by a processor, the steps of the 3C device AI detection effect evaluation method described in any one of the above are implemented.

[0014] As can be seen from the above, a 3C device AI detection effect evaluation method, system, and medium provided by this application evaluate the AI visual detection effect, performance detection effect, and scalability detection effect of the 3C device, so as to achieve the purpose of comprehensively evaluating the AI detection effect.

[0015] Other features and advantages of this application will be described in the subsequent specification, and part of them will become obvious from the specification, or will be understood by implementing the embodiments of this application. The objectives and other advantages of this application can be achieved and obtained through the structures specifically pointed out in the written specification and the drawings. Description of the Drawings

[0016] In order to more clearly illustrate the technical solutions of the embodiments of this application, the following will briefly introduce the drawings required to be used in the embodiments of this application. It should be understood that the following drawings only show some embodiments of this application, and therefore should not be regarded as limiting the scope. For those of ordinary skill in the art, other related drawings can be obtained based on these drawings without creative efforts.

[0017] Figure 1 It is a flowchart of the 3C device AI detection effect evaluation method provided by the embodiment of this application; Figure 2 It is a flowchart of obtaining the visual detection effect evaluation index and the performance detection effect evaluation index of the 3C device AI detection effect evaluation method provided by the embodiment of this application; Figure 3Flowchart for obtaining the multi-task response time evaluation factor in the 3C device AI detection effect evaluation method provided by the embodiments of the present application; Figure 4 Flowchart for obtaining the resource utilization efficiency evaluation factor and the multi-task detection effect evaluation factor in the 3C device AI detection effect evaluation method provided by the embodiments of the present application. Detailed implementation manners

[0018] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application usually described and illustrated in the accompanying drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the present application claimed, but only represents the selected embodiments of the present application. All other embodiments obtained by those skilled in the art based on the embodiments of the present application without creative efforts belong to the scope of protection of the present application.

[0019] It should be noted that similar reference numerals and letters indicate similar items in the following drawings. Therefore, once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings. At the same time, in the description of the present application, the terms "first", "second", etc. are only used for distinguishing descriptions and cannot be understood as indicating or implying relative importance.

[0020] Please refer to Figure 1 , Figure 1 which is a flowchart of the 3C device AI detection effect evaluation method in some embodiments of the present application. The 3C device AI detection effect evaluation method is used in terminal devices, such as computer, mobile phone terminal, etc. The 3C device AI detection effect evaluation method includes the following steps: S11. Obtain the AI detection data of the 3C device, including AI vision detection data and AI performance detection data; S12. Process the AI vision detection data to obtain a vision detection effect evaluation index, and process the AI performance detection data to obtain a performance detection effect evaluation index; S13. Conduct an AI detection effect test under multi-concurrent tasks to obtain scalability test data, and process the scalability test data to obtain a multi-task response time evaluation factor; S14. Process the scalability test data to obtain a resource utilization efficiency evaluation factor, obtain the vision detection effect evaluation index and the performance detection effect evaluation index under multi-concurrent tasks, and process them to obtain a multi-task detection effect evaluation factor; S15. Obtain a scalability evaluation index based on the multi-task response time evaluation factor, the multi-task detection effect evaluation factor, and the resource utilization efficiency evaluation factor; S16. Determine the AI detection effect based on the visual detection effect evaluation index, the performance detection effect evaluation index, and the scalability evaluation index.

[0021] It should be noted that, in order to comprehensively, accurately, and effectively evaluate the AI detection effect of 3C devices, this application conducts quantitative evaluations on AI visual detection data, AI performance detection data, and scalability test data respectively to obtain evaluation results for the AI visual detection effect, performance detection effect, and scalability detection effect, thereby achieving the purpose of comprehensively evaluating the AI detection effect.

[0022] According to an embodiment of the present invention, the acquisition of AI detection data of 3C devices, including AI visual detection data and AI performance detection data, includes: The AI visual detection data includes the surface defect detection rate, the shape abnormality detection rate, the color abnormality detection rate, and the assembly integrity detection rate; The AI performance detection data includes the hardware performance abnormality detection rate, the screen performance abnormality detection rate, the battery performance abnormality detection rate, and the communication performance abnormality detection rate.

[0023] It should be noted that, in order to evaluate the visual detection effect of AI, it can be obtained by quantitatively evaluating the surface defect detection rate, the shape abnormality detection rate, the color abnormality detection rate, and the assembly integrity detection rate of 3C devices. In order to evaluate the performance detection effect of AI, it can be obtained by quantitatively evaluating the hardware performance abnormality detection rate, the screen performance abnormality detection rate, the battery performance abnormality detection rate, and the communication performance abnormality detection rate of 3C devices.

[0024] Please refer to Figure 2 , Figure 2 is a flowchart for obtaining the visual detection effect evaluation index and the performance detection effect evaluation index of the 3C device AI detection effect evaluation method in some embodiments of this application. According to an embodiment of the present invention, the visual detection effect evaluation index is obtained by processing the AI visual detection data, and the performance detection effect evaluation index is obtained by processing the AI performance detection data, including: S21. Process the surface defect detection rate, the shape abnormality detection rate, the color abnormality detection rate, and the assembly integrity detection rate to obtain the visual detection effect evaluation index; S22. Process the hardware performance abnormality detection rate, the screen performance abnormality detection rate, the battery performance abnormality detection rate, and the communication performance abnormality detection rate to obtain the performance detection effect evaluation index.

[0025] It should be noted that the calculation formula for the visual detection effect evaluation index is as follows: ; Wherein, is the visual detection effect evaluation index, , , and are the detection rates of hardware performance anomalies, screen performance anomalies, battery performance anomalies, and communication performance anomalies respectively, , , and are preset characteristic coefficients (which can be obtained by querying the preset AI detection effect evaluation platform database).

[0026] The calculation formula for the performance detection effect evaluation index is as follows: ; Wherein, is the performance detection effect evaluation index, , , and are the detection rates of hardware performance anomalies, screen performance anomalies, battery performance anomalies, and communication performance anomalies respectively, , , and are preset characteristic coefficients (which can be obtained by querying the preset AI detection effect evaluation platform database).

[0027] Please refer to Figure 3 , Figure 3 is a flowchart for obtaining the multi-task response time evaluation factor of the 3C device AI detection effect evaluation method in some embodiments of the present application. According to the embodiments of the present invention, the AI detection effect is tested under multi-concurrent tasks to obtain scalability test data, and the multi-task response time evaluation factor is obtained by processing the scalability test data, including: S31. The scalability test data includes the single-task average response duration, multi-task average response duration, response time standard deviation, throughput, CPU occupancy rate, and memory occupancy rate; S32. The multi-task response time evaluation factor is obtained by processing the single-task average response duration, multi-task average response duration, and response time standard deviation.

[0028] It should be noted that when testing the AI detection effect under multiple concurrent tasks, the multi-task response time evaluation factor is obtained by processing the single-task average response time, the multi-task average response time, and the standard deviation of the response time. The standard deviation of the response time refers to the standard deviation of the response times of multiple tasks.

[0029] The calculation formula for the multi-task response time evaluation factor is: ; where is the multi-task response time evaluation factor, , and are the single-task average response time, the multi-task average response time, and the standard deviation of the response time respectively, and are preset characteristic coefficients (which can be obtained by querying the database of the preset AI detection effect evaluation platform).

[0030] Please refer to Figure 4 , Figure 4 which is a flowchart for obtaining the resource utilization efficiency evaluation factor and the multi-task detection effect evaluation factor of the AI detection effect evaluation method for 3C devices in some embodiments of the present application. According to the embodiments of the present invention, the resource utilization efficiency evaluation factor is obtained by processing the scalability test data, the visual detection effect evaluation index and the performance detection effect evaluation index under multiple concurrent tasks are obtained, and the multi-task detection effect evaluation factor is obtained, including: S41. Obtain the resource utilization efficiency evaluation factor according to the throughput, CPU occupancy rate, and memory occupancy rate; S42. Obtain the visual detection effect evaluation index and the performance detection effect evaluation index under multiple concurrent tasks, and process them in combination with the visual detection effect evaluation index and the performance detection effect evaluation index under a single task to obtain the multi-task detection effect evaluation factor.

[0031] It should be noted that the calculation formula for the resource utilization efficiency evaluation factor is: ; where is the resource utilization efficiency evaluation factor, , and are the throughput, CPU occupancy rate, and memory occupancy rate respectively, , and are preset characteristic coefficients (which can be obtained by querying the database of the preset AI detection effect evaluation platform).

[0032] The calculation formula for the multi-task detection effect evaluation factor is: ; Among them, is the multi-task detection effect evaluation factor, and are respectively the visual detection effect evaluation index and the performance detection effect evaluation index under multi-concurrent tasks, and are respectively the visual detection effect evaluation index and the performance detection effect evaluation index under single-task, and are preset feature coefficients (which can be obtained by querying the preset AI detection effect evaluation platform database).

[0033] According to the embodiments of the present invention, obtaining the scalability evaluation index by processing the multi-task response time evaluation factor, the multi-task detection effect evaluation factor and the resource utilization efficiency evaluation factor includes: Inputting the multi-task response time evaluation factor, the multi-task detection effect evaluation factor and the resource utilization efficiency evaluation factor into a preset scalability evaluation model for processing to obtain the scalability evaluation index.

[0034] It should be noted that the calculation formula of the scalability evaluation model is: ; Among them, is the scalability evaluation index, , and are respectively the multi-task response time evaluation factor, the multi-task detection effect evaluation factor and the resource utilization efficiency evaluation factor, , and are preset feature coefficients (which can be obtained by querying the preset AI detection effect evaluation platform database).

[0035] According to the embodiments of the present invention, determining the AI detection effect according to the visual detection effect evaluation index, the performance detection effect evaluation index and the scalability evaluation index includes: Obtaining the comprehensive evaluation index of the AI detection effect by processing the visual detection effect evaluation index, the performance detection effect evaluation index and the scalability evaluation index; Comparing the comprehensive evaluation index of the AI detection effect with the preset threshold of the comprehensive evaluation index of the AI detection effect. If the threshold comparison result does not meet the requirements of the preset threshold comparison result, an unsatisfactory AI detection effect determination is made.

[0036] It should be noted that the calculation formula of the comprehensive evaluation index of the AI detection effect is: ; Among them, is the comprehensive evaluation index for the AI detection effect, , and are respectively the evaluation index for the visual detection effect, the evaluation index for the performance detection effect, and the evaluation index for scalability, , and are preset weight coefficients (which can be obtained by querying the preset AI detection effect evaluation platform database).

[0037] The present invention also discloses an AI detection effect evaluation system for 3C devices, including a memory and a processor. A program for the AI detection effect evaluation method for 3C devices is stored in the memory. When the program for the AI detection effect evaluation method for 3C devices is executed by the processor, the following steps are implemented: Obtain the AI detection data of the 3C device, including AI visual detection data and AI performance detection data; Process the AI visual detection data to obtain the evaluation index for the visual detection effect, and process the AI performance detection data to obtain the evaluation index for the performance detection effect; Conduct an AI detection effect test under multi-concurrent tasks to obtain scalability test data, and process the scalability test data to obtain a multi-task response time evaluation factor; Process the scalability test data to obtain a resource utilization efficiency evaluation factor, obtain the evaluation index for the visual detection effect and the evaluation index for the performance detection effect under multi-concurrent tasks, and process to obtain a multi-task detection effect evaluation factor; Process the multi-task response time evaluation factor, the multi-task detection effect evaluation factor, and the resource utilization efficiency evaluation factor to obtain the evaluation index for scalability; Determine the AI detection effect based on the evaluation index for the visual detection effect, the evaluation index for the performance detection effect, and the evaluation index for scalability.

[0038] It should be noted that, in order to comprehensively, accurately, and effectively evaluate the AI detection effect of 3C devices, this application quantifies and evaluates the AI visual detection data, AI performance detection data, and scalability test data respectively to obtain the evaluation results for the AI visual detection effect, performance detection effect, and scalability detection effect, so as to achieve the purpose of comprehensively evaluating the AI detection effect.

[0039] According to the embodiments of the present invention, the obtaining of the AI detection data of the 3C device, including AI visual detection data and AI performance detection data, includes: The AI visual detection data includes the surface defect detection rate, the shape abnormality detection rate, the color abnormality detection rate, and the assembly integrity detection rate; The AI performance detection data includes the detection rate of hardware performance anomalies, the detection rate of screen performance anomalies, the detection rate of battery performance anomalies, and the detection rate of communication performance anomalies.

[0040] It should be noted that in order to evaluate the visual detection effect of the AI, it can be obtained by quantitatively evaluating the detection rate of surface defects, the detection rate of shape anomalies, the detection rate of color anomalies, and the detection rate of assembly integrity of 3C devices. In order to evaluate the performance detection effect of the AI, it can be obtained by quantitatively evaluating the detection rate of hardware performance anomalies, the detection rate of screen performance anomalies, the detection rate of battery performance anomalies, and the detection rate of communication performance anomalies of 3C devices.

[0041] According to an embodiment of the present invention, the visual detection effect evaluation index is obtained by processing the AI visual detection data, and the performance detection effect evaluation index is obtained by processing the AI performance detection data, including: Processing according to the detection rate of surface defects, the detection rate of shape anomalies, the detection rate of color anomalies, and the detection rate of assembly integrity to obtain a visual detection effect evaluation index; Processing according to the detection rate of hardware performance anomalies, the detection rate of screen performance anomalies, the detection rate of battery performance anomalies, and the detection rate of communication performance anomalies to obtain a performance detection effect evaluation index.

[0042] It should be noted that the calculation formula of the visual detection effect evaluation index is: ; Wherein, is the visual detection effect evaluation index, , , and are respectively the detection rate of hardware performance anomalies, the detection rate of screen performance anomalies, the detection rate of battery performance anomalies, and the detection rate of communication performance anomalies, , , and are preset characteristic coefficients (which can be obtained by querying the preset AI detection effect evaluation platform database).

[0043] The calculation formula of the performance detection effect evaluation index is: ; Wherein, is the performance detection effect evaluation index, , , and are respectively the detection rate of hardware performance anomalies, the detection rate of screen performance anomalies, the detection rate of battery performance anomalies, and the detection rate of communication performance anomalies, , , and are preset feature coefficients (which can be obtained by querying the database of the preset AI detection effect evaluation platform).

[0044] According to the embodiments of the present invention, for the AI detection effect test under multi-concurrent tasks to obtain scalability test data, and processing the scalability test data to obtain a multi-task response time evaluation factor, including: The scalability test data includes the average response duration of a single task, the average response duration of multi-tasks, the standard deviation of response time, throughput, CPU occupancy rate, and memory occupancy rate; Processing the average response duration of a single task, the average response duration of multi-tasks, and the standard deviation of response time to obtain a multi-task response time evaluation factor.

[0045] It should be noted that for the AI detection effect test under multi-concurrent tasks, processing the average response duration of a single task, the average response duration of multi-tasks, and the standard deviation of response time to obtain a multi-task response time evaluation factor, where the standard deviation of response time refers to the standard deviation of the response times of multiple tasks.

[0046] The calculation formula of the multi-task response time evaluation factor is: ; where is the multi-task response time evaluation factor, , and are respectively the average response duration of a single task, the average response duration of multi-tasks, and the standard deviation of response time, and are preset feature coefficients (which can be obtained by querying the database of the preset AI detection effect evaluation platform).

[0047] According to the embodiments of the present invention, for processing the scalability test data to obtain a resource utilization efficiency evaluation factor, obtaining a visual detection effect evaluation index and a performance detection effect evaluation index under multi-concurrent tasks, and processing to obtain a multi-task detection effect evaluation factor, including: Processing the throughput, CPU occupancy rate, and memory occupancy rate to obtain a resource utilization efficiency evaluation factor; Obtaining a visual detection effect evaluation index and a performance detection effect evaluation index under multi-concurrent tasks, and combining the visual detection effect evaluation index and the performance detection effect evaluation index under a single task to process and obtain a multi-task detection effect evaluation factor.

[0048] It should be noted that the calculation formula of the resource utilization efficiency evaluation factor is: ; Among them, is the resource utilization efficiency evaluation factor, , and are throughput, CPU occupancy rate, and memory occupancy rate respectively, , and are preset feature coefficients (which can be obtained by querying the database of the preset AI detection effect evaluation platform).

[0049] The calculation formula of the multi-task detection effect evaluation factor is: ; Among them, is the multi-task detection effect evaluation factor, and are the visual detection effect evaluation index and performance detection effect evaluation index under multi-concurrent tasks respectively, and are the visual detection effect evaluation index and performance detection effect evaluation index under single-task respectively, and are preset feature coefficients (which can be obtained by querying the database of the preset AI detection effect evaluation platform).

[0050] According to the embodiments of the present invention, the scalability evaluation index obtained by processing according to the multi-task response time evaluation factor, multi-task detection effect evaluation factor, and resource utilization efficiency evaluation factor includes: Input the multi-task response time evaluation factor, multi-task detection effect evaluation factor, and resource utilization efficiency evaluation factor into a preset scalability evaluation model for processing to obtain the scalability evaluation index.

[0051] It should be noted that the calculation formula of the scalability evaluation model is: ; Among them, is the scalability evaluation index, , and are the multi-task response time evaluation factor, multi-task detection effect evaluation factor, and resource utilization efficiency evaluation factor respectively, , and are preset feature coefficients (which can be obtained by querying the database of the preset AI detection effect evaluation platform).

[0052] According to the embodiments of the present invention, the determination of the AI detection effect according to the visual detection effect evaluation index, performance detection effect evaluation index, and scalability evaluation index includes: Process to obtain a comprehensive evaluation index of AI detection effect based on the visual detection effect evaluation index, performance detection effect evaluation index, and scalability evaluation index; Compare the comprehensive evaluation index of AI detection effect with a preset threshold of the comprehensive evaluation index of AI detection effect. If the threshold comparison result does not meet the requirements of the preset threshold comparison result, determine that the AI detection effect is poor.

[0053] It should be noted that the calculation formula for the comprehensive evaluation index of AI detection effect is: ; Wherein, is the comprehensive evaluation index of AI detection effect, , and are the visual detection effect evaluation index, performance detection effect evaluation index, and scalability evaluation index respectively, , and are preset weight coefficients (which can be obtained by querying the database of the preset AI detection effect evaluation platform).

[0054] The third aspect of the present invention provides a readable storage medium, in which a program for the 3C device AI detection effect evaluation method is stored. When the program for the 3C device AI detection effect evaluation method is executed by a processor, the steps of the 3C device AI detection effect evaluation method as described in any one of the above are implemented.

[0055] A 3C device AI detection effect evaluation method, system, and medium disclosed by the present invention evaluate the AI visual detection effect, performance detection effect, and scalability detection effect of 3C devices, so as to achieve the purpose of comprehensively evaluating the AI detection effect.

[0056] In several embodiments provided by the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are only illustrative. For example, the division of the units is only a logical function division, and there may be other division methods in actual implementation, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored, or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed with each other can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical, or other forms.

[0057] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units; they may be located in one place or distributed across multiple network units; some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0058] In addition, each functional unit in the embodiments of the present invention may be fully integrated into a processing unit, or each unit may be separately regarded as a unit, or two or more units may be integrated into one unit; the above-mentioned integrated unit may be implemented in the form of hardware, or in the form of a combination of hardware and software functional units.

[0059] Those of ordinary skill in the art can understand that all or part of the steps of implementing the above method embodiments can be completed by hardware related to program instructions. The foregoing program can be stored in a readable storage medium. When the program is executed, it performs the steps including the above method embodiments; and the foregoing storage medium includes: removable storage devices, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), magnetic disks or optical discs and other various media that can store program codes.

[0060] Alternatively, if the above-mentioned integrated unit of the present invention is implemented in the form of a software functional module and sold or used as an independent product, it can also be stored in a readable storage medium. Based on such an understanding, the technical solution of the embodiments of the present invention essentially or the part that contributes to the prior art can be embodied in the form of a software product. The software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the methods described in the embodiments of the present invention. And the foregoing storage medium includes: removable storage devices, ROM, RAM, magnetic disks or optical discs and other various media that can store program codes.

Claims

1. A 3C device AI detection effect evaluation method, characterized in that: The following steps are involved: Obtain AI detection data of 3C devices, including AI visual detection data and AI performance detection data; Obtain a visual inspection effect evaluation index according to the AI ​​visual inspection data processing, and obtain a performance inspection effect evaluation index according to the AI ​​performance inspection data processing; Conduct AI detection effect tests under multiple concurrent tasks to obtain scalability test data, and obtain multi-task response time evaluation factors based on scalability test data processing; Processing the scalability test data to obtain a resource utilization efficiency evaluation factor, obtaining a visual inspection effect evaluation index and a performance inspection effect evaluation index under multiple concurrent tasks, and processing to obtain a multi-task inspection effect evaluation factor; Obtaining a scalability evaluation index according to the multi-task response time evaluation factor, the multi-task detection effect evaluation factor and the resource utilization efficiency evaluation factor; The AI ​​detection effect is determined based on the visual detection effect evaluation index, the performance detection effect evaluation index and the scalability evaluation index.

2. The 3C device AI detection effect evaluation method according to claim 1, characterized in that: The acquisition of AI detection data of 3C devices, including AI visual detection data and AI performance detection data, includes: The AI ​​visual inspection data includes surface defect detection rate, shape abnormality detection rate, color abnormality detection rate and assembly integrity detection rate; The AI ​​performance detection data includes hardware performance anomaly detection rate, screen performance anomaly detection rate, battery performance anomaly detection rate and communication performance anomaly detection rate.

3. The 3C device AI detection effect evaluation method according to claim 2, characterized in that: The obtaining of a visual inspection effect evaluation index according to the AI ​​visual inspection data processing and the obtaining of a performance inspection effect evaluation index according to the AI ​​performance inspection data processing include: Processing is performed according to the surface defect detection rate, shape abnormality detection rate, color abnormality detection rate and assembly integrity detection rate to obtain a visual inspection effect evaluation index; Processing is performed according to the hardware performance anomaly detection rate, screen performance anomaly detection rate, battery performance anomaly detection rate and communication performance anomaly detection rate to obtain a performance detection effect evaluation index.

4. The 3C device AI detection effect evaluation method according to claim 3, characterized in that: The AI ​​detection effect test is performed under multiple concurrent tasks to obtain scalability test data, and a multi-task response time evaluation factor is obtained according to the scalability test data processing, including: The scalability test data includes single-task average response time, multi-task average response time, response time standard deviation, throughput, CPU occupancy, and memory occupancy; A multi-task response time evaluation factor is obtained according to the single-task average response time, the multi-task average response time and the response time standard deviation.

5. The 3C device AI detection effect evaluation method according to claim 4, characterized in that: The processing according to the scalability test data to obtain a resource utilization efficiency evaluation factor, obtaining a visual detection effect evaluation index and a performance detection effect evaluation index under multiple concurrent tasks, and processing to obtain a multi-task detection effect evaluation factor includes: Obtaining a resource utilization efficiency evaluation factor according to the throughput, CPU occupancy rate and memory occupancy rate; The visual inspection effect evaluation index and the performance inspection effect evaluation index under multiple concurrent tasks are obtained, and the multi-task inspection effect evaluation factor is obtained by combining the visual inspection effect evaluation index and the performance inspection effect evaluation index under a single task.

6. The 3C device AI detection effect evaluation method according to claim 5, characterized in that: The step of obtaining a scalability evaluation index according to the multi-task response time evaluation factor, the multi-task detection effect evaluation factor and the resource utilization efficiency evaluation factor includes: The multi-task response time evaluation factor, the multi-task detection effect evaluation factor and the resource utilization efficiency evaluation factor are input into a preset scalability evaluation model for processing to obtain a scalability evaluation index.

7. The 3C device AI detection effect evaluation method according to claim 6, characterized in that: The AI ​​detection effect determination according to the visual detection effect evaluation index, the performance detection effect evaluation index and the scalability evaluation index includes: Obtaining an AI detection effect comprehensive evaluation index according to the visual detection effect evaluation index, the performance detection effect evaluation index and the scalability evaluation index; The AI ​​detection effect comprehensive evaluation index is compared with a preset AI detection effect comprehensive evaluation index threshold. If the threshold comparison result does not meet the preset threshold comparison result requirement, a poor AI detection effect determination is made.

8. A 3C device AI detection effect evaluation system, characterized in that: The method comprises a memory and a processor, wherein the memory stores a program of a 3C device AI detection effect evaluation method, and when the program of the 3C device AI detection effect evaluation method is executed by the processor, the following steps are implemented: Obtain AI detection data of 3C devices, including AI visual detection data and AI performance detection data; Obtain a visual inspection effect evaluation index according to the AI ​​visual inspection data processing, and obtain a performance inspection effect evaluation index according to the AI ​​performance inspection data processing; Conduct AI detection effect tests under multiple concurrent tasks to obtain scalability test data, and obtain multi-task response time evaluation factors based on scalability test data processing; Processing the scalability test data to obtain a resource utilization efficiency evaluation factor, obtaining a visual inspection effect evaluation index and a performance inspection effect evaluation index under multiple concurrent tasks, and processing to obtain a multi-task inspection effect evaluation factor; Obtaining a scalability evaluation index according to the multi-task response time evaluation factor, the multi-task detection effect evaluation factor and the resource utilization efficiency evaluation factor; The AI ​​detection effect is determined based on the visual detection effect evaluation index, the performance detection effect evaluation index and the scalability evaluation index.

9. The 3C device AI detection effect evaluation system according to claim 8, characterized in that: The acquisition of AI detection data of 3C devices, including AI visual detection data and AI performance detection data, includes: The AI ​​visual inspection data includes surface defect detection rate, shape abnormality detection rate, color abnormality detection rate and assembly integrity detection rate; The AI ​​performance detection data includes hardware performance anomaly detection rate, screen performance anomaly detection rate, battery performance anomaly detection rate and communication performance anomaly detection rate.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a 3C device AI detection effect evaluation program, and when the 3C device AI detection effect evaluation program is executed by the processor, the steps of the 3C device AI detection effect evaluation method according to any one of claims 1 to 7 are implemented.

Images