Heterogeneous model-based comparison and measurement method and related equipment
By training multiple heterogeneous models under a unified model running environment and training samples, and using unified real-time data for comparison and testing, the problem of lack of universality and comparability of model comparison and testing in the existing technology is solved, and a more accurate and general model performance comparison is achieved.
Patent Information
- Application Number
- CN202510075548.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-01-17
- Publication Date
- 2025-04-29
AI Technical Summary
The existing model comparison method lacks universality and comparability, making it difficult to compare the performance of different models fairly and effectively in a unified environment.
By creating a unified model running environment and model training samples, multiple heterogeneous models are trained, and model comparison and measurement is performed using unified real-time data.
The accuracy and versatility of model comparison measurement are improved, and the fairness and accuracy of comparison measurement results are ensured.
Smart Images
Figure CN119902988A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the technical field of model comparison, and in particular to a method and related equipment based on heterogeneous model comparison. Background Art
[0002] With the development of artificial intelligence technology, many different models have been developed to solve various problems. However, since these models may have differences in structure, algorithm and data format, how to compare the performance of these models fairly and effectively becomes a challenge. Existing comparison methods often rely on specific hardware and software environments and lack universality and comparability. Summary of the invention
[0003] The embodiments of the present application provide a method and related equipment based on heterogeneous model comparison to achieve performance comparison of different models in a unified environment.
[0004] Other features and advantages of the present application will become apparent from the following detailed description, or may be learned in part by the practice of the present application.
[0005] According to a first aspect of an embodiment of the present application, a method based on heterogeneous model comparison is provided, comprising:
[0006] Create a model running environment and model training samples;
[0007] Obtain multiple heterogeneous models that need to be compared;
[0008] Based on the model training samples, training multiple heterogeneous models one by one in the model running environment;
[0009] Get real-time data;
[0010] Model comparison is performed on the trained heterogeneous models based on the real-time data.
[0011] In some embodiments of the present application, based on the above solution, the step of creating a model operating environment and a model training sample includes:
[0012] The model operating environment and the model training samples are created according to the requirements of the manufacturers who sign up for the comparison test.
[0013] In some embodiments of the present application, based on the aforementioned solution, the performing model comparison test on the trained model based on the real-time data includes:
[0014] Inputting the real-time data into the trained heterogeneous model for calculation;
[0015] Obtain output results from multiple heterogeneous models;
[0016] Set the comparison test parameters;
[0017] The output results of the multiple heterogeneous models are compared with the comparison parameters respectively to obtain the comparison results.
[0018] In some embodiments of the present application, based on the above solution, it also includes:
[0019] The comparison test results are reported to the manufacturer and displayed.
[0020] According to a second aspect of an embodiment of the present application, a device based on heterogeneous model comparison is provided, including:
[0021] The creation unit is used to create the model running environment and model training samples;
[0022] A first acquisition unit is used to acquire a plurality of heterogeneous models that need to be compared and tested;
[0023] A training unit, used for training a plurality of heterogeneous models one by one in the model running environment based on the model training samples;
[0024] A second acquisition unit, used for acquiring real-time data;
[0025] A comparison unit is used to perform model comparison on the trained heterogeneous models based on the real-time data.
[0026] According to a third aspect of an embodiment of the present application, a computer-readable storage medium is provided, wherein the storage medium stores computer instructions, and when the computer instructions are executed on a computer, the computer executes the method according to the first aspect.
[0027] According to a fourth aspect of the embodiments of the present application, there is provided an electronic device, including: a memory and a processor;
[0028] The memory is used to store computer instructions;
[0029] The processor is used to call the computer instructions stored in the memory so that the electronic device executes the method as described in the first aspect.
[0030] The technical solution of the present application trains multiple heterogeneous models through a unified model operating environment and model training samples, and uses unified real-time data to perform model comparison, thereby improving the accuracy and versatility of model comparison.
[0031] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS
[0032] The drawings herein are incorporated into the specification and constitute a part of the specification, showing embodiments consistent with the present application, and together with the specification, are used to explain the principles of the present application. Obviously, the drawings described below are only some embodiments of the present application, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. In the drawings:
[0033] Figure 1 A schematic flow chart of a method based on heterogeneous model comparison according to an embodiment of the present application is shown;
[0034] Figure 2 A block diagram of a device based on heterogeneous model comparison according to an embodiment of the present application is shown;
[0035] Figure 3 A block diagram of an electronic device according to an embodiment of the present application is shown;
[0036] Figure 4 A schematic diagram of the structure of a computer system suitable for implementing an electronic device of an embodiment of the present application is shown. DETAILED DESCRIPTION
[0037] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this application will be more comprehensive and complete and fully convey the concept of the example embodiments to those skilled in the art.
[0038] In addition, described feature, structure or characteristic can be combined in one or more embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present application. However, those skilled in the art will appreciate that the technical scheme of the present application can be put into practice without one or more of the specific details, or other methods, components, devices, steps, etc. can be adopted. In other cases, known methods, devices, realizations or operations are not shown or described in detail to avoid blurring the various aspects of the application.
[0039] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities may be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.
[0040] The flowcharts shown in the accompanying drawings are only exemplary and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps can be decomposed, and some operations / steps can be combined or partially combined, so the actual execution order may change according to actual conditions.
[0041] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the objects used in this way can be interchanged where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those shown or described.
[0042] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution in the embodiment of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiment of the present invention. Obviously, the described embodiment is only a part of the embodiment of the present invention, not all the embodiments. Based on the embodiment of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0043] Some embodiments of the present application will be described in detail below in conjunction with the accompanying drawings. In the absence of conflict, the following embodiments and features in the embodiments may be combined with each other.
[0044] See also Figure 1 , shows a flow chart of a method based on heterogeneous model comparison according to an embodiment of the present application.
[0045] like Figure 1 As shown, a method based on heterogeneous model comparison is presented, including steps S100 to S500.
[0046] refer to Figure 1 , step S100, creating a model running environment and model training samples.
[0047] In some feasible embodiments, based on the above solution, the step of creating a model operating environment and a model training sample includes:
[0048] The model operating environment and the model training samples are created according to the requirements of the manufacturers who sign up for the comparison test.
[0049] It is understandable that setting the model operating environment and model training samples according to the manufacturer's needs can make the model's comparison and testing closer to actual application.
[0050] It should be noted that in this application, a unified model operating environment and model training samples are created. For example, only one model operating environment and one model training sample are created, and then based on this unique model operating environment, multiple heterogeneous models are trained using the unique model training sample to ensure the fairness of the comparison. Alternatively, multiple model operating environments with unified standards and multiple model training samples with unified standards are created. No matter how the number of model operating environments and model training samples changes, it needs to be ensured that the model operating environment is unified and the model training samples are unified.
[0051] Continue to refer Figure 1 , step S200, obtaining multiple heterogeneous models that need to be compared.
[0052] Continue to refer Figure 1 , step S300, training multiple heterogeneous models one by one in the model running environment based on the model training samples.
[0053] It can be understood that in the present application, multiple heterogeneous models are trained in the same model operating environment and with the same model training samples, which ensures the fairness and accuracy of the comparison.
[0054] Continue to refer Figure 1 , step S400, obtaining real-time data.
[0055] For example, real-time video streaming data of actual UAV flight can be obtained in real time for comparison and testing, so that the comparison and evaluation of the model is closer to the actual application scenario, thereby improving the performance in practical applications.
[0056] Continue to refer Figure 1 , step S500, performing model comparison test on the trained heterogeneous models based on the real-time data.
[0057] In some feasible embodiments, based on the above solution, the performing model comparison test on the trained model based on the real-time data includes:
[0058] Inputting the real-time data into the trained heterogeneous model for calculation;
[0059] Obtain output results from multiple heterogeneous models;
[0060] Set the comparison test parameters;
[0061] The output results of the multiple heterogeneous models are compared with the comparison parameters respectively to obtain the comparison results.
[0062] In some feasible embodiments, based on the above solution, the method further includes:
[0063] The comparison test results are reported to the manufacturer and displayed.
[0064] In summary, the method based on heterogeneous model comparison provided by the present application provides a unified standard operating environment for comparison, thereby improving the versatility and accuracy of the comparison.
[0065] The following describes an apparatus embodiment of the present application, which can be used to execute a method based on heterogeneous model comparison in the above embodiment of the present application. For details not disclosed in the apparatus embodiment of the present application, please refer to the above method embodiment of the present application.
[0066] Reference Figure 2 As shown, according to an embodiment of the present application, a device 200 based on heterogeneous model comparison measurement includes:
[0067] A creation unit 201 is used to create a model operating environment and a model training sample;
[0068] A first acquisition unit 202 is used to acquire a plurality of heterogeneous models that need to be compared and tested;
[0069] A training unit 203, configured to train a plurality of heterogeneous models one by one in the model running environment based on the model training samples;
[0070] A second acquisition unit 204, used to acquire real-time data;
[0071] The comparison unit 205 is used to perform model comparison on the trained heterogeneous models based on the real-time data.
[0072] In some feasible embodiments, based on the above solution, the creating unit 201 is configured as follows:
[0073] The model operating environment and the model training samples are created according to the requirements of the manufacturers who sign up for the comparison test.
[0074] In some feasible embodiments, based on the above solution, the comparison unit 205 is configured as follows:
[0075] Inputting the real-time data into the trained heterogeneous model for calculation;
[0076] Obtain output results from multiple heterogeneous models;
[0077] Set the comparison test parameters;
[0078] The output results of the multiple heterogeneous models are compared with the comparison parameters respectively to obtain the comparison results.
[0079] like Figure 3As shown, an embodiment of the present application also provides an electronic device 300, including a memory 310, a processor 320, and a computer program 311 stored in the memory 310 and executable on the processor. When the processor 320 executes the computer program 311, the steps of the above-mentioned method based on heterogeneous model comparison are implemented.
[0080] Since the electronic device introduced in this embodiment is a device used to implement a device based on heterogeneous model comparison in the embodiment of the present application, based on the method introduced in the embodiment of the present application, technical personnel in this field can understand the specific implementation mode of the electronic device of this embodiment and its various variations. Therefore, how the electronic device implements the method in the embodiment of the present application is not introduced in detail here. As long as the equipment adopted by technical personnel in this field to implement the method in the embodiment of the present application is within the scope of protection of this application.
[0081] During the specific implementation process, when the computer program 311 is executed by a processor, any implementation method in the embodiments corresponding to the first aspect can be implemented.
[0082] Figure 4 A schematic diagram of the structure of a computer system suitable for implementing an electronic device of an embodiment of the present application is shown.
[0083] It should be noted that Figure 4 The computer system 400 of the electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.
[0084] like Figure 4 As shown, the computer system 400 includes a central processing unit (CPU) 401, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 402 or the program loaded from the storage part 408 to the random access memory (RAM) 403, such as executing the method described in the above embodiment. In the RAM 403, various programs and data required for system operation are also stored. The CPU 401, ROM 402 and RAM 403 are connected to each other through a bus 404. An input / output (I / O) interface 405 is also connected to the bus 404.
[0085] The following components are connected to the I / O interface 405: an input section 406 including a keyboard, a mouse, etc.; an output section 407 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 408 including a hard disk, etc.; and a communication section 409 including a network interface card such as a LAN (Local Area Network) card, a modem, etc. The communication section 409 performs communication processing via a network such as the Internet. A drive 410 is also connected to the I / O interface 405 as needed. A removable medium 411, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 410 as needed so that a computer program read therefrom is installed into the storage section 408 as needed.
[0086] In particular, according to an embodiment of the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication section 409, and / or installed from a removable medium 411. When the computer program is executed by a central processing unit (CPU) 401, various functions defined in the system of the present application are executed.
[0087] It should be noted that the computer-readable medium shown in the embodiment of the present application may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, - but not limited to - an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by an instruction execution system, device or device or used in combination with it. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, wherein a computer-readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which may send, propagate, or transmit programs for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.
[0088] The flowchart and block diagram in the accompanying drawings illustrate the possible architecture, functions and operations of the system, method and computer program product according to various embodiments of the present application. Wherein, each box in the flowchart or block diagram can represent a module, a program segment, or a part of the code, and the above-mentioned module, program segment, or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.
[0089] The units involved in the embodiments described in this application may be implemented by software or hardware, and the units described may also be set in a processor. The names of these units do not, in some cases, constitute limitations on the units themselves.
[0090] As another aspect, the present application also provides a computer program product or a computer program, which includes a computer instruction stored in a computer-readable storage medium. A processor of a computer device reads the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device executes a method based on heterogeneous model comparison described in the above embodiment.
[0091] As another aspect, the present application also provides a computer-readable medium, which may be included in the electronic device described in the above embodiment; or may exist independently without being assembled into the electronic device. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed by an electronic device, the electronic device implements a method based on heterogeneous model comparison described in the above embodiment.
[0092] It should be noted that, although several modules or units of the equipment for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present application, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into being embodied by multiple modules or units.
[0093] Through the description of the above implementation methods, it is easy for those skilled in the art to understand that the example implementation methods described here can be implemented by software or by combining software with necessary hardware. Therefore, the technical solution according to the implementation methods of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which can be a personal computer, a server, a touch terminal, or a network device, etc.) to execute the method according to the implementation methods of the present application.
[0094] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the embodiments disclosed herein. The present application is intended to cover any variations, uses or adaptations of the present application, which follow the general principles of the present application and include common knowledge or customary techniques in the art that are not disclosed in the present application. It should be understood that the present application is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.
Claims
1. A method based on heterogeneous model comparison, characterized in that: include: Create a model running environment and model training samples; Obtain multiple heterogeneous models that need to be compared; Based on the model training samples, training multiple heterogeneous models one by one in the model running environment; Get real-time data; Model comparison and testing are performed on the trained heterogeneous models based on the real-time data.
2. The method according to claim 1, characterized in that The step of creating a model operating environment and a model training sample includes: The model operating environment and the model training samples are created according to the requirements of the manufacturers who sign up for the comparison test.
3. The method according to claim 1, characterized in that The performing model comparison test on the trained model based on the real-time data includes: Inputting the real-time data into the trained heterogeneous model for calculation; Obtain output results from multiple heterogeneous models; Set the comparison test parameters; The output results of the multiple heterogeneous models are compared with the comparison parameters respectively to obtain the comparison results.
4. The method according to claim 1, characterized in that: Also includes: The comparison test results are reported to the manufacturer and displayed.
5. A device based on heterogeneous model comparison, characterized in that: include: The creation unit is used to create the model running environment and model training samples; A first acquisition unit is used to acquire a plurality of heterogeneous models that need to be compared and tested; A training unit, used for training a plurality of heterogeneous models one by one in the model running environment based on the model training samples; A second acquisition unit, used for acquiring real-time data; A comparison unit is used to perform model comparison on the trained heterogeneous models based on the real-time data.
6. A computer-readable storage medium, characterized in that: The storage medium stores computer instructions, and when the computer instructions are executed on a computer, the computer is enabled to execute the method according to any one of claims 1 to 4.
7. An electronic device, characterized in that: include: Memory and processor; The memory is used to store computer instructions; The processor is used to call the computer instructions stored in the memory so that the electronic device executes the method as described in any one of claims 1-4.