Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Model evaluation method and device and electronic equipment

A model and predictive model technology, applied in the field of machine learning, can solve the problems of increasing the number of online models, affecting the effect of online services, and long model evaluation cycles, achieving the effect of reducing the number of online times, reducing the impact, and low development costs

Pending Publication Date: 2021-11-26
ALIBABA SINGAPORE HLDG PTE LTD
View PDF6 Cites 1 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

This results in a longer model evaluation cycle and higher development costs
Moreover, after the model is deployed online, if the evaluation result of the model is not good, the model needs to be taken offline for iterative training before going online, which will increase the number of times the model goes online and affect the effect of online services

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Model evaluation method and device and electronic equipment
  • Model evaluation method and device and electronic equipment
  • Model evaluation method and device and electronic equipment

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0059] In this embodiment, a model evaluation method is provided. The method may be implemented by an electronic device. The electronic device can be as Figure 2a server 1100 as shown, or as Figure 2b Terminal device 1200 is shown.

[0060] according to image 3 As shown, the model evaluation method in this embodiment may include the following steps S3100-S3500:

[0061] Step S3100, acquiring a first processing result obtained by processing a preset target task by the first model running online.

[0062] Step S3200, acquiring a second processing result obtained by processing the target task by the second model to be deployed online.

[0063] In this embodiment, the mapping relationships reflected by the first model and the second model are the same, that is, the functions of the first model and the second model are the same. For example, both the first model and the second model are used to recommend a plurality of alternative routes provided in advance. Specifically, ...

Embodiment 2

[0081]On the basis of the aforementioned method embodiment 1, the target index is the route yaw rate or route coverage of the navigation route; then, the model evaluation method of the present disclosure may also include the training step of the target index prediction model, including: Figure 4 Steps S4100-S4200 shown:

[0082] Step S4100, acquire a first sample set.

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention provides a model evaluation method and device and electronic equipment. The method comprises the steps of obtaining a first processing result obtained by processing a preset target task through a first model running online, and obtaining a second processing result obtained by processing the target task through a second model to be deployed online; predicting a target index of the first processing result according to a preset target index prediction model to obtain a first prediction result; predicting the target index of the second processing result according to the target index prediction model to obtain a second prediction result; and evaluating the second model according to the first prediction result and the second prediction result to obtain an evaluation result of the second model.

Description

technical field [0001] The present disclosure relates to the technical field of machine learning, and more specifically, to a model evaluation method, a model evaluation device, an electronic device, and a computer program product. Background technique [0002] Under normal circumstances, if you want to evaluate the performance of the offline trained model on the target index, you must deploy the model to run online for a period of time, and then evaluate the model's performance on the target index according to the online operation of the model . This results in longer evaluation cycles for models and higher development costs. Moreover, after the model is deployed online, if the evaluation result of the model is not good, the model needs to be taken offline for iterative training before going online, which will increase the number of times the model goes online and affect the effect of online services. [0003] Therefore, it is a problem to be solved by those skilled in th...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): G06N20/00G01C21/34
CPCG06N20/00G01C21/3492G01C21/343G01C21/3461
Inventor 陈超徐龙飞崔恒斌
Owner ALIBABA SINGAPORE HLDG PTE LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com