A negative feedback data acquisition method, device and system

Through the negative feedback data acquisition method, the AI ​​SDK components and data report SDK components collect and report key data in AI operation, solving the problem of difficult to identify the quality of AI capabilities, achieving the improvement of AI model effects and generalization capabilities, and forming an automated and systematic closed loop of iteration of AI capabilities.

CN114492553BActive Publication Date: 2025-05-27GUANGZHOU HUYA TECH CO LTD
View PDF 1 Cites 0 Cited by

Patent Information

Application Number
CN202011264434.4
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2020-11-12
Publication Date
2025-05-27
Estimated Expiration
2040-11-12

AI Technical Summary

Technical Problem

The data reported in the prior art is difficult to identify the quality of AI capabilities, making it difficult to systematically drive iterative improvements in AI capabilities.

Method used

A negative feedback data acquisition method is proposed. Through the combination of AI SDK components, feedback background and data reporting SDK components, the negative feedback trigger condition information sent by the feedback background is received, the target object that meets the conditions is detected during online running, and the data acquisition request is sent to the data reporting SDK components, the original object data is collected and reported to the feedback background.

Benefits of technology

Through directed data annotation and addition of AI training, the effect and generalization capabilities of AI models can be improved, and the iterative closed loop of automated and systematic AI capabilities can be achieved, thereby improving production efficiency.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114492553B_ABST
    Figure CN114492553B_ABST
Patent Text Reader

Abstract

The present application discloses a negative feedback data acquisition method, device and system. The system includes a feedback background and a client. The client includes an artificial intelligence (AI) SDK component and a data reporting SDK component. The feedback background is configured to send negative feedback trigger condition information to the AI SDK component and receive first negative feedback data sent by the data reporting SDK component. The AI SDK component is configured to, when running online, if a target object that meets the negative feedback trigger condition information is detected, send a negative feedback data acquisition request to the data reporting SDK component. The data reporting SDK component is configured to, according to the negative feedback data acquisition request, collect original object data corresponding to the target object as the first negative feedback data and feed the first negative feedback data back to the feedback background. After these first negative feedback data are directionally data-labeled and added to AI training, it can have better effects and performances in the same scenario, and can improve the effects and generalization ability of the AI model.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The embodiments of the present application relate to data processing technology, and in particular to a negative feedback data collection method, device and system. Background Art

[0002] With the rapid development of artificial intelligence (AI) technology, AI technology has been widely used in various fields. Since AI capabilities are data-driven, the effect is better when the application scenario is consistent with the distribution of training data, otherwise the effect is poor.

[0003] In an online production environment, you may encounter some scenarios where AI does not perform well. In related technologies, the general solution is to collect operation exceptions or some performance-related data (such as crash status reports or time-consuming logs) during the AI ​​operation process and report this data. However, it is difficult to identify the quality of AI capabilities through these data, making it difficult to systematically drive the iterative improvement of AI capabilities. Summary of the invention

[0004] The present application provides a negative feedback data collection method, device and system to solve the problem in the prior art that it is difficult to identify the quality of AI capability operation with the reported data.

[0005] In a first aspect, an embodiment of the present application provides a negative feedback data collection system, the system comprising: a feedback backend and a client, the client comprising an artificial intelligence AI SDK component and a data reporting SDK component;

[0006] The feedback backend is used to send negative feedback trigger condition information to the AI ​​SDK component, and receive the first negative feedback data sent by the data reporting SDK component;

[0007] The AI ​​SDK component is used to send a negative feedback data acquisition request to the data reporting SDK component when the target object that meets the negative feedback trigger condition information is detected when the AI ​​SDK component is running online;

[0008] The data reporting SDK component is used to collect the original object data corresponding to the target object as the first negative feedback data according to the negative feedback data acquisition request, and feed back the first negative feedback data to the feedback background.

[0009] In a second aspect, an embodiment of the present application further provides a negative feedback data collection method, which is applied to an AISDK component, wherein the AI ​​SDK component, the feedback background, and the data reporting SDK component form a negative feedback data collection link; the method includes:

[0010] Receive negative feedback trigger condition information sent by the feedback background;

[0011] When running online, if a target object that meets the negative feedback trigger condition information is detected, a negative feedback data acquisition request is sent to the data reporting SDK component to trigger the data reporting SDK component to collect the original object data corresponding to the target object, and report the original object data as the first negative feedback data to the feedback background.

[0012] In a third aspect, the embodiment of the present application further provides a negative feedback data collection device, the device is located in the AISDK component, the AI ​​SDK component and the feedback background and data reporting SDK component form a negative feedback data collection link; the device includes:

[0013] A negative feedback condition receiving module is used to receive negative feedback trigger condition information sent by the feedback background;

[0014] The negative feedback condition detection module is used to send a negative feedback data acquisition request to the data reporting SDK component when running online. If a target object that meets the negative feedback trigger condition information is detected, the data reporting SDK component is triggered to collect the original object data corresponding to the target object, and report the original object data as the first negative feedback data to the feedback background.

[0015] In a fourth aspect, an embodiment of the present application further provides an electronic device, the electronic device comprising:

[0016] one or more processors;

[0017] a storage device for storing one or more programs,

[0018] When the one or more programs are executed by the one or more processors, the one or more processors implement the above method.

[0019] In a fifth aspect, an embodiment of the present application further provides a computer-readable storage medium on which a computer program is stored, and the program implements the above method when executed by a processor.

[0020] The technical solution provided by this application has the following beneficial effects:

[0021] This embodiment proposes an automated negative feedback data collection system, in which the AI ​​capability of negative feedback data collection is connected to the AI ​​SDK component. When the AI ​​SDK component is running online, according to the negative feedback trigger condition information sent by the feedback background, the target object that meets the negative feedback trigger condition information can be detected in the real online environment, and a negative feedback data acquisition request is sent to the data reporting SDK component according to the target object, so as to trigger the data reporting SDK component to collect the original object data corresponding to the target object as the first negative feedback data, and report the first negative feedback data to the feedback background. After the first negative feedback data is labeled with targeted data and added with AI training, it can have better effect and performance for the same scenario, can improve the effect and generalization ability of the AI ​​model, and form an iterative closed loop of automated and systematic AI capability improvement, thereby improving production efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 It is a structural schematic diagram of an embodiment of a negative feedback data acquisition system provided in Embodiment 1 of the present application;

[0023] Figure 2 It is a flow chart of an embodiment of a negative feedback data collection method provided in Embodiment 2 of the present application;

[0024] Figure 3 It is a structural block diagram of an embodiment of a negative feedback data acquisition device provided in Embodiment 3 of the present application;

[0025] Figure 4 It is a structural schematic diagram of an electronic device provided in Example 4 of the present application. DETAILED DESCRIPTION

[0026] The present application will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the present application, rather than to limit the present application. It should also be noted that, for ease of description, only the parts related to the present application, rather than all structures, are shown in the accompanying drawings.

[0027] Embodiment 1

[0028] Figure 1A schematic diagram of the structure of a negative feedback data collection system embodiment provided in the first embodiment of the present application, the negative feedback data collection system includes a feedback background 10 and a client 20, and the client 20 may further include an AI (Artificial Intelligence) SDK (Software Development Kit) component 201 and a data reporting SDK component 202. The feedback background 10, the AI ​​SDK component 201 and the data reporting SDK component 202 may constitute a negative feedback link of the present embodiment, through which the negative feedback data may be collected.

[0029] The feedback backend 10 is used to send negative feedback trigger condition information to the AI ​​SDK component 201 , and receive the first negative feedback data sent by the data reporting SDK component 202 .

[0030] The AI ​​SDK component 201 is used to send a negative feedback data acquisition request to the data reporting SDK component 202 when the target object that meets the negative feedback trigger condition information is detected during online operation;

[0031] The data reporting SDK component 202 is used to collect the original object data corresponding to the target object as the first negative feedback data according to the negative feedback data acquisition request, and feed back the first negative feedback data to the feedback background 10 .

[0032] In this embodiment, after the AI ​​capability with the negative feedback judgment mechanism is connected to the AI ​​SDK component 201 and deployed to the online environment, the AI ​​SDK component 201 monitors the processing status of each processing object by receiving the negative feedback trigger condition information issued by the feedback background 10. If a processing object triggers the negative feedback trigger condition corresponding to the negative feedback trigger condition information, the processing object is used as the target object, and a negative feedback data acquisition request is generated according to the target object, and then the negative feedback data acquisition request is sent to the data reporting SDK component 202. After receiving the negative feedback data acquisition request, the data reporting SDK component 202 will collect the original object data corresponding to the target object as the first negative feedback data, and feed the first negative feedback data back to the feedback background 10.

[0033] It should be noted that the feedback background 10 can dynamically update the negative feedback triggering condition information and issue the latest negative feedback triggering condition information.

[0034] In one embodiment, the feedback background 10 is also used to: send the first negative feedback data to the model training server, the model training server is used to label the first negative feedback data, and use the labeled first negative feedback data as a training sample, update the AI ​​model corresponding to the AI ​​SDK component, and send the updated AI model to the AI ​​SDK component.

[0035] In this embodiment, the AI ​​capability of the AI ​​SDK component 201 can be implemented through an AI model. The AI ​​SDK component 201 may include one or more AI models for implementing different functions. The AI ​​model may be a deep neural network model. For example, the AI ​​model may include a face detection model, a portrait segmentation model, a limb recognition model, a speech recognition model, etc.

[0036] After the feedback background 10 receives the first negative feedback data, it can feed back these first negative feedback data to the model training server used to train the above-mentioned AI model. After these first negative feedback data are labeled with targeted data and added to AI training in the model training server, the corresponding AI model can be updated. The updated AI model is then sent to the client by the model training server, which can improve the effect and generalization ability of the AI ​​model for the corresponding scenario, forming an iterative closed loop of automated and systematic AI capability improvement.

[0037] In practice, the negative feedback triggering condition information sent by the feedback backend can be set according to actual needs, and can include one or a combination of the following: critical confidence value, collection frequency, or a list of users who need to report negative feedback. Among them, the critical confidence value is used to determine whether the processing object is at a critical point and whether the negative feedback condition is triggered; the collection frequency is used to indicate the frequency of negative feedback judgment within a set time period; the list of users who need to report negative feedback is used to indicate which users need to report negative feedback and which users do not need to report negative feedback.

[0038] In one embodiment, the AI ​​SDK component 201 is specifically used to: obtain the confidence value of the inference result output by the AI ​​model when using the AI ​​model for model inference online; determine whether the confidence value of the inference result is within the set range of the critical confidence value; if so, determine the processing object corresponding to the inference result confidence value as the target object that meets the negative feedback trigger condition information, and generate a negative feedback data acquisition request based on the identifier of the target object.

[0039] In this embodiment, when the AI SDK component 201 performs model inference using the AI model, in addition to outputting the inference result, it can also output the confidence value of the inference result. Then, the AI SDK component 201 will compare each confidence value of the inference result with the critical confidence value to determine whether each confidence value of the inference result is within the set range of the critical confidence value. If not, the negative feedback condition is not triggered; if so, it means that the confidence value of the inference result is at the critical point and the negative feedback condition is triggered. At this time, the processing object corresponding to the confidence value of the inference result can be determined as the target object that meets the negative feedback trigger condition information. Then, the AI SDK component 201 can generate a negative feedback data acquisition request according to the identifier of the target object and send it to the data reporting SDK component 202 to trigger the data acquisition of the data reporting SDK component 202.

[0040] For example, assume that the current scenario is a face detection scenario, and the AI model outputs the number of faces, positions in the current image, and the confidence value corresponding to each predicted face. Assume that the critical confidence value is set to 0.9. If the confidence value of at least one predicted face in the current image is between 0.85 and 0.95, the AI SDK component 201 can determine that the current image is at the critical point, trigger the negative feedback condition, and then generate a negative feedback data acquisition request according to the identifier of the current image and send it to the data reporting SDK component 202 to trigger the source data acquisition requirement to the data reporting SDK component 202.

[0041] Another example, assume that the current scenario is a portrait segmentation scenario, and the AI model outputs the portrait mask of the current image. The value of each pixel in the portrait mask ranges from 0 (background) to 1 (portrait foreground), indicating the confidence value that the pixel is the portrait foreground. In the case of portrait segmentation, the entropy of the mask can be calculated. The larger the entropy, the worse the segmentation effect usually is (that is, the confidence values of more pixels are not near 0 or 1). If the entropy is greater than the issued critical confidence value, the AI SDK component 201 can determine that the current image is at the critical point, trigger the negative feedback condition, and then generate a negative feedback data acquisition request according to the identifier of the current image and send it to the data reporting SDK component 202 to trigger the source data acquisition requirement to the data reporting SDK component 202.

[0042] In other embodiments, if the negative feedback trigger condition information includes a user list that needs to perform negative feedback reporting, before performing the above negative feedback condition judgment, the client 20 can first determine whether the account logged in to the current client is in the user list according to the user list. If so, the critical confidence value is sent to the AI SDK component 201 to trigger the subsequent negative feedback condition judgment process; otherwise, the process ends.

[0043] In one implementation, the data reporting SDK component 202 is specifically configured to obtain current environment information after receiving the negative feedback data acquisition request, and determine whether to execute or ignore the negative feedback data acquisition request according to the environment information.

[0044] In this embodiment, for the received negative feedback data acquisition request, the data reporting SDK component 202 can choose to execute or ignore the negative feedback data acquisition request according to the current environmental information. Exemplarily, the environmental information may include current hardware load information and / or network status information. For example, if the current network bandwidth is not very good, the data reporting SDK component 202 can choose to only obtain and report part of the first negative feedback data, without having to report each first negative feedback data. For another example, if the current CPU or storage capacity is insufficient, and frequent storage may cause the application to freeze, the data reporting SDK component 202 may also choose to only obtain and report part of the first negative feedback data.

[0045] In other embodiments, when the data reporting SDK component 202 decides whether to process a negative feedback data acquisition request, the collection frequency issued by the feedback background 10 may also be considered. If relatively intensive requests are received from the AI ​​SDK component 201, it may choose to process only some of the requests.

[0046] When the data reporting SDK component 202 decides to execute the currently received request, the original object data corresponding to the target object can be obtained according to the identifier in the request, and the original object data is used as the first negative feedback data. Among them, the target object is usually a pre-processed object, and the original object data is the source data before pre-processing.

[0047] In one embodiment, the AI ​​SDK component 201 is also used to: collect abnormal data occurring during the online operation process as second negative feedback data, and send the second negative feedback data to the data reporting SDK component, so that the data reporting SDK component 202 feeds back the second negative feedback data to the feedback background.

[0048] In this embodiment, in addition to judging the negative feedback conditions, the AI ​​SDK component 201 can also collect abnormal data that occurs during the online operation process, such as crash data or time-consuming log data, as the second negative feedback data, and then report the second negative feedback data to the feedback background 10 via the data reporting SDK component 202.

[0049] This embodiment proposes an automated negative feedback data collection system, in which the AI ​​capability of negative feedback data collection is connected to the AI ​​SDK component. When the AI ​​SDK component is running online, according to the negative feedback trigger condition information sent by the feedback background, the target object that meets the negative feedback trigger condition information can be detected in the real online environment, and a negative feedback data acquisition request is sent to the data reporting SDK component according to the target object, so as to trigger the data reporting SDK component to collect the original object data corresponding to the target object as the first negative feedback data, and report the first negative feedback data to the feedback background. After the first negative feedback data is labeled with targeted data and added with AI training, it can have better effect and performance for the same scenario, can improve the effect and generalization ability of the AI ​​model, and form an iterative closed loop of automated and systematic AI capability improvement, thereby improving production efficiency.

[0050] Embodiment 2

[0051] Figure 2 This is a flow chart of a negative feedback data collection method embodiment provided in Embodiment 2 of the present application. This embodiment is executed by the AI ​​SDK component 201 in Embodiment 1 and may specifically include the following steps:

[0052] Step 210: Receive negative feedback trigger condition information sent by the feedback background.

[0053] Step 220, when running online, if a target object that meets the negative feedback trigger condition information is detected, a negative feedback data acquisition request is sent to the data reporting SDK component to trigger the data reporting SDK component to collect the original object data corresponding to the target object, and report the original object data as the first negative feedback data to the feedback background.

[0054] In one implementation, the negative feedback trigger condition information includes a critical confidence value, and step 220 may further include the following steps:

[0055] Step 220-1, when using the AI ​​model to perform model reasoning online, obtain the confidence value of the reasoning result output by the AI ​​model.

[0056] Step 220-2, determining whether the confidence value of the inference result is within the set range of the critical confidence value.

[0057] Step 220 - 3 : If yes, the processing object corresponding to the confidence value of the inference result is determined as a target object that meets the negative feedback trigger condition information, and a negative feedback data acquisition request is generated according to the identifier of the target object.

[0058] In one implementation, this embodiment may further include the following steps:

[0059] Abnormal data occurring during the online operation is collected as second negative feedback data, and the second negative feedback data is sent to the data reporting SDK component, so that the data reporting SDK component feeds back the second negative feedback data to the feedback background.

[0060] In this embodiment, the AI ​​capability of collecting negative feedback data is connected to the AI ​​SDK component. When the AI ​​SDK component is running online, the target object that meets the negative feedback trigger condition information can be detected in the real online environment according to the negative feedback trigger condition information sent by the feedback background. A negative feedback data acquisition request is sent to the data reporting SDK component according to the target object, so as to trigger the data reporting SDK component to collect the original object data corresponding to the target object as the first negative feedback data, and report the first negative feedback data to the feedback background. After the first negative feedback data is labeled with targeted data and added with AI training, it can have better effect and performance for the same scenario, and can improve the effect and generalization ability of the AI ​​model, forming an iterative closed loop of automated and systematic AI capability improvement, thereby improving production efficiency.

[0061] Embodiment 3

[0062] Figure 3 This is a structural block diagram of an embodiment of a negative feedback data collection device provided in Embodiment 3 of the present application. The device is located in the AI ​​SDK component. The AI ​​SDK component, the feedback background, and the data reporting SDK component form a negative feedback data collection link. The device may include the following modules:

[0063] A negative feedback condition receiving module 310 is used to receive negative feedback trigger condition information sent by the feedback background;

[0064] The negative feedback condition detection module 320 is used to send a negative feedback data acquisition request to the data reporting SDK component when running online. If a target object that meets the negative feedback trigger condition information is detected, the data reporting SDK component is triggered to collect the original object data corresponding to the target object, and report the original object data as the first negative feedback data to the feedback background.

[0065] A negative feedback data collection device provided in the embodiment of the present application can execute the negative feedback data collection method provided in the second embodiment of the present application, and has the corresponding functional modules and beneficial effects of the execution method.

[0066] Embodiment 4

[0067] Figure 4 A schematic diagram of the structure of an electronic device provided in Embodiment 4 of the present application is shown in FIG. Figure 4As shown, the electronic device includes a processor 410, a memory 420, an input device 430, and an output device 440; the number of the processor 410 in the electronic device can be one or more. Figure 4 A processor 410 is taken as an example; the processor 410, the memory 420, the input device 430 and the output device 440 in the electronic device can be connected via a bus or other means. Figure 4 The example of connecting through bus is taken in the following.

[0068] The memory 420 is a computer-readable storage medium that can be used to store software programs, computer executable programs and modules, such as program instructions / modules corresponding to the above embodiments in the embodiments of the present application. The processor 410 executes various functional applications and data processing of the electronic device by running the software programs, instructions and modules stored in the memory 420, that is, implements the method mentioned in the above method embodiment.

[0069] The memory 420 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system and at least one application required for a function; the data storage area may store data created according to the use of the terminal, etc. In addition, the memory 420 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some instances, the memory 420 may further include a memory remotely arranged relative to the processor 410, and these remote memories may be connected to the device / terminal / server via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0070] The input device 430 may be used to receive input digital or character information and generate key signal input related to user settings and function control of the electronic device. The output device 440 may include a display device such as a display screen.

[0071] Embodiment 5

[0072] Embodiment 5 of the present application also provides a storage medium containing computer executable instructions, and the computer executable instructions are used to execute the method in the above method embodiment when executed by a computer processor.

[0073] Of course, the storage medium containing computer executable instructions provided in the embodiments of the present application is not limited to the method operations described above, and the computer executable instructions can also execute related operations in the method provided in any embodiment of the present application.

[0074] Through the above description of the implementation method, the technicians in the relevant field can clearly understand that the present application can be implemented with the help of software and necessary general hardware, and of course it can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application can be essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as a computer floppy disk, read-only memory (ROM), random access memory (RAM), flash memory (FLASH), hard disk or optical disk, etc., including a number of instructions for an electronic device (which can be a personal computer, server, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0075] It is worth noting that in the embodiment of the above-mentioned device, the various units and modules included are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application.

[0076] Note that the above are only preferred embodiments of the present application and the technical principles used. Those skilled in the art will understand that the present application is not limited to the specific embodiments described herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present application. Therefore, although the present application is described in more detail through the above embodiments, the present application is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present application, and the scope of the present application is determined by the scope of the appended claims.

Claims

1. A negative feedback data acquisition system, characterized in that, the system includes: a feedback background and a client, and the client includes an artificial intelligence AI SDK component and a data reporting SDK component; the feedback background is used to send negative feedback trigger condition information to the AI SDK component, and receive the first negative feedback data sent by the data reporting SDK component; the AI SDK component is used to, when running online, if a target object that meets the negative feedback trigger condition information is detected, send a negative feedback data acquisition request to the data reporting SDK component; the data reporting SDK component is used to collect the original object data corresponding to the target object as the first negative feedback data according to the negative feedback data acquisition request, and feedback the first negative feedback data to the feedback background; wherein, the negative feedback trigger condition information includes: a critical confidence value and a user list that needs to report negative feedback; wherein, the critical confidence value is used to determine whether the processing object is at the critical point and whether to trigger the negative feedback condition; the client is used to determine whether the account logging in to this client is in the user list, and if so, send the critical confidence value to the AI SDK component.

2. The system according to claim 1, characterized in that, the feedback background is further used for: sending the first negative feedback data to a model training server, and the model training server is used to annotate the first negative feedback data, and use the annotated first negative feedback data as a training sample to update the AI model corresponding to the AI SDK component, and send the updated AI model to the AI SDK component.

3. The system according to claim 1 or 2, characterized in that, the data reporting SDK component is specifically used to, after receiving the negative feedback data acquisition request, obtain the current environment information, and determine whether to execute or ignore the negative feedback data acquisition request according to the environment information; wherein, the environment information includes the current hardware load information and / or network condition information.

4. The system according to claim 1, characterized in that, the negative feedback trigger condition information includes a critical confidence value, and the AI SDK component is specifically used for: when performing model inference using an AI model online, obtaining the inference result confidence value output by the AI model; judging whether the inference result confidence value is within the set range of the critical confidence value; if so, determining the processing object corresponding to the inference result confidence value as the target object that meets the negative feedback trigger condition information, and generating a negative feedback data acquisition request according to the identifier of the target object.

5. The system according to claim 1, characterized in that, the AI SDK component is further used for: collecting abnormal data that appears during the online operation process as the second negative feedback data, and sending the second negative feedback data to the data reporting SDK component, so that the data reporting SDK component feeds back the second negative feedback data to the feedback background.

6. A negative feedback data acquisition method, characterized in that, The method is applied to an AI SDK component, and the AI SDK component, the feedback background, and the data reporting SDK component form a negative feedback data collection link; the method includes: Receiving negative feedback trigger condition information sent by the feedback background; When running online, if a target object that meets the negative feedback trigger condition information is detected, a negative feedback data acquisition request is sent to the data reporting SDK component to trigger the data reporting SDK component to collect the original object data corresponding to the target object, and the original object data is reported to the feedback background as the first negative feedback data; Among them, the negative feedback trigger condition information includes: A critical confidence value and a user list that needs to report negative feedback; among them, the critical confidence value is used to determine whether the processing object is at the critical point and whether to trigger the negative feedback condition; Determine whether the account logged in to the client is in the user list. If so, send the critical confidence value to the AI SDK component, where the client includes the AI SDK component and the data reporting SDK component.

7. A negative feedback data collection device Characterized in that The device is located in the AI SDK component, and the AI SDK component, the feedback background, and the data reporting SDK component form a negative feedback data collection link; the device includes: A negative feedback condition receiving module, configured to receive negative feedback trigger condition information sent by the feedback background; A negative feedback condition detection module, configured to, when running online, if a target object that meets the negative feedback trigger condition information is detected, send a negative feedback data acquisition request to the data reporting SDK component to trigger the data reporting SDK component to collect the original object data corresponding to the target object, and report the original object data to the feedback background as the first negative feedback data; Among them, the negative feedback trigger condition information includes: A critical confidence value and a user list that needs to report negative feedback; among them, the critical confidence value is used to determine whether the processing object is at the critical point and whether to trigger the negative feedback condition; Determine whether the account logged in to the client is in the user list. If so, send the critical confidence value to the AI SDK component, where the client includes the AI SDK component and the data reporting SDK component.

8. An electronic device Characterized in that The electronic device includes: One or more processors; A storage device for storing one or more programs When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to claim 6.

9. A computer-readable storage medium, on which a computer program is stored Characterized in that When the program is executed by a processor, the method according to claim 6 is implemented.

Citation Information

Patent Citations

  • Model training method and device, computer equipment and storage medium

    CN111210024A