Model fine-tuning method, device, equipment, medium and program product

Abstract

The present disclosure provides a model fine-tuning method, device, equipment, medium and program product, which can be applied to the fields of artificial intelligence and computer technology. The method includes: determining one or more image data based on image data screening conditions; using an image recognition model to identify the image data to obtain algorithm tags of one or more image data; using a quality control model to identify the image data to obtain quality control tags of one or more image data; in determining one or more image data, if there are algorithm tags and quality control tags corresponding to the image data indicating that the image recognition model is wrong, determining a fine-tuning data set based on the one or more image data and the quality control tags corresponding to the one or more image data; and fine-tuning the parameters of the image recognition model based on the fine-tuning data set to obtain a fine-tuned target image recognition model.

Description

Technical Field

[0001] The present disclosure relates to the fields of artificial intelligence and computer technology, and in particular to a model fine-tuning method, device, equipment, medium and program product. Background Art

[0002] With the rapid development of image processing technology, compared with completely manual inspection of image data, the use of image recognition algorithms for analysis and identification is more efficient and more reliable in a short period of time. Therefore, it is becoming more and more common to use image recognition algorithms to analyze the scanned image data collected in real time by security inspection equipment to determine whether there are prohibited items.

[0003] In security inspection scenarios, since the computing devices used are limited in size and usually have limited computing power, it is necessary to control the number of parameters of the image recognition algorithm deployed therein to ensure the normal operation of the computing devices. In cases where the recognition ability of the image recognition algorithm needs to be improved, such as when a new display method of prohibited items is found, the image data and its labels corresponding to the new display method can be determined through expert calibration, and the image recognition algorithm can be retrained.

[0004] In the process of implementing the present disclosure, the inventors found that the prior art has at least the following problems: determining the labels of image data by expert calibration has low processing efficiency and high labor costs, and due to the limited computing power of computing devices in the security inspection environment, it is difficult to quickly complete model retraining, thereby affecting the normal progress of security inspection work. Summary of the invention

[0005] In view of the above problems, the present disclosure provides a model fine-tuning method, apparatus, device, medium and program product.

[0006] According to a first aspect of the present disclosure, a model fine-tuning method is provided, comprising: determining one or more image data based on image data screening conditions; identifying the image data using an image recognition model to obtain algorithm tags of the one or more image data; identifying the image data using a quality control model to obtain quality control tags of the one or more image data; in determining that in the one or more image data, there are algorithm tags and quality control tags corresponding to the image data indicating that the image recognition model has an error, determining a fine-tuning data set based on the one or more image data and the quality control tags corresponding to the one or more image data; and fine-tuning parameters of the image recognition model based on the fine-tuning data set to obtain a fine-tuned target image recognition model.

[0007] According to an embodiment of the present disclosure, the model fine-tuning method also includes: for each image data, determining the number of marks and the mark position of the algorithm mark and the quality control mark corresponding to the image data; when the number of marks is consistent and the mark position is consistent, determining that the algorithm mark and the quality control mark corresponding to the image data represent that the image recognition model is not wrong; and when the number of marks is different or the position of the algorithm mark and the quality control mark are inconsistent, determining that the algorithm mark and the quality control mark corresponding to the image data represent that the image recognition model is wrong.

[0008] According to an embodiment of the present disclosure, a fine-tuning dataset is determined based on one or more image data and quality control marks corresponding to each of the one or more image data, including: for each image data, determining the quality control mark corresponding to the image data as a label of the image data; and determining the fine-tuning dataset based on one or more image data and the labels corresponding to each of the one or more image data.

[0009] According to an embodiment of the present disclosure, image data screening conditions include a target time period, a target data source, and a type of object to be identified; based on the image data screening conditions, one or more image data are determined, including: determining the target time period data based on the target time period from all data acquired from the target data source; and filtering out one or more image data from the target time period data based on the type of object to be identified.

[0010] According to an embodiment of the present disclosure, a quality control model is obtained by training based on edge images and image labels corresponding to the edge images, wherein the object to be identified in the edge image is in an edge state, and the edge state includes being partially occluded and the presence of similar objects; the quality control model is obtained in the following manner: determining multiple sample edge images from multiple sample image data, wherein the object to be identified in the sample edge image is in an edge state; determining sample labels of the sample edge images based on the objects to be identified in the edge state in the sample edge images; determining sample labels of the remaining sample images based on multiple remaining sample images other than the multiple sample edge images in the multiple sample image data, and the objects to be identified in the remaining sample images; determining a quality control training set for training an initial quality control model based on multiple sample edge images, multiple remaining sample images, and sample labels corresponding to the multiple sample edge images and the multiple remaining sample images respectively; and adjusting parameters of the initial quality control model based on the quality control training set to determine the quality control model.

[0011] According to an embodiment of the present disclosure, an image recognition model is obtained in the following manner: based on a plurality of remaining sample images and sample labels respectively corresponding to the plurality of remaining sample images, a recognition training set for training an initial image recognition model is determined; and based on the recognition training set, parameters of the initial image recognition model are adjusted to determine the image recognition model.

[0012] According to an embodiment of the present disclosure, the model fine-tuning method also includes: uploading the target image recognition model to a version control server and updating the current model version saved in the version control server; determining the client model version corresponding to the version verification request in response to a version verification request sent by the client; and in the event that the current model version is inconsistent with the client model version, sending the target image recognition model to the client, so that the client updates the image recognition model based on the target image recognition model.

[0013] A second aspect of the present disclosure provides a model fine-tuning device, including: an image determination module, for determining one or more image data based on image data screening conditions; a first mark determination module, for identifying the image data using an image recognition model to obtain algorithm marks of the one or more image data; a second mark determination module, for identifying the image data using a quality control model to obtain quality control marks of the one or more image data; a data set determination module, for determining a fine-tuning data set based on the one or more image data and the quality control marks corresponding to the one or more image data when it is determined that there are algorithm marks and quality control marks corresponding to the image data in the one or more image data, indicating that the image recognition model has an error; and a parameter fine-tuning module, for fine-tuning the parameters of the image recognition model based on the fine-tuning data set to obtain a fine-tuned target image recognition model.

[0014] A third aspect of the present disclosure provides an electronic device, comprising: one or more processors; and a memory for storing one or more computer programs, wherein the one or more processors execute the one or more computer programs to implement the steps of the above method.

[0015] The fourth aspect of the present disclosure further provides a computer-readable storage medium having a computer program or instructions stored thereon, which implements the steps of the above method when the above computer program or instructions are executed by a processor.

[0016] The fifth aspect of the present disclosure further provides a computer program product, including a computer program or instructions, which implement the steps of the above method when the above computer program or instructions are executed by a processor.

[0017] According to an embodiment of the present disclosure, after the image data collected during the actual work of the client is screened and the image data is obtained, the image recognition model and the quality control model are used to identify and mark the objects to be identified in the image data, and by comparing the recognition results of the two models, it is determined whether there are errors in the recognition results of the image recognition model and whether the model needs to be adjusted. By comparing the results between the models, it is possible to determine whether the model needs to be adjusted and upgraded without manual intervention, thereby improving the degree of automation of model fine-tuning. In the case where model adjustment and upgrade are required, the recognition results of the quality control model are used as labels for the image data to determine the fine-tuning data set and fine-tune the parameters of the image recognition model. By automatically determining the fine-tuning data set, the degree of automation of model fine-tuning is further improved. In addition, since the image data in the fine-tuning data set are all collected during the actual work of the client, fine-tuning the image recognition model enables it to recognize such image data, thereby improving the efficiency and accuracy of the client's work. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The above contents and other purposes, features and advantages of the present disclosure will become more apparent through the following description of the embodiments of the present disclosure with reference to the accompanying drawings, in which:

[0019] Figure 1 A diagram schematically illustrates an application scenario of a model fine-tuning method, apparatus, device, medium, and program product according to an embodiment of the present disclosure;

[0020] Figure 2 A flowchart of a model fine-tuning method according to an embodiment of the present disclosure is schematically shown;

[0021] Figure 3 A flowchart of a model fine-tuning method according to another embodiment of the present disclosure is schematically shown;

[0022] Figure 4 A structural block diagram of a model fine-tuning device according to an embodiment of the present disclosure is schematically shown; and

[0023] Figure 5 A block diagram of an electronic device suitable for implementing a model fine-tuning method according to an embodiment of the present disclosure is schematically shown. DETAILED DESCRIPTION

[0024] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present disclosure. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiments of the present disclosure. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of the concepts of the present disclosure.

[0025] The terms used herein are only for describing specific embodiments and are not intended to limit the present disclosure. The terms "comprise", "include", etc. used herein indicate the existence of the features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.

[0026] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0027] When using expressions such as "at least one of A, B, and C, etc.", they should generally be interpreted according to the meaning of the expression commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0028] In the technical solution of the present disclosure, the user information (including but not limited to user personal information, user image information, user device information, such as location information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved are all information and data authorized by the user or fully authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of the relevant data comply with relevant laws, regulations and standards, take necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0029] In the scenario of using personal information for automated decision-making, the methods, devices, and systems provided by the embodiments of the present disclosure provide users with corresponding operation portals for users to choose to agree or reject the automated decision-making results; if the user chooses to reject, the expert decision-making process will be entered. The expression "automated decision-making" here refers to the activity of automatically analyzing and evaluating a person's behavioral habits, interests and hobbies, or economic, health, credit status, etc. through computer programs, and making decisions. The expression "expert decision-making" here refers to the activity of making decisions by people who specialize in a certain field, have specialized experience, knowledge and skills, and have reached a certain level of professionalism.

[0030] An embodiment of the present disclosure provides a model fine-tuning method, comprising: determining one or more image data based on image data screening conditions; using an image recognition model to identify the image data to obtain algorithm tags of the one or more image data; using a quality control model to identify the image data to obtain quality control tags of the one or more image data; in the case where the one or more image data are determined to have algorithm tags and quality control tags corresponding to the image data, indicating that the image recognition model has errors, determining a fine-tuning data set based on the one or more image data and the quality control tags corresponding to the one or more image data; and fine-tuning parameters of the image recognition model based on the fine-tuning data set to obtain a fine-tuned target image recognition model.

[0031] Figure 1 The application scenario diagram of the model fine-tuning method, apparatus, device, medium and program product according to the embodiments of the present disclosure is schematically shown.

[0032] like Figure 1 As shown, the application scenario 100 according to this embodiment may include a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104, and a server 105. The network 104 is used to provide a medium for a communication link between the first terminal device 101, the second terminal device 102, the third terminal device 103, and the server 105. The network 104 may include various connection types, such as wired, wireless communication links, or optical fiber cables, etc.

[0033] The user can use the first terminal device 101, the second terminal device 102, and the third terminal device 103 to interact with the server 105 through the network 104 to receive or send messages, etc. Various communication client applications can be installed on the first terminal device 101, the second terminal device 102, and the third terminal device 103.

[0034] The first terminal device 101, the second terminal device 102, and the third terminal device 103 may be various electronic devices with display screens, such as computers connected to the security inspection equipment for displaying image data acquired by the security inspection equipment.

[0035] The server 105 may be a server that provides various services, such as a server that supports the image recognition model used by the user using the first terminal device 101, the second terminal device 102, and the third terminal device 103. The server may detect the recognition effect of the image recognition model currently used in the first terminal device 101, the second terminal device 102, and the third terminal device 103, and automatically fine-tune and upgrade the image recognition model when it is found that the recognition effect needs to be improved, and feed back the upgraded image recognition model to the terminal device.

[0036] It should be noted that the model fine-tuning method provided in the embodiment of the present disclosure can generally be executed by the server 105. Accordingly, the model fine-tuning device provided in the embodiment of the present disclosure can generally be set in the server 105. The model fine-tuning method provided in the embodiment of the present disclosure can also be executed by a server or server cluster that is different from the server 105 and can communicate with the first terminal device 101, the second terminal device 102, the third terminal device 103 and / or the server 105. Accordingly, the model fine-tuning device provided in the embodiment of the present disclosure can also be set in a server or server cluster that is different from the server 105 and can communicate with the first terminal device 101, the second terminal device 102, the third terminal device 103 and / or the server 105.

[0037] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0038] The following will be based on Figure 1 The scene described by Figure 2~Figure 3 The model fine-tuning method of the embodiment of the present disclosure is described in detail.

[0039] Figure 2 The flowchart of the model fine-tuning method according to an embodiment of the present disclosure is schematically shown.

[0040] like Figure 2 As shown, the model fine-tuning method of this embodiment includes operations S210 to S250.

[0041] In operation S210, one or more image data are determined based on the image data screening condition.

[0042] According to the embodiments of the present disclosure, the image data may include objects to be identified, and the objects to be identified may include prohibited items that need to be detected, etc. One or more image data may be screened out from a large amount of data based on the image data screening conditions, and the image data may be used for subsequent model fine-tuning, which can reduce the amount of data transmission, thereby reducing the transmission occupancy of the system line and improving resource utilization.

[0043] In operation S220, the image data is recognized using an image recognition model to obtain algorithmic labels of one or more image data.

[0044] According to an embodiment of the present disclosure, the image recognition model can recognize the object to be recognized included in the image data, and mark the object to be recognized at the corresponding position of the image data to obtain an algorithm mark corresponding to the object to be recognized. After the image recognition model recognizes the image data, if no object to be recognized is found, the image data is not marked.

[0045] In operation S230, the image data is identified using the quality control model to obtain one or more quality control marks of the image data.

[0046] According to an embodiment of the present disclosure, the quality control model can identify the object to be identified included in the image data, and mark the object to be identified at the corresponding position of the image data to obtain a quality control mark corresponding to the object to be identified. Among them, the quality control model can identify the object to be identified that is more difficult to detect in the image data, and the image recognition model usually does not have the ability to identify such objects to be identified. After the quality control model identifies the image data, if no object to be identified is found, the image data is not marked.

[0047] According to an embodiment of the present disclosure, the quality control model can be trained and applied on the server. In addition to being trained and applied on the server, the image recognition model can also be configured in the client to perform image recognition on the client. The client may include an image acquisition device, that is, after the image acquisition device is used to acquire the image at the client, image recognition can be completed locally without going through the server.

[0048] According to the embodiments of the present disclosure, the quality control model has stronger recognition ability than the image recognition model, but the network parameter amount and model magnitude are larger, and the state or position of the object to be identified in the image data used to train the quality control model may not be easy to appear in actual use. Therefore, if the quality control model is directly configured in the client, it will cause the number of parameters in the client to be too large, and the extra parameters are used to identify the state or position of the object to be identified that will not be encountered in the actual scene, and the actual recognition accuracy is low.

[0049] According to the embodiments of the present disclosure, since the recognition ability of the quality control model is stronger, during the actual use of the image recognition model, if the recognition results of the quality control model and the image recognition model for the same actual image data are inconsistent, it can be determined that the recognition ability of the image recognition model cannot meet the accuracy requirements of actual use, and thus it can be determined that the image recognition model needs to be fine-tuned and updated.

[0050] In operation S240, when it is determined that in one or more image data, there are algorithm marks and quality control marks corresponding to the image data indicating that the image recognition model has an error, a fine-tuning data set is determined based on the one or more image data and the quality control marks corresponding to each of the one or more image data.

[0051] According to an embodiment of the present disclosure, since the quality control model has a stronger ability to recognize objects to be recognized in image data than the image recognition model, when the quality control mark is compared with the algorithm mark, if the comparison result indicates that the recognition result of the quality control model on the image data is inconsistent with the recognition result of the image recognition model on the image data, it can be determined that the image recognition model is wrong.

[0052] According to an embodiment of the present disclosure, image data and quality control marks corresponding to the image data are respectively used as samples and corresponding labels to determine a fine-tuning data set for fine-tuning parameters of a model.

[0053] In operation S250, parameters of the image recognition model are fine-tuned based on the fine-tuning data set to obtain a fine-tuned target image recognition model.

[0054] According to an embodiment of the present disclosure, the image data in the image recognition model is input into the image recognition model, or the recognition result of the image recognition model on the image data is determined according to the algorithm mark determined in operation S220, and the recognition loss is determined according to the recognition result and the quality control mark corresponding to the image data.

[0055] According to an embodiment of the present disclosure, parameters of an image recognition model are adjusted based on recognition loss and labels corresponding to image data until the adjusted image recognition model recognizes image data, and the obtained algorithm marks and quality control marks indicate that the adjusted image recognition model has no recognition errors. Based on the model parameters of the current image recognition model, a target image recognition model is determined.

[0056] According to an embodiment of the present disclosure, after the image data collected during the actual work of the client is screened and the image data is obtained, the image recognition model and the quality control model are used to identify and mark the objects to be identified in the image data, and by comparing the recognition results of the two models, it is determined whether there are errors in the recognition results of the image recognition model and whether the model needs to be adjusted. By comparing the results between the models, it is possible to determine whether the model needs to be adjusted and upgraded without manual intervention, thereby improving the degree of automation of model fine-tuning. In the case where model adjustment and upgrade are required, the recognition results of the quality control model are used as labels for the image data to determine the fine-tuning data set and fine-tune the parameters of the image recognition model. By automatically determining the fine-tuning data set, the degree of automation of model fine-tuning is further improved. In addition, since the image data in the fine-tuning data set are all collected during the actual work of the client, fine-tuning the image recognition model enables it to recognize such image data, thereby improving the efficiency and accuracy of the client's work.

[0057] According to an embodiment of the present disclosure, the model fine-tuning method also includes: for each image data, determining the number of marks and the mark position of the algorithm mark and the quality control mark corresponding to the image data; when the number of marks is consistent and the mark position is consistent, determining that the algorithm mark and the quality control mark corresponding to the image data represent that the image recognition model is not wrong; and when the number of marks is different or the position of the algorithm mark and the quality control mark are inconsistent, determining that the algorithm mark and the quality control mark corresponding to the image data represent that the image recognition model is wrong.

[0058] According to an embodiment of the present disclosure, for each image data, the number and position of the algorithm mark in the image data are determined, and the number and position of the quality control mark in the image data are determined. The number and position of the algorithm mark and the quality control mark are compared to determine whether the number and position of the mark of the two are consistent.

[0059] According to an embodiment of the present disclosure, when the number and position of the algorithm mark and the quality control mark are consistent, it can be determined that the recognition results obtained by the image recognition model and the quality control model for the same image data are consistent, and therefore it can be determined that the image recognition model is correct.

[0060] According to an embodiment of the present disclosure, when the number of marks of the algorithm mark and the number of marks of the quality control mark are inconsistent, when the number of marks of the algorithm mark is greater than the number of marks of the quality control mark, it can be determined that the recognition result obtained by the image recognition model for recognizing the image data has a false detection, that is, the part that does not belong to the object to be recognized is mistakenly determined as the object to be recognized, so it can be determined that the image recognition model is wrong. When the number of marks of the algorithm mark is less than the number of marks of the quality control mark, it can be determined that the recognition result obtained by the image recognition model for recognizing the image data has a missed detection, that is, the object to be recognized is not correctly detected, so it can be determined that the image recognition model is wrong.

[0061] According to an embodiment of the present disclosure, when the number of marks of the algorithm mark and the quality control mark are consistent but the mark positions are inconsistent, it can be determined that the recognition result obtained by the image recognition model for recognizing the image data does not correctly mark the position of the object to be identified, and therefore it can be determined that the image recognition model is wrong.

[0062] According to the embodiments of the present disclosure, the recognition results of the image recognition model can be accurately judged by the number and position of the algorithm marks and quality control marks, so as to determine whether the image recognition model is wrong, whether the recognition ability of the image recognition model needs to be further improved, and improve the efficiency of model fine-tuning.

[0063] According to an embodiment of the present disclosure, a fine-tuning dataset is determined based on one or more image data and quality control marks corresponding to each of the one or more image data, including: for each image data, determining the quality control mark corresponding to the image data as a label of the image data; and determining the fine-tuning dataset based on one or more image data and the labels corresponding to each of the one or more image data.

[0064] According to an embodiment of the present disclosure, since the quality control model has a stronger ability to recognize image data, the quality control mark of the image data determined by the quality control model can be determined as a label of the image data, and the label and the image data can be used as a set of fine-tuning data. Based on multiple sets of fine-tuning data, a fine-tuning data set can be determined.

[0065] According to an embodiment of the present disclosure, quality control marks are determined as labels for image data, and a fine-tuning data set is further determined, so that the recognition capability of the fine-tuned image recognition model is sufficient to recognize previously erroneous image data, thereby improving the recognition accuracy of the image recognition model.

[0066] According to an embodiment of the present disclosure, image data screening conditions include a target time period, a target data source, and a type of object to be identified; based on the image data screening conditions, one or more image data are determined, including: determining the target time period data based on the target time period from all data acquired from the target data source; and filtering out one or more image data from the target time period data based on the type of object to be identified.

[0067] According to an embodiment of the present disclosure, the target time period may be, for example, a specified day, a specified hour, etc. The target data source may be, for example, a specified one or several image acquisition devices, and the image acquisition device may include security inspection instruments such as an X-ray machine. The types of objects to be identified may include, for example, liquid containers, sharp objects, electronic devices, etc.

[0068] According to an embodiment of the present disclosure, all data collected by the target data source is determined, and the target period data of the target period is determined by screening based on the target period. From the target period data, one or more image data are screened based on the type of object to be identified, wherein the type of object to be identified can be determined based on the recognition result of the image recognition model.

[0069] According to the embodiments of the present disclosure, by filtering image data using image data filtering conditions, the amount of data transmission can be reduced, the utilization of network resources and hardware resources can be improved, and the load of data transmission from the client to the server can be reduced, thereby improving the processing efficiency of the client.

[0070] According to an embodiment of the present disclosure, a quality control model is obtained by training based on edge images and image labels corresponding to the edge images, wherein the object to be identified in the edge image is in an edge state, and the edge state includes being partially occluded and the presence of similar objects; the quality control model is obtained in the following manner: determining multiple sample edge images from multiple sample image data, wherein the object to be identified in the sample edge image is in an edge state; determining sample labels of the sample edge images based on the objects to be identified in the edge state in the sample edge images; determining sample labels of the remaining sample images based on multiple remaining sample images other than the multiple sample edge images in the multiple sample image data, and the objects to be identified in the remaining sample images; determining a quality control training set for training an initial quality control model based on multiple sample edge images, multiple remaining sample images, and sample labels corresponding to the multiple sample edge images and the multiple remaining sample images respectively; and adjusting parameters of the initial quality control model based on the quality control training set to determine the quality control model.

[0071] According to an embodiment of the present disclosure, the sample image data is image data used to train a quality control model or an image recognition model, and the position and number of the objects to be recognized included in the sample image data are sample labels of the sample image data. From a plurality of sample image data, image data with edge state objects are determined and determined as sample edge images, and image data without edge state objects are determined as remaining sample images.

[0072] According to the embodiments of the present disclosure, since the recognition capability of the quality control model needs to be able to perform quality control on the recognition effect of the image recognition model, the quality control model needs to be able to accurately recognize the objects to be recognized in all situations. The initial quality control model is trained using the quality control training set consisting of the sample edge image and the remaining sample image and its sample label, thereby ensuring that the quality control model can recognize the image data of the objects to be recognized in various states.

[0073] According to the embodiments of the present disclosure, by forming a quality control training set with all the data in the sample image data and training the initial quality control model, the recognition capability of the quality control model can correctly identify the objects to be identified in various situations, thereby ensuring the accuracy and reliability of quality control of the recognition results of the image recognition model using the quality control capability.

[0074] According to an embodiment of the present disclosure, an image recognition model is obtained in the following manner: based on a plurality of remaining sample images and sample labels respectively corresponding to the plurality of remaining sample images, a recognition training set for training an initial image recognition model is determined; and based on the recognition training set, parameters of the initial image recognition model are adjusted to determine the image recognition model.

[0075] According to the embodiments of the present disclosure, since the image recognition model needs to be configured in each client, and the client may not configure too many computing resources for ease of use, in order to ensure that the client can perform image recognition normally and ensure the processing efficiency of image recognition, the number of parameters of the image recognition model should be controlled to be as small as possible. Under the above premise, in order to ensure that the image recognition model can recognize the objects to be recognized in most common scenarios, the initial image recognition model can be trained using the recognition training set composed of the remaining sample images and the corresponding sample labels, thereby ensuring that the image recognition model can correctly recognize image data without edge state objects.

[0076] According to an embodiment of the present disclosure, the initial image recognition model is trained using a recognition training set that includes objects to be recognized in common scenarios, which can reduce the number of model parameters, thereby avoiding excessive impact on the running speed after configuring it in the client, thereby improving the efficiency of image recognition.

[0077] According to an embodiment of the present disclosure, the model fine-tuning method also includes: uploading the target image recognition model to a version control server and updating the current model version saved in the version control server; determining the client model version corresponding to the version verification request in response to a version verification request sent by the client; and in the event that the current model version is inconsistent with the client model version, sending the target image recognition model to the client, so that the client updates the image recognition model based on the target image recognition model.

[0078] According to an embodiment of the present disclosure, after completing fine-tuning of the target image recognition model, it is uploaded to the version control server and the current model version is updated. The version verification request may include the version number of the image recognition model currently applied by the client, that is, the client model version, and the current model version is compared with the client model version. When the current model version is consistent with the client model version, it indicates that the image recognition model currently applied by the client is the latest version of the model and does not need to be updated. When the current model version is inconsistent with the client model version, it indicates that the image recognition model currently applied by the client is not the latest version of the model, and the target image recognition model is sent to the client, and the client uses the target image recognition model to update and replace the target recognition model.

[0079] According to an embodiment of the present disclosure, the version check request may be triggered by the client in response to a periodic task. For example, a periodic task is triggered once a week, and the client sends a version check request. The version check request may also be sent when the image recognition task load of the client is light, so as to reduce the impact on the image recognition task caused by the image recognition model version update.

[0080] According to the embodiments of the present disclosure, by uploading the updated model to the version control server, the client model version in the client is verified using the updated current model version, and when the target image recognition model saved in the version control server is higher than the image recognition model version configured in the client, the image recognition model in the client is updated. Through the client's version verification request, the model configured in the client can be updated in time when the version needs to be updated. By controlling the sending time of the version verification request, the impact on the normal operation of the client can be reduced and the system stability can be improved.

[0081] Figure 3 The figure schematically shows a flow chart of a model fine-tuning method according to another embodiment of the present disclosure.

[0082] like Figure 3 As shown, the model fine-tuning method includes operations S301 to S310.

[0083] In operation S301, in response to the triggering of the first scheduled task, image data is determined. The first scheduled task can be used to collect data regularly and determine whether the image recognition model needs to be fine-tuned and updated. In operation S302, the image data is processed using the image recognition model to determine the algorithm mark. In operation S303, the image data is processed using the quality control model to determine the quality control mark. In operation S304, the algorithm mark and the quality control mark are compared to determine whether the image recognition model is wrong. If not, perform operation S301; if yes, perform operation S304. In operation S305, based on the quality control mark and the image data, a fine-tuning data set is determined. In operation S306, based on the fine-tuning data set, the image recognition model is fine-tuned to obtain a target image recognition model. In operation S307, the target image recognition model is uploaded to the version control server and the current model version is updated. In operation S308, in response to the triggering of the second scheduled task, the client sends a version check request to the server. In operation S309, after receiving the version check request, the server determines whether the current model version is inconsistent with the client model version. If not, perform operation S308; if yes, perform operation S310. In operation S310, the target image recognition model is sent to the client, so that the client updates the image recognition model installed locally on the client based on the target image recognition model.

[0084] Based on the above model fine-tuning method, the present disclosure also provides a model fine-tuning device. Figure 4 The device is described in detail.

[0085] Figure 4 The structural block diagram of the model fine-tuning device according to an embodiment of the present disclosure is schematically shown.

[0086] like Figure 4 As shown, the model fine-tuning device 400 of this embodiment includes an image determination module 410 , a first label determination module 420 , a second label determination module 430 , a data set determination module 440 and a parameter fine-tuning module 450 .

[0087] The image determination module 410 is used to determine one or more image data based on the image data screening condition. In one embodiment, the image determination module 410 can be used to perform the operation S210 described above, which will not be described in detail here.

[0088] The first tag determination module 420 is used to identify the image data using the image recognition model to obtain one or more algorithm tags of the image data. In one embodiment, the first tag determination module 420 can be used to perform the operation S220 described above, which will not be repeated here.

[0089] The second mark determination module 430 is used to identify the image data using the quality control model to obtain one or more quality control marks of the image data. In one embodiment, the second mark determination module 430 can be used to perform the operation S230 described above, which will not be repeated here.

[0090] The data set determination module 440 is used to determine the fine-tuning data set based on the one or more image data and the quality control marks corresponding to the one or more image data when it is determined that the algorithm mark and the quality control mark corresponding to the image data indicate that the image recognition model is wrong. In one embodiment, the data set determination module 440 can be used to perform the operation S240 described above, which will not be repeated here.

[0091] The parameter fine-tuning module 450 is used to fine-tune the parameters of the image recognition model based on the fine-tuning data set to obtain a fine-tuned target image recognition model. In one embodiment, the parameter fine-tuning module 450 can be used to perform the operation S250 described above, which will not be described in detail here.

[0092] According to an embodiment of the present disclosure, the model fine-tuning apparatus 400 further includes a marking attribute determination module, a first result determination module and a second result determination module.

[0093] The marking attribute determination module is used to determine the marking quantity and marking position of the algorithm marking and quality control marking corresponding to each image data.

[0094] The first result determination module is used to determine that the algorithm mark and the quality control mark corresponding to the image data represent that the image recognition model has no errors when the number of marks and the positions of the marks are consistent.

[0095] The second result determination module is used to determine that the algorithm mark and the quality control mark corresponding to the image data represent an error in the image recognition model when the number of marks is different or the positions of the algorithm mark and the quality control mark are inconsistent.

[0096] According to an embodiment of the present disclosure, the data set determination module 440 includes a quality control label determination submodule and a data set determination submodule.

[0097] The quality control label determination submodule is used to determine, for each image data, a quality control mark corresponding to the image data as a label of the image data.

[0098] The data set determination submodule is used to determine a fine-tuning data set based on one or more image data and labels corresponding to the one or more image data.

[0099] According to an embodiment of the present disclosure, the image determination module 410 includes a period data determination submodule and an image determination submodule.

[0100] The time period data determination submodule is used to determine the target time period data based on the target time period from all the data obtained from the target data source.

[0101] The image determination submodule is used to filter out one or more image data from the target period data based on the type of the object to be identified.

[0102] According to an embodiment of the present disclosure, the model fine-tuning device 400 further includes a sample image determination module, an edge label determination module, a remaining label determination module, a quality control data set determination module and a quality control model training module.

[0103] The sample image determination module is used to determine a plurality of sample edge images from a plurality of sample image data, wherein the objects to be identified in the sample edge images are in an edge state.

[0104] The edge label determination module is used to determine the sample label of the sample edge image based on the to-be-identified object in the edge state in the sample edge image.

[0105] The remaining label determination module is used to determine the sample labels of the remaining sample images based on the multiple remaining sample images excluding the multiple sample edge images in the multiple sample image data and the objects to be identified in the remaining sample images.

[0106] The quality control data set determination module is used to determine a quality control training set for training an initial quality control model based on multiple sample edge images, multiple remaining sample images, and sample labels corresponding to the multiple sample edge images and the multiple remaining sample images respectively.

[0107] The quality control model training module is used to adjust the parameters of the initial quality control model based on the quality control training set to determine the quality control model.

[0108] According to an embodiment of the present disclosure, the model fine-tuning device 400 further includes a recognition training set determination module and a recognition model training module.

[0109] The recognition training set determination module is used to determine the recognition training set used to train the initial image recognition model based on the multiple remaining sample images and the sample labels respectively corresponding to the multiple remaining sample images.

[0110] The recognition model training module is used to adjust the parameters of the initial image recognition model based on the recognition training set to determine the image recognition model.

[0111] According to an embodiment of the present disclosure, the model fine-tuning device 400 further includes a model uploading module, a version determining module and a model sending module.

[0112] The model upload module is used to upload the target image recognition model to the version control server and update the current model version saved in the version control server.

[0113] The version determination module is used to respond to a version verification request sent by the client and determine the client model version corresponding to the version verification request.

[0114] The model sending module is used to send the target image recognition model to the client when the current model version is inconsistent with the client model version, so that the client can update the image recognition model based on the target image recognition model.

[0115] According to an embodiment of the present disclosure, any multiple modules among the image determination module 410, the first marker determination module 420, the second marker determination module 430, the data set determination module 440 and the parameter fine-tuning module 450 can be combined into one module for implementation, or any one of the modules can be split into multiple modules. Alternatively, at least part of the functions of one or more of these modules can be combined with at least part of the functions of other modules and implemented in one module. According to an embodiment of the present disclosure, at least one of the image determination module 410, the first marker determination module 420, the second marker determination module 430, the data set determination module 440 and the parameter fine-tuning module 450 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application specific integrated circuit (ASIC), or can be implemented by hardware or firmware such as any other reasonable way of integrating or packaging the circuit, or implemented in any one of the three implementation modes of software, hardware and firmware or in a suitable combination of any of them. Alternatively, at least one of the image determination module 410, the first marker determination module 420, the second marker determination module 430, the data set determination module 440 and the parameter fine-tuning module 450 can be at least partially implemented as a computer program module, which can perform corresponding functions when executed.

[0116] Figure 5 A block diagram of an electronic device suitable for implementing a model fine-tuning method according to an embodiment of the present disclosure is schematically shown.

[0117] like Figure 5 As shown, the electronic device 500 according to an embodiment of the present disclosure includes a processor 501, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 502 or a program loaded from a storage part 508 to a random access memory (RAM) 503. The processor 501 may include, for example, a general-purpose microprocessor (such as a CPU), an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (for example, an application-specific integrated circuit (ASIC)), etc. The processor 501 may also include an onboard memory for caching purposes. The processor 501 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present disclosure.

[0118] In RAM 503, various programs and data required for the operation of electronic device 500 are stored. Processor 501, ROM 502 and RAM 503 are connected to each other via bus 504. Processor 501 performs various operations of the method flow according to the embodiment of the present disclosure by executing the program in ROM 502 and / or RAM 503. It should be noted that the program can also be stored in one or more memories other than ROM 502 and RAM 503. Processor 501 can also perform various operations of the method flow according to the embodiment of the present disclosure by executing the program stored in the one or more memories.

[0119] According to an embodiment of the present disclosure, the electronic device 500 may further include an input / output (I / O) interface 505, which is also connected to the bus 504. The electronic device 500 may further include one or more of the following components connected to the input / output (I / O) interface 505: an input portion 506 including a keyboard, a mouse, etc.; an output portion 507 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage portion 508 including a hard disk, etc.; and a communication portion 509 including a network interface card such as a LAN card, a modem, etc. The communication portion 509 performs communication processing via a network such as the Internet. A drive 510 is also connected to the input / output (I / O) interface 505 as needed. A removable medium 511, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 510 as needed, so that a computer program read therefrom is installed into the storage portion 508 as needed.

[0120] The present disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or may exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiment of the present disclosure is implemented.

[0121] According to an embodiment of the present disclosure, the computer-readable storage medium may be a non-volatile computer-readable storage medium, for example, it may include but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, an apparatus or a device. For example, according to an embodiment of the present disclosure, the computer-readable storage medium may include the ROM 502 and / or RAM 503 described above and / or one or more memories other than ROM 502 and RAM 503.

[0122] The embodiment of the present disclosure also includes a computer program product, which includes a computer program, and the computer program contains program code for executing the method shown in the flowchart. When the computer program product is run in a computer system, the program code is used to enable the computer system to implement the method provided by the embodiment of the present disclosure.

[0123] The above functions defined in the system / device of the embodiment of the present disclosure are performed when the computer program is executed by the processor 501. According to the embodiment of the present disclosure, the system, device, module, unit, etc. described above can be implemented by a computer program module.

[0124] In one embodiment, the computer program may rely on tangible storage media such as optical storage devices, magnetic storage devices, etc. In another embodiment, the computer program may also be transmitted and distributed in the form of signals on a network medium, and downloaded and installed through the communication part 509, and / or installed from the removable medium 511. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0125] In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 509, and / or installed from the removable medium 511. When the computer program is executed by the processor 501, the above functions defined in the system of the embodiment of the present disclosure are performed. According to the embodiment of the present disclosure, the system, device, means, module, unit, etc. described above can be implemented by a computer program module.

[0126] According to an embodiment of the present disclosure, the program code for executing the computer program provided by the embodiment of the present disclosure can be written in any combination of one or more programming languages. Specifically, these computing programs can be implemented using high-level process and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, Java, C++, python, "C" language or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on the remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (for example, using an Internet service provider to connect through the Internet).

[0127] The flow charts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0128] It will be appreciated by those skilled in the art that the features described in the various embodiments of the present disclosure may be combined and / or combined in a variety of ways, even if such combinations or combinations are not explicitly described in the present disclosure. In particular, without departing from the spirit and teachings of the present disclosure, the features described in the various embodiments of the present disclosure may be combined and / or combined in a variety of ways. All of these combinations and / or combinations fall within the scope of the present disclosure.

[0129] The embodiments of the present disclosure are described above. However, these embodiments are only for illustrative purposes and are not intended to limit the scope of the present disclosure. Although the embodiments are described above, this does not mean that the measures in the various embodiments cannot be used in combination to advantage. Without departing from the scope of the present disclosure, those skilled in the art may make a variety of substitutions and modifications, which should all fall within the scope of the present disclosure.

Claims

1. A model fine-tuning method, characterized in that: The method comprises: Determine one or more image data based on the image data screening condition; Using an image recognition model, the image data is recognized to obtain algorithmic labels for the one or more image data; Using the quality control model to identify the image data, and obtain the quality control marks of the one or more image data; In the case where it is determined that the algorithm mark and the quality control mark corresponding to the image data in the one or more image data represent that the image recognition model has an error, determining a fine-tuning data set based on the one or more image data and the quality control marks corresponding to each of the one or more image data; and The image recognition model is fine-tuned based on the fine-tuning data set to obtain a fine-tuned target image recognition model.

2. The method according to claim 1, characterized in that The method further comprises: For each of the image data, Determining the number and position of each of the algorithm mark and the quality control mark corresponding to the image data; In the case where the number of marks and the positions of the marks are consistent, determining that the algorithm mark and the quality control mark corresponding to the image data represent that the image recognition model is error-free; and In the case where the number of the marks is different or the positions of the algorithm mark and the quality control mark are inconsistent, determining that the algorithm mark and the quality control mark corresponding to the image data represent an error in the image recognition model.

3. The method according to claim 1, characterized in that The determining of the fine-tuning data set based on the one or more image data and the quality control marks respectively corresponding to the one or more image data comprises: For each of the image data, determining the quality control mark corresponding to the image data as a label of the image data; and The fine-tuning dataset is determined based on the one or more image data and labels corresponding to the one or more image data.

4. The method according to claim 1, characterized in that: The image data screening conditions include target time period, target data source, and type of object to be identified; The determining one or more image data based on the image data screening condition includes: Determining target period data based on the target period from all data acquired from the target data source; as well as Based on the type of the object to be identified, the one or more image data are filtered out from the target period data.

5. The method according to claim 4, characterized in that The quality control model is obtained by training based on an edge image and an image label corresponding to the edge image, wherein the object to be identified in the edge image is in an edge state, and the edge state includes being partially blocked and having similar objects; The quality control model is obtained based on the following method: Determine a plurality of sample edge images from a plurality of sample image data, wherein the object to be identified in the sample edge images is in an edge state; Determining a sample label of the sample edge image based on the to-be-identified object in the edge state in the sample edge image; Determine sample labels of the remaining sample images based on a plurality of remaining sample images excluding the plurality of sample edge images among the plurality of sample image data, and the objects to be identified in the remaining sample images; Determining a quality control training set for training an initial quality control model based on the multiple sample edge images, the multiple remaining sample images, and the sample labels corresponding to the multiple sample edge images and the multiple remaining sample images respectively; and The parameters of the initial quality control model are adjusted based on the quality control training set to determine the quality control model.

6. The method according to claim 5, characterized in that The image recognition model is obtained based on the following method: Determining a recognition training set for training an initial image recognition model based on the plurality of remaining sample images and the sample labels respectively corresponding to the plurality of remaining sample images; as well as The parameters of the initial image recognition model are adjusted based on the recognition training set to determine the image recognition model.

7. The method according to any one of claims 1 to 6, characterized in that: The method further comprises: Uploading the target image recognition model to a version control server, and updating the current model version stored in the version control server; In response to a version check request sent by the client, determining a client model version corresponding to the version check request; and In the case that the current model version is inconsistent with the client model version, the target image recognition model is sent to the client so that the client updates the image recognition model based on the target image recognition model.

8. A model fine-tuning device, characterized in that: The device comprises: An image determination module, used to determine one or more image data based on an image data screening condition; A first label determination module, configured to identify the image data using an image recognition model to obtain an algorithm label for the one or more image data; A second mark determination module, used to identify the image data using a quality control model to obtain quality control marks for the one or more image data; a data set determination module, configured to determine a fine-tuning data set based on the one or more image data and the quality control marks corresponding to the one or more image data, if it is determined that the algorithm mark and the quality control mark corresponding to the one or more image data indicate that the image recognition model has an error; and A parameter fine-tuning module is used to fine-tune the parameters of the image recognition model based on the fine-tuning data set to obtain a fine-tuned target image recognition model.

9. An electronic device, comprising: one or more processors; a memory for storing one or more computer programs, It is characterized in that the one or more processors execute the one or more computer programs to implement the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program or instruction stored thereon, characterized in that: When the computer program or instruction is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

11. A computer program product comprising a computer program or instructions, characterized in that When the computer program or instruction is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images