Data processing method and related apparatus, and device and storage medium

Abstract

Disclosed in the present application are a data processing method and related apparatus, and a device and a storage medium. The method comprises: sending to a server K images which are captured in the current site environment, and the server acquiring K first prediction results by means of an image recognition model; constructing a fine-tuning training set according to the K images and the K first prediction results; by means of a model to be trained, acquiring a second prediction result of each image in the fine-tuning training set; according to the second prediction result of each image and a first prediction result of the image in the fine-tuning training set, updating model parameters of the model to be trained, so as to obtain a local recognition model and model tuning parameters; and if a model fine-tuning condition is met, sending the model tuning parameters to the server, and the server updating model parameters of the image recognition model according to a set of model tuning parameters. Thus, the present application enables an image recognition model to be applicable to various site environments, improves the precision of model recognition, and also saves on processing resources of a server and improves the efficiency of model learning.

Description

A data processing method, related device, equipment and storage medium

[0001] This application claims priority to the Chinese patent application filed with the China Patent Office on July 20, 2023, application number 202310894889.1, and application name “A method, related devices, equipment and storage medium for data processing”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of artificial intelligence technology, and in particular to a data processing method, related apparatus, equipment, and storage medium. Background Art

[0003] In recent years, artificial intelligence (AI) technology has continued to develop, and its application in image recognition has become widespread. AI can use complex algorithms and models to identify biological objects (e.g., faces, irises, palm prints, etc.), objects, and text in images, enabling intelligent image processing and analysis.

[0004] Capturing images in different environments often involves complex environmental factors, such as varying light levels and background noise. These factors can affect image recognition accuracy. Therefore, related techniques involve collecting a large number of images in different environments for model training to enhance the model's recognition capabilities.

[0005] However, in related technologies, on the one hand, the images used for model training rarely cover a wide range of environments, thus limiting the types of samples the model can learn from, resulting in poor learning results. On the other hand, training the model on a large number of images not only requires a lot of computing power but also consumes a considerable amount of time. Currently, no effective solutions have been proposed to address these issues.

[0006] Summary of the Invention

[0007] The embodiments of the present application provide a data processing method, related devices, equipment and storage medium, which not only enable the image recognition model to be applicable to various specific on-site environments, thereby improving the accuracy of model recognition, but also save server processing resources and improve the efficiency of model learning.

[0008] In view of this, the present application provides, on one hand, a data processing method, which is executed by a field terminal and includes:

[0009] Under the current scene environment, K images are captured by an image acquisition device, where K is an integer greater than or equal to 1;

[0010] Sending K images to the server, so that the server obtains K first prediction results based on the K images through the image recognition model;

[0011] Constructing a fine-tuning training set based on the K images and the K first prediction results sent by the server, wherein the fine-tuning training set includes K groups of fine-tuning training data, each group of fine-tuning training data includes an image and the first prediction result of the image;

[0012] Fine-tuning the model to be trained on the on-site terminal using the fine-tuning training set. During fine-tuning the model to be trained on the on-site terminal, obtaining a second prediction result corresponding to each image by the model to be trained based on the images included in each group of fine-tuning training data in the fine-tuning training set, and updating model parameters of the model to be trained based on the second prediction result corresponding to each image and the first prediction result of the image in the fine-tuning training set to obtain a local recognition model and model adjustment parameters corresponding to the local recognition model.

[0013] If the local recognition model meets the model fine-tuning conditions, the model adjustment parameters are sent to the server so that the server updates the model parameters of the image recognition model according to the model adjustment parameter set from at least one terminal, wherein the model adjustment parameter set includes the model adjustment parameters.

[0014] Another aspect of the present application provides a data processing method, which is executed by a server and includes:

[0015] Receiving K images sent by the on-site terminal, wherein the K images are captured by the on-site terminal through a collection device under the current on-site environment, and K is an integer greater than or equal to 1;

[0016] Based on K images, obtain K first prediction results through the image recognition model;

[0017] Sending K first prediction results to an on-site terminal so that the on-site terminal constructs a fine-tuning training set based on the K images and the K first prediction results, and using the fine-tuning training set to fine-tune the model to be trained on the on-site terminal. In the process of fine-tuning the model to be trained on the on-site terminal, based on the images included in each group of fine-tuning training data in the fine-tuning training set, a second prediction result corresponding to each image is obtained by the model to be trained, and according to the second prediction result corresponding to each image and the first prediction result of the image in the fine-tuning training set, a model parameter of the model to be trained is updated to obtain a local recognition model and a model adjustment parameter corresponding to the local recognition model, wherein the fine-tuning training set includes K groups of fine-tuning training data, and each group of fine-tuning training data includes an image and the first prediction result of the image;

[0018] If the local recognition model meets the model fine-tuning conditions, the model adjustment parameters sent by the on-site terminal are received;

[0019] In a case where a model adjustment parameter set is obtained from at least one terminal, the model parameters of the image recognition model are updated, wherein the model adjustment parameter set includes the model adjustment parameters.

[0020] Another aspect of the present application provides a data processing device, which is deployed on a field terminal and includes:

[0021] a shooting module, configured to capture K images using an image acquisition device under a current on-site environment, wherein K is an integer greater than or equal to 1;

[0022] a sending module, configured to send the K images to a server, so that the server obtains K first prediction results based on the K images through an image recognition model;

[0023] an acquisition module, configured to construct a fine-tuning training set based on the K images and the K first prediction results sent by the server, wherein the fine-tuning training set includes K groups of fine-tuning training data, each group of fine-tuning training data including an image and the first prediction result of the image;

[0024] an updating module, configured to fine-tune the model to be trained on the on-site terminal using the fine-tuning training set; during fine-tuning the model to be trained on the on-site terminal, obtaining a second prediction result corresponding to each image by the model to be trained based on the images included in each group of fine-tuning training data in the fine-tuning training set; and updating model parameters of the model to be trained based on the second prediction result corresponding to each image and the first prediction result of the image in the fine-tuning training set, thereby obtaining a local recognition model and model adjustment parameters corresponding to the local recognition model;

[0025] The sending module is also used to send the model adjustment parameters to the server if the local recognition model meets the model fine-tuning conditions, so that the server updates the model parameters of the image recognition model according to the model adjustment parameter set from at least one terminal, wherein the model adjustment parameter set includes the model adjustment parameters.

[0026] Another aspect of the present application provides a data processing device, which is deployed on a server and includes:

[0027] A receiving module, configured to receive K images sent by a field terminal, wherein the K images are captured by the field terminal through a collection device under a current field environment, and K is an integer greater than or equal to 1;

[0028] An acquisition module, configured to acquire K first prediction results based on the K images through an image recognition model;

[0029] a sending module, configured to send K first prediction results to an on-site terminal, so that the on-site terminal constructs a fine-tuning training set based on the K images and the K first prediction results, and uses the fine-tuning training set to fine-tune the model to be trained on the on-site terminal; in the process of fine-tuning the model to be trained on the on-site terminal, based on the images included in each group of fine-tuning training data in the fine-tuning training set, obtain a second prediction result corresponding to each image through the model to be trained; and based on the second prediction result corresponding to each image and the first prediction result of the image in the fine-tuning training set, update the model parameters of the model to be trained to obtain a local recognition model and model adjustment parameters corresponding to the local recognition model, wherein the fine-tuning training set includes K groups of fine-tuning training data, and each group of fine-tuning training data includes an image and the first prediction result of the image;

[0030] The receiving module is further configured to receive the model adjustment parameters sent by the on-site terminal if the local recognition model satisfies the model fine-tuning conditions;

[0031] An updating module is used to update the model parameters of the image recognition model when a model adjustment parameter set is obtained from at least one terminal, wherein the model adjustment parameter set includes the model adjustment parameters.

[0032] On the other hand, the present application provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the above-mentioned methods when executing the computer program.

[0033] Another aspect of the present application provides a computer-readable storage medium having a computer program stored thereon, which implements the above-mentioned methods when the computer program is executed by a processor.

[0034] Another aspect of the present application provides a computer program product, including a computer program, which implements the above-mentioned methods when executed by a processor.

[0035] It can be seen from the above technical solutions that the embodiments of the present application have the following advantages:

[0036] In an embodiment of the present application, a data processing method is provided. First, in the current on-site environment, an on-site terminal obtains K images through an image acquisition device. The on-site environment may affect the accuracy of image recognition, so model fine-tuning is required. To achieve fine-tuning, K images are sent to a server. Based on the K images, the server obtains K first prediction results through an image recognition model, thereby using the first prediction results as annotation information for the corresponding images. Therefore, the on-site terminal can construct a fine-tuning training set based on the K images and the K first prediction results sent by the server, and then use the fine-tuning training set to fine-tune the model to be trained on the on-site terminal. In the process of fine-tuning the model to be trained on the on-site terminal, based on the images included in each group of fine-tuning training data in the fine-tuning training set, the second prediction result corresponding to each image is obtained by the model to be trained, thereby using the second prediction result as prediction information. Next, the on-site terminal updates the model parameters of the model to be trained based on the second prediction result corresponding to each image and the first prediction result of the image in the fine-tuning training set, thereby obtaining a local recognition model and model adjustment parameters corresponding to the local recognition model. If the local recognition model meets the model fine-tuning conditions, the on-site terminal sends the model adjustment parameters to the server, which then updates the model parameters of the image recognition model based on the model adjustment parameter set from at least one terminal. In this way, the terminal can fine-tune the local model based on the images captured in the on-site environment and report the fine-tuned target parameters to the server. The server updates the image recognition model based on the parameter sets reported by each terminal. This not only enables the image recognition model to be adapted to various specific on-site environments, thereby improving the accuracy of model recognition, but also saves server processing resources and improves the efficiency of model learning. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] FIG1 is a schematic diagram of an implementation environment of the data processing method according to an embodiment of the present application;

[0038] FIG2 is a schematic diagram of an implementation environment of the image recognition method according to an embodiment of the present application;

[0039] FIG3 is a flow chart of a data processing method according to an embodiment of the present application;

[0040] FIG4 is a schematic diagram of an interactive process for voting based on a synchronous voting mechanism in an embodiment of the present application;

[0041] FIG5 is a schematic diagram of another interactive process for voting based on a synchronous voting mechanism in an embodiment of the present application;

[0042] FIG6 is a schematic diagram of establishing an association relationship between terminals in an embodiment of the present application;

[0043] FIG7 is another schematic diagram of establishing an association relationship between terminals in an embodiment of the present application;

[0044] FIG8 is another schematic diagram of establishing an association relationship between terminals in an embodiment of the present application;

[0045] FIG9 is a schematic diagram of an overall flow chart of a data processing method according to an embodiment of the present application;

[0046] FIG10 is a flow chart of a data processing method according to an embodiment of the present application;

[0047] FIG11 is a schematic diagram of a framework for data processing between a terminal and a server according to an embodiment of the present application;

[0048] FIG12 is a schematic diagram of a data processing device according to an embodiment of the present application;

[0049] FIG13 is another schematic diagram of a data processing device according to an embodiment of the present application;

[0050] FIG14 is a schematic structural diagram of a computer device in an embodiment of the present application. DETAILED DESCRIPTION

[0051] The embodiments of the present application provide a data processing method, related devices, equipment and storage medium, which not only enable the image recognition model to be applicable to various specific on-site environments, thereby improving the accuracy of model recognition, but also save server processing resources and improve the efficiency of model learning.

[0052] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequential order. It should be understood that the numbers used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can, for example, be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "corresponding to" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0053] Image acquisition often presents complex environmental factors that can affect image recognition accuracy. Therefore, to improve image recognition accuracy, large-scale training data and models with large parameter counts can be used for training. Large-scale training data provides the model with sufficient material to learn from, while large-scale parameters enhance the model's learning capabilities and make it easier to absorb the knowledge in the material. However, training data often lacks coverage for all real-world environments, and large-scale training data and parameter counts increase the difficulty of model training.

[0054] Based on this, an embodiment of this application provides a data processing method that enables on-site model fine-tuning based on real-time collected images, synchronization of fine-tuning results, and optimization of the backend model, thereby improving the effectiveness and stability of image recognition. The data processing method of this application can be applied in at least one of the following scenarios.

[0055] 1. Biometric recognition scenario;

[0056] Biometrics technology uses computers, optics, acoustics, biosensors, and biostatistics to authenticate individuals using the inherent physiological characteristics of the human body (such as the palm, face, and iris). Because images collected in different environments often vary, the accuracy of biometric models remains challenging. The following uses palmprint recognition as an example.

[0057] Taking into account the complexity of the on-site environment, there are differences in light, background noise, etc. For example, the light is darker in the laboratory and brighter in the outdoor environment. If each palm-swiping terminal uses a local recognition model with the same model parameters to recognize the collected images, it may lead to large differences in the recognition results. Therefore, in this application, the palm-swiping terminal adopts a local fine-tuning strategy to train the local recognition model, that is, the palm-swiping terminals in different on-site environments can respectively adopt corresponding model tuning strategies. In this way, the effect and stability of palm-swiping recognition can be improved. At the same time, the palm-swiping terminal also needs to feed back the fine-tuned model adjustment parameters to the server side, and the server side maintains an image recognition model. Based on this, the server tunes the image recognition model based on the model adjustment parameters reported by each palm-swiping terminal, thereby improving the recognition ability of the image recognition model.

[0058] Compared to the local recognition model of the palm-swiping terminal, the server-side image recognition model has more model parameters and a more complex model structure. Therefore, the image recognition model has stronger computing power and higher recognition accuracy. If a palm-swiping terminal cannot recognize a palm image collected on-site using its local recognition model, the palm-swiping terminal can send the palm image to the server. The server will call the image recognition model to recognize the palm image and feedback the recognition result to the palm-swiping terminal to execute the corresponding service.

[0059] 2. Autonomous driving scenarios;

[0060] Image recognition is a crucial component of autonomous driving. Image recognition uses computer technology to extract features from images and then classify, identify, and judge them. In autonomous driving, image recognition is primarily responsible for identifying various objects around the vehicle, such as pedestrians, road signs, and traffic lights, to assist the autonomous vehicle in making appropriate decisions.

[0061] Taking into account the complexity of the driving environment, there are differences in different weather, driving sections and time. For example, the light is weaker on rainy days, while the light is stronger on sunny days. For example, the light is stronger when driving on an overpass, but weaker when driving in a tunnel. For another example, the light is stronger at noon, while the light is weaker at night. If each on-board terminal uses a local recognition model with the same model parameters to recognize the collected images, it may lead to large differences in recognition results. Therefore, in this application, the on-board terminal adopts a local fine-tuning strategy to train the local recognition model, that is, the on-board terminals in different field environments can adopt corresponding model tuning strategies respectively. In this way, the effect and stability of object recognition can be improved. At the same time, the on-board terminal also needs to feed back the fine-tuned model adjustment parameters to the server side, and the server side maintains an image recognition model. Based on this, the server tunes the image recognition model based on the model adjustment parameters reported by each on-board terminal, thereby improving the recognition ability of the image recognition model.

[0062] If a vehicle-mounted terminal cannot recognize a road image collected on-site using a local recognition model, the vehicle-mounted terminal can send the road image to the server, which will call the image recognition model to recognize the road image and feedback the recognition result to the vehicle-mounted terminal, so that the vehicle can make corresponding feedback in a timely manner.

[0063] 3. Security precautions scenarios;

[0064] Security systems use optical fiber, coaxial cable, or microwaves to transmit video signals within a closed loop, forming a self-contained, complete system from camera capture to image display and recording. Security systems not only significantly extend the human eye's observation range but also expand its capabilities, allowing them to replace manual labor for extended periods of time in harsh environments.

[0065] Taking into account the complexity of the actual environment, there are differences in weather, deployment location, time, etc. If each security system uses a local recognition model with the same model parameters to recognize the collected images, it may lead to large differences in the recognition results. Therefore, in this application, the security system adopts a local fine-tuning strategy to train the local recognition model, that is, the security systems in different on-site environments can adopt corresponding model tuning strategies respectively. In this way, the effect and stability of object recognition can be improved. At the same time, the security system also needs to feed back the fine-tuned model adjustment parameters to the server side, and the server side maintains an image recognition model. Based on this, the server tunes the image recognition model based on the model adjustment parameters reported by each vehicle-mounted terminal, thereby improving the recognition ability of the image recognition model.

[0066] If a security system cannot identify an image captured on-site using its local recognition model, it can send the image to a server, which will use the image recognition model to identify the image and provide feedback to the security system. If a security risk is identified, an alert can be triggered.

[0067] It should be noted that the above application scenarios are only examples, and the data processing method provided in this embodiment can also be applied to other scenarios, which are not limited here.

[0068] It is understandable that in this application, computer vision (CV) technology can be used to recognize images. Among them, CV technology is a science that studies how to make machines "see". More specifically, it refers to machine vision such as using cameras and computers to replace human eyes to identify and measure targets, and further performing graphic processing so that computer processing becomes an image that is more suitable for human eye observation or transmission to instrument detection. As a scientific discipline, CV studies related theories and technologies, attempting to establish an artificial intelligence system that can obtain information from images or multidimensional data. CV technology generally includes image processing, image recognition, image semantic understanding, image retrieval, optical character recognition (OCR), video processing, video semantic understanding, video content / behavior recognition, three-dimensional object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous positioning and map construction, and also includes common biometric recognition technologies such as face recognition and fingerprint recognition.

[0069] The data processing method provided herein can be applied to the implementation environment shown in FIG1 , which includes a field terminal 110 and a server 120. Field terminal 110 and server 120 can communicate via a communication network 130. Communication network 130 utilizes standard communication technologies and / or protocols, typically the Internet, but can also be any network, including but not limited to Bluetooth, a local area network (LAN), a metropolitan area network (MAN), a wide area network (WAN), a mobile network, a dedicated network, or any combination of virtual private networks. In some embodiments, customized or dedicated data communication technologies can be used in place of or in addition to the aforementioned data communication technologies.

[0070] The field terminals 110 involved in this application include, but are not limited to, mobile phones, tablet computers, laptop computers, desktop computers, intelligent voice interaction devices, smart home appliances, vehicle-mounted terminals, aircraft, etc. The client is deployed on the field terminal 110 and can be run on the field terminal 110 in the form of a browser or an independent application (APP).

[0071] The server 120 involved in this application can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers. It can also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence (AI) platforms.

[0072] In conjunction with the above implementation environment, in step A1, the on-site terminal 110 sends K images captured in the current on-site environment to the server 120 via the communication network 130. In step A2, the server 120 recognizes the K images and sends the first prediction result of each image to the on-site terminal 110 via the communication network 130. In step A3, the on-site terminal 110 constructs a fine-tuning training set based on the K images and the K first prediction results sent by the server. In step A4, the on-site terminal 110 obtains K second prediction results through the model to be trained. That is, based on each image included in each set of fine-tuning training data in the fine-tuning training set, the second prediction result corresponding to each image is obtained through the model to be trained. In step A5, the on-site terminal 110 trains the model to be trained based on the K second prediction results and the fine-tuning training set to obtain a local recognition model and model adjustment parameters. That is, based on the second prediction result corresponding to each image and the first prediction result of the image in the fine-tuning training set, the model to be trained is trained to obtain a local recognition model and model adjustment parameters. In step A6, the on-site terminal 110 sends the model adjustment parameters to the server 120 via the communication network 130. In step A7, the server 120 updates the model parameters of the image recognition model based on the model adjustment parameters (i.e., the model adjustment parameter set) reported by at least one terminal.

[0073] The following describes the implementation environment of the image recognition method, using the on-site terminal 110 as a palm-swipe terminal as an example. Please refer to Figure 2, which is a schematic diagram of an implementation environment of the image recognition method in an embodiment of the present application. As shown in the figure, specifically, in step B1, the on-site terminal 110 uses a local recognition model to identify the collected image to be identified, obtaining a seventh prediction result. The image to be identified may be a palm image. In step B2, the seventh prediction result includes a category score. If the category score included in the seventh prediction result is greater than or equal to the category score threshold, the image to be identified is determined to belong to the predicted category included in the seventh prediction result. In step B3, if the category score included in the seventh prediction result is less than the category score threshold, the on-site terminal 110 transmits the image to be identified to the server 120 via the communication network 130. In step B4, the server 120 recognizes the image to be identified using the image recognition model, obtaining an image recognition result. In step B5, the server 120 transmits the image recognition result to the on-site terminal 110 via the communication network 130, thereby enabling the on-site terminal 110 to execute corresponding services based on the image recognition result.

[0074] In combination with the above introduction, the data processing method in this application will be introduced from the perspective of the on-site terminal. Please refer to Figure 3. The data processing method in the embodiment of this application can be completed independently by the on-site terminal or by the on-site terminal and the server. The method of this application includes:

[0075] 210. Under the current scene environment, obtain K images by capturing them with an image acquisition device, where K is an integer greater than or equal to 1;

[0076] In one or more embodiments, the on-site terminal calls an image acquisition device (eg, a camera, a scanner, etc.) to capture several images of the current environment, thereby obtaining K images.

[0077] 220. Send the K images to the server, so that the server obtains K first prediction results based on the K images using the image recognition model;

[0078] In one or more embodiments, the on-site terminal may send K images sequentially to the server, or directly package the K images and send them together to the server. Based on this, the server inputs each of the K images into an image recognition model, which then outputs a first prediction result for each image, thereby obtaining K first prediction results. Each first prediction result includes a predicted category and a category score for the image.

[0079] It is understood that the models involved in this application are deep learning models, for example, convolutional neural networks (CNNs) can be used. Deep learning is a machine learning technique whose goal is to simulate the workings of neurons in the human brain so that computers can learn and make decisions autonomously. Deep learning models are typically composed of multiple layers, each of which can learn different levels of representation of data.

[0080] In this application, the image recognition model deployed on the server side is a "big model", that is, compared with the model deployed on the terminal, it has stronger computing power and higher recognition accuracy. It is trained on a large amount of data, learns a wider range of image features, and can accurately identify a variety of objects. However, due to its large amount of computation, it is usually deployed on the server side and is not suitable for running on the terminal. In actual applications, the local model of the terminal will be compared and fed back with the "big model" on the server side to achieve the purpose of self-adjustment and optimization.

[0081] 230. Construct a fine-tuning training set based on the K images and the K first prediction results sent by the server, wherein the fine-tuning training set includes K groups of fine-tuning training data, and each group of fine-tuning training data includes an image and a first prediction result of the image;

[0082] In one or more embodiments, the on-site terminal can construct a fine-tuning training set based on the K collected images and the first prediction result of each image. The following uses 5 images as an example to describe the process of constructing the fine-tuning training set.

[0083] Assume that the K images sent by the on-site terminal to the server are Image 1, Image 2, Image 3, Image 4, and Image 5. After invoking the image recognition model, the server recognizes these images in sequence. See Table 1, which illustrates the first prediction results obtained for each image after recognition. Assume that the predicted category in the first prediction result is an object identifier, and each object identifier uniquely identifies an object (e.g., User A).

[0084] Table 1

[0085] Based on this, K sets of fine-tuning training data can be constructed, each set of fine-tuning training data including an image and a first prediction result of the image. For example, a set of fine-tuning training data includes image 1, labeled category 10003, and labeled category score 0.95.

[0086] 240. Fine-tune the to-be-trained model on the on-site terminal using the fine-tuning training set. During fine-tuning of the to-be-trained model on the on-site terminal, obtain a second prediction result corresponding to each image using the to-be-trained model based on the images included in each set of fine-tuning training data in the fine-tuning training set.

[0087] In one or more embodiments, a fine-tuning training set is used to fine-tune a model to be trained on a field terminal. During fine-tuning of the model to be trained on the field terminal, the field terminal sequentially inputs K captured images into the model to be trained, and the model to be trained outputs a second prediction result for each image. Each second prediction result includes a predicted category and a category score for the image.

[0088] 250. Update the model parameters of the to-be-trained model according to the second prediction result corresponding to each image and the first prediction result of the image in the fine-tuning training set to obtain a local recognition model and model adjustment parameters corresponding to the local recognition model;

[0089] In one or more embodiments, the fine-tuning training set includes K images and a first prediction result for each image, wherein the first prediction result serves as the image's annotation information. The second prediction result for each of the K images serves as the image's prediction information. Based on this, using the annotation information and prediction information of each image, a corresponding loss function (e.g., a multi-classification loss function) can be used to update the model parameters of the to-be-trained model to obtain a local recognition model and model adjustment parameters corresponding to the local recognition model. The model adjustment parameters include, but are not limited to, model parameters, gradients, optimization algorithm parameters, and a fine-tuning training set.

[0090] In the embodiments of the present application, updating the model parameters of the training model can be understood as fine-tuning the training model. In machine learning, fine-tuning is a transfer learning technique that is usually performed on a pre-trained model (e.g., a model trained on a large dataset). Based on a new, usually smaller dataset, fine-tuning adjusts the model parameters to optimize performance for a specific task.

[0091] 260. If the local recognition model meets the model fine-tuning conditions, the model adjustment parameters are sent to the server so that the server updates the model parameters of the image recognition model according to the model adjustment parameter set from at least one terminal, wherein the model adjustment parameter set includes the model adjustment parameters.

[0092] In one or more embodiments, if the local recognition model meets the model fine-tuning conditions, it indicates that the current fine-tuning method of the on-site terminal is worthy of adoption. Based on this, the on-site terminal can send model adjustment parameters to the server. The server combines the model adjustment parameters uploaded by different terminals to obtain a model adjustment parameter set. Based on this, the server uses the model adjustment parameter set to update the model parameters of the image recognition model, that is, to fine-tune the image recognition model.

[0093] In an embodiment of the present application, the server is based on distributed training to obtain an image recognition model with better recognition effect. Among them, distributed training refers to splitting the workload of the training model and sharing it with multiple microprocessors (for example, multiple terminals). The image recognition model has a large number of parameters and a large amount of training data, which exceeds the capacity of a single machine. Therefore, distributed parallel speed-up is required. Parallel mechanisms include data parallel (DP), model parallel (MP), pipeline parallel (PP), and hybrid parallel (HP). Structural design includes structures based on parameter server, reduce, and message-passing interface (MPI).

[0094] In an embodiment of the present application, a data processing method is provided. Using the above method, a terminal can fine-tune a local model based on images captured from a live environment and report the fine-tuned model adjustment parameters to a server. The server integrates the model adjustment parameter sets reported by each terminal and updates the image recognition model. This not only enables the image recognition model to be adapted to various specific live environments, thereby improving the accuracy of model recognition, but also conserves server processing resources and improves the efficiency of model learning.

[0095] Based on one or more embodiments corresponding to FIG. 3 above, another optional embodiment provided by the embodiment of the present application may further include:

[0096] Sending a model training request to the server, so that the server determines a training data set to be applied to the field terminal according to the model training request;

[0097] Receive an initial training set sent by the server, where the initial training set includes M groups of initial training data, each group of initial training data includes an image and an image annotation result;

[0098] Based on the M images included in the initial training set, M initial prediction results are obtained through the initial recognition model, where each initial prediction result includes a predicted category and a category score of the image;

[0099] According to the M initial prediction results and the M annotation results included in the initial training set, the model parameters of the initial recognition model are updated to obtain the model to be trained.

[0100] In one or more embodiments, a method for obtaining a model to be trained is described. As can be seen from the aforementioned embodiments, after a field terminal is deployed in a site, the field terminal can send a model training request to a server, which then transmits a training dataset back to the field terminal. It should be noted that other terminals can also generate models to be trained in a similar manner, which is not detailed here.

[0101] Assuming the on-site terminal is a palm-scanning terminal at an amusement park, the server can obtain a training dataset from a large number of datasets based on the model training request sent by the on-site terminal. The training dataset includes palm images of users who have registered for fast passes at the amusement park, as well as the image annotation results (e.g., user identification), and is used as the initial training set. The on-site terminal receives the initial training set sent by the server, where the initial training set includes M groups of initial training data, each group of initial training data including an image and the image annotation results, that is, the initial training set includes M images.

[0102] Based on this, the on-site terminal sequentially inputs M images into the initial recognition model, which then outputs an initial prediction result corresponding to each image. Each initial prediction result includes the image's predicted category and category score. Using the annotation results and initial prediction results for each image, a corresponding loss function (e.g., a multi-classification loss function) is used to update the model parameters of the initial recognition model to obtain the model to be trained.

[0103] It's important to note that the initial recognition model is a pre-training model (PTM), which refers to a specific, large-scale deep neural network (DNN). It's trained on massive amounts of unlabeled data. Leveraging the function approximation capabilities of large-scale DNNs, the PTM extracts common features from the data. Through fine-tuning, parameter-efficient fine-tuning (PERT), and prompt-tuning techniques, it becomes suitable for downstream tasks. Therefore, the PTM can achieve ideal results in few-shot or zero-shot scenarios.

[0104] Secondly, in an embodiment of the present application, a method for obtaining a model to be trained is provided. Through the above method, the terminal uses the training data set issued by the server to train a model to be trained that can be used locally. On the one hand, the terminal is given image recognition capabilities. On the other hand, when the terminal fine-tunes the model, it can better adapt to the local environment.

[0105] Based on one or more embodiments corresponding to FIG. 3 above, another optional embodiment provided by the embodiment of the present application may further include:

[0106] Acquiring on-site environment information of the on-site terminal, wherein the on-site environment information includes at least one of light intensity and background noise;

[0107] If the light intensity included in the scene environment information is not within the light intensity range, in response to a first adjustment operation on the image acquisition device, adjusting a first application parameter of the image acquisition device, wherein the first application parameter includes at least one of a shutter speed, a sensitivity parameter, and an exposure compensation parameter;

[0108] If the background noise included in the scene environment information is greater than or equal to the background noise threshold, the second application parameter of the image acquisition device is adjusted in response to the second adjustment operation on the image acquisition device, wherein the second application parameter includes at least one of a sharpness parameter, a sensitivity parameter and a noise reduction parameter.

[0109] In one or more embodiments, a method for adjusting an image acquisition device is described. As can be seen from the aforementioned embodiments, to capture higher-quality images for model training and inference, the image acquisition device can also be adjusted based on on-site environmental information. The following description uses an on-site terminal as an example. In actual applications, other terminals can also perform similar image acquisition device adjustments, which will not be detailed here.

[0110] In one possible implementation, a camera or light sensor of a field terminal may be used to obtain field environment information of the field terminal. For example, the field light intensity and background noise may be obtained through a camera, or the field light intensity and color temperature may be obtained through a light sensor.

[0111] Exemplarily, if the light intensity is not within the light intensity range (i.e., the light intensity is greater than or equal to the maximum light intensity, or the light intensity is less than or equal to the minimum light intensity), the first application parameter of the image acquisition device needs to be adjusted. The first application parameter includes but is not limited to shutter speed, sensitivity parameter, and exposure compensation parameter. In one case, if the light intensity is greater than or equal to the maximum light intensity, the speed can be increased, the sensitivity (ISO) parameter can be lowered, and the exposure compensation (EV) parameter can be reduced. In another case, if the light intensity is less than or equal to the minimum light intensity, the speed can be slowed down, the ISO parameter can be increased, and the EV parameter can be increased.

[0112] It is understood that the maximum light intensity can be set to 1000 lux, and the minimum light intensity can be set to 10 lux. This is for illustration only and should not be construed as limiting the present application.

[0113] For example, if the background noise is greater than or equal to the background noise threshold, the second application parameter of the image acquisition device needs to be adjusted. The second application parameter includes, but is not limited to, a sharpness parameter, an ISO parameter, and a noise reduction parameter. Based on this, when the background noise is large, the sharpness parameter can be reduced, the ISO parameter can be lowered, and the noise reduction parameter (for example, a spatial noise reduction parameter or a temporal noise reduction parameter) can be increased.

[0114] It is understandable that the background noise threshold may be set to 50 decibels, which is merely for illustration and should not be construed as limiting the present application.

[0115] It should be noted that the term "in response to" in this application is used to indicate the conditions or states upon which the execution of an operation depends. When certain conditions or states are met, one or more operations may be executed. These operations may be in real time or with a certain delay.

[0116] Secondly, embodiments of the present application provide a method for adjusting the image acquisition device. By adjusting the image acquisition device based on on-site environmental information, the quality of the captured images can be improved. Furthermore, the local model is automatically fine-tuned, allowing the terminal's local model to better adapt to on-site lighting conditions and improve the model's recognition capabilities.

[0117] Based on one or more embodiments corresponding to FIG. 3 above, another optional embodiment provided by the embodiments of the present application may include updating the model parameters of the to-be-trained model based on the second prediction result corresponding to each image and the first prediction results of the images in the fine-tuning training set, and obtaining the local recognition model and the model adjustment parameters corresponding to the local recognition model. The following steps may also be included:

[0118] Obtaining recognition accuracy of the local recognition model for N images, where N is an integer greater than or equal to 1; the N images are captured by an image acquisition device;

[0119] If the recognition accuracy is greater than or equal to the accuracy threshold, it is determined that the local recognition model meets the model fine-tuning conditions.

[0120] In one or more embodiments, a method for determining whether the model fine-tuning conditions are met is introduced. As can be seen from the aforementioned embodiments, after completing the model fine-tuning, the on-site terminal needs to evaluate the effect of the fine-tuning. That is, by comparing the recognition results of the local recognition model and the image recognition model on the server side. If the recognition result of the local recognition model is consistent with or close to the result of the image recognition model, then the fine-tuning can be considered successful. Otherwise, it may be necessary to fine-tune again. The following will be explained using the on-site terminal as an example. In actual applications, other terminals can also judge the quality of the fine-tuning results in a similar manner, which will not be described here.

[0121] The on-site terminal automatically fine-tunes the local trained model using the collected on-site information to generate a local recognition model. Fine-tuning refers to making subtle adjustments to the model parameters based on the trained model, tailored to the new task or dataset. This fine-tuning is performed based on the on-site environmental information. For example, if the lighting intensity changes, the model's sensitivity to these changes may need to be adjusted. Another example is if background noise increases, the model's noise immunity may need to be enhanced. The fine-tuning process typically involves gradient descent or other optimization algorithms to minimize recognition errors in the new environmental conditions.

[0122] In one possible implementation, the method for obtaining the recognition accuracy of the local recognition model for N images can be to send N images captured by an image acquisition device to a server, so that the server obtains N third prediction results based on the N images through the image recognition model; receive the N third prediction results sent by the server; obtain N fourth prediction results based on the N images through the local recognition model; verify the N fourth prediction results based on the N third prediction results to obtain the recognition accuracy for the N images.

[0123] During the actual execution process, the on-site terminal can send N images captured by the image acquisition device to the server, wherein these N images can be collected by the on-site terminal after training the local recognition model, or can be some images randomly selected from K images. The N images meet the on-site lighting conditions of the on-site terminal. Based on this, on the one hand, the server uses the N images as inputs to the image recognition model respectively, and obtains the third prediction result of each image through the image recognition model. Each third prediction result includes the predicted category and category score of the image. On the other hand, the on-site terminal uses the N images as inputs to the local recognition model respectively, and obtains the fourth prediction result of each image through the local recognition model. Each fourth prediction result includes the predicted category and category score of the image.

[0124] After receiving the N third prediction results, the on-site terminal can use the N third prediction results as standard results to verify the N fourth prediction results and obtain the recognition accuracy for the N images. For ease of introduction, please refer to Table 2, which is an example of the N third prediction results.

[0125] Table 2

[0126] Please refer to Table 3, which is a schematic diagram of N fourth prediction results.

[0127] Table 3

[0128] Based on this, the N third prediction results and the N fourth prediction results are compared. If the third prediction result and the fourth prediction result of the image are consistent or close, it means that the recognition of the image is successful. Assume that the third prediction result and the fourth prediction result are close, specifically "the prediction categories are consistent, and the absolute value of the difference in category scores is less than or equal to 0.2". It can be seen that the third prediction result and the fourth prediction result of image 1 are close. The third prediction result and the fourth prediction result of image 2 are different. The third prediction result and the fourth prediction result of image 3 are consistent. The third prediction result and the fourth prediction result of image 4 are different. The third prediction result and the fourth prediction result of image 5 are close. Therefore, 3 out of 5 images are successfully recognized, that is, the recognition accuracy rate is 0.6. When the recognition accuracy rate is greater than or equal to the accuracy rate threshold, it is considered that the local recognition model of the on-site terminal meets the model fine-tuning conditions, that is, this fine-tuning is successful.

[0129] The above method uses the N third prediction results output by the server through the image recognition model as the judgment standard to verify the N fourth prediction results. Since the image recognition model deployed on the server is relatively accurate, the third prediction results it outputs are also relatively accurate and can be used as the judgment standard, so that the recognition accuracy of the local recognition model can be verified more accurately.

[0130] Secondly, the present embodiment provides a method for determining whether the model fine-tuning conditions are met. Through this method, the on-site automatic fine-tuning module implements a closed loop from acquiring on-site environmental information, constructing a fine-tuning training set, performing model fine-tuning, and evaluating the fine-tuning results, thereby automatically optimizing the model to better adapt to on-site lighting conditions.

[0131] Based on one or more embodiments corresponding to FIG. 3 above, another optional embodiment provided by the embodiments of the present application may include updating the model parameters of the to-be-trained model based on the second prediction result corresponding to each image and the first prediction results of the images in the fine-tuning training set, and obtaining the local recognition model and the model adjustment parameters corresponding to the local recognition model. The following steps may also be included:

[0132] Obtaining recognition accuracy of the local recognition model for N images, where N is an integer greater than or equal to 1; the N images are captured by an image acquisition device;

[0133] If the recognition accuracy is greater than or equal to the accuracy threshold, the model adjustment parameters are sent to T terminals, so that the T terminals update the model parameters of their corresponding models to be trained according to the model adjustment parameters, and obtain T recognition models, wherein the T terminals are associated with the on-site terminals, and T is an integer greater than or equal to 1; the voting score corresponding to each of the T terminals is obtained, wherein the voting score is determined based on the prediction results of the recognition model and the prediction results of the image recognition model; the comprehensive recognition score is determined based on the voting score corresponding to each terminal; if the comprehensive recognition score is greater than or equal to the recognition score threshold, it is determined that the local recognition model meets the model fine-tuning conditions.

[0134] In one or more embodiments, another method for determining whether model fine-tuning conditions are met is described. As can be seen from the aforementioned embodiments, after completing model fine-tuning, the on-site terminal needs to evaluate the effectiveness of the fine-tuning using recognition accuracy. The calculation method for recognition accuracy can be found in the aforementioned embodiments and will not be further described here.

[0135] Specifically, when the recognition accuracy is greater than or equal to the accuracy threshold, the on-site terminal can send model adjustment parameters to each of the T associated terminals. This achieves the goal of synchronizing the model adjustment parameters (i.e., fine-tuning results), namely, synchronizing the model adjustment parameters of the on-site terminal to other terminals in the same environment. Each terminal fine-tunes the trained model based on the model adjustment parameters, respectively, to obtain a corresponding recognition model. In addition, each terminal needs to determine whether the model adjustment parameters are appropriate based on a synchronized voting mechanism. If the model adjustment parameters are appropriate, the terminal can vote in favor; conversely, if the model adjustment parameters are inappropriate, the terminal can vote against. For example, the voting score corresponding to a vote in favor can be set to 1, and the voting score corresponding to a vote against can be set to 0. Based on this, a comprehensive recognition score can be calculated using the voting scores of the T terminals. The comprehensive recognition score is used to determine whether the local recognition model meets the model fine-tuning conditions.

[0136] In the embodiment of the present application, the method of obtaining the recognition accuracy of the local recognition model for N images is similar to the method described above and will not be repeated here.

[0137] In this application, a synchronous voting mechanism is used to synchronize model fine-tuning results across multiple terminals. This mechanism allows terminals to vote on the effectiveness of their model adjustment parameters, deciding whether to accept the fine-tuning methods proposed by other terminals. By aggregating the voting scores, a decision is made on whether to feed the fine-tuning method back to the server for integration into image recognition model training.

[0138] Secondly, in an embodiment of the present application, another method for determining whether the model fine-tuning conditions are met is provided. Through the above method, the terminal synchronizes the model adjustment parameters obtained after model optimization to other terminals on site, further improving the accuracy of image recognition. At the same time, by implementing a synchronous voting mechanism for fine-tuning results between terminals, the terminal can self-learn and optimize, thereby improving the intelligence of the system. Based on the synchronization of fine-tuning results and the voting mechanism between terminals, the terminal can perform real-time optimization according to the on-site environmental information, so that the terminal can better adapt to environmental changes and improve the effect of image recognition.

[0139] Based on one or more embodiments corresponding to FIG. 3 above, in another optional embodiment provided by the present application, determining a comprehensive identification score based on the voting score corresponding to each terminal may specifically include:

[0140] Sum the voting scores of T terminals to get the total voting score;

[0141] The comprehensive recognition score is obtained based on the ratio between the total voting score and the T value.

[0142] In one or more embodiments, a method for determining a comprehensive recognition score based on a voting score is introduced. As can be seen from the aforementioned embodiments, the on-site terminal transmits the model adjustment parameters (i.e., the fine-tuning results) to T terminals based on network communication technology. After receiving the model adjustment parameters, each of the T terminals needs to fine-tune the local model to be trained according to the model adjustment parameters and verify the effect of the fine-tuning. The verification method includes comparing the recognition accuracy of the model before and after fine-tuning, or comparing the difference in recognition results between the fine-tuned recognition model and the image recognition model on the server side.

[0143] Based on this, the terminals vote based on the fine-tuning effect, and this process can adopt a majority voting method. Thus, based on the voting results of T terminals, it can be judged whether the current model adjustment parameters are worthy of adoption. For ease of understanding, please refer to Figure 4, which is a schematic diagram of an interactive process of voting based on the synchronous voting mechanism in an embodiment of the present application. As shown in the figure, specifically:

[0144] In step C1, the on-site terminal packages the model adjustment parameters into an information package and sends it to terminal B, where the model adjustment parameters include model parameters, gradients, and optimization algorithm parameters, where the optimization algorithm parameters include but are not limited to optimization algorithms, learning rates, number of iterations, etc.

[0145] In step C2, the on-site terminal packages the model adjustment parameters into an information packet and sends it to terminal C.

[0146] It should be noted that the execution order of step C1 and step C2 is not limited.

[0147] In step C3, terminal B adjusts parameters according to the received model, fine-tunes the local model to be trained, and obtains a recognition model.

[0148] In step C4, terminal C adjusts parameters according to the received model, fine-tunes the local model to be trained, and obtains a recognition model.

[0149] It should be noted that the execution order of step C3 and step C4 is not limited.

[0150] In step C5, Terminal B performs a performance test based on the fine-tuned recognition model. If the model's performance improves after fine-tuning, Terminal B votes in favor. If the model's performance deteriorates after fine-tuning, Terminal B votes against it. For example, Terminal B compares the prediction output by the recognition model with the prediction output by the server based on the image recognition model to obtain the recognition accuracy rate, and Terminal B votes based on the recognition accuracy rate.

[0151] In step C6, terminal C performs a performance test based on the fine-tuned recognition model. If the model's performance improves after fine-tuning, the user votes in favor. If the model's performance deteriorates after fine-tuning, the user votes against it. For example, terminal C compares the predictions output by the recognition model with the predictions output by the server based on the image recognition model to determine the recognition accuracy. Terminal C then votes based on the recognition accuracy.

[0152] It should be noted that the execution order of step C5 and step C6 is not limited.

[0153] In step C7, if the recognition accuracy obtained by terminal B is greater than or equal to the accuracy threshold, terminal B's voting fine-tuning is successful, and thus, a voting score of 1 is obtained. Conversely, if the recognition accuracy obtained by terminal B is less than the accuracy threshold, terminal B's voting fine-tuning fails, and thus, a voting score of 0 is obtained.

[0154] In step C8, if the recognition accuracy obtained by terminal C is greater than or equal to the accuracy threshold, the voting fine-tuning of terminal C is successful, and thus, the voting score is 1. Conversely, if the recognition accuracy obtained by terminal C is less than the accuracy threshold, the voting fine-tuning of terminal C fails, and thus, the voting score is 0.

[0155] It should be noted that the execution order of step C7 and step C8 is not limited.

[0156] In step C9, for example, one approach is for terminal B to receive the voting score sent by terminal C and combine it with its own voting score to calculate a comprehensive identification score. Alternatively, terminals B and C each send their voting scores to an on-site terminal, which aggregates and calculates the comprehensive identification score. The on-site terminal then sends this comprehensive identification score to terminal B.

[0157] In step C10, for example, one approach is for terminal C to receive the voting score sent by terminal B and combine it with its own voting score to calculate a comprehensive identification score. Alternatively, terminals B and C each send their voting scores to an on-site terminal, which aggregates and calculates the comprehensive identification score. The on-site terminal then sends this comprehensive identification score to terminal C.

[0158] It should be noted that the execution order of step C9 and step C10 is not limited.

[0159] In step C11, if the comprehensive identification score is greater than or equal to the score threshold (e.g., 0.5), it means that most terminals voted in favor, and therefore, terminal B can accept the fine-tuning method. If the comprehensive identification score is less than the score threshold, it means that most terminals voted against, and therefore, terminal B does not adopt this fine-tuning method.

[0160] In step C12, if the comprehensive identification score is greater than or equal to the score threshold (e.g., 0.5), it means that most terminals voted in favor, and therefore, terminal C can accept the fine-tuning method. If the comprehensive identification score is less than the score threshold, it means that most terminals voted against, and therefore, terminal C does not adopt this fine-tuning method.

[0161] The majority voting method will be described below with a specific example. Assuming that Terminal B's voting score is 1 and Terminal C's voting score is 0, the total voting score is 1 (i.e., 1 + 0 = 1). The comprehensive identification score is calculated based on the ratio of the total voting score to the value of T. In this example, T is 2, so the comprehensive identification score is 0.5.

[0162] Again, in the embodiment of the present application, a method for determining a comprehensive identification score based on the voting score is provided. Through the above method, each terminal with an associated relationship votes on the fine-tuning result (i.e., the model adjustment parameter) and averages the voting score to serve as a basis for measuring whether to accept the fine-tuning method. This provides a specific and feasible method for implementing the solution. As a result, the feasibility and operability of the solution are improved.

[0163] Based on one or more embodiments corresponding to FIG. 3 above, in another optional embodiment provided by the present application, determining a comprehensive identification score based on the voting score corresponding to each terminal may specifically include:

[0164] Obtaining a weight parameter set corresponding to each of the T terminals, wherein the weight parameter set includes at least one of a device weight, an environment weight, and a preference weight;

[0165] For each of the T terminals, the terminal's voting score is weighted using the terminal's weight parameter set to obtain the terminal's voting weighted score;

[0166] The comprehensive identification score is determined based on the voting weighted score corresponding to each terminal in the T terminals.

[0167] In one or more embodiments, another method for determining a comprehensive recognition score based on voting scores is introduced. As can be seen from the aforementioned embodiments, the on-site terminal transmits the model adjustment parameters (i.e., fine-tuning results) to T terminals based on network communication technology. After receiving the model adjustment parameters, each of the T terminals needs to fine-tune the local model to be trained based on the model adjustment parameters and verify the effect of the fine-tuning.

[0168] Based on this, the terminal votes according to the fine-tuning effect, and this process can adopt the weighted voting method. Thus, based on the voting results of T terminals, it can be judged whether the current model adjustment parameters are worthy of adoption. Among them, the weighted voting method needs to take into account the weight parameter set corresponding to each terminal, and the weight parameter set includes at least one of the device weight, environment weight, and preference weight. For ease of understanding, please refer to Figure 5, which is another interactive process diagram for voting based on the synchronous voting mechanism in an embodiment of the present application. As shown in the figure, specifically:

[0169] Steps D1 to D8 in this embodiment are similar to steps C1 to C8 in the embodiment shown in FIG. 4 , and their details are not repeated here.

[0170] In step D9, for example, one method is for terminal B to calculate a weighted voting score based on its own voting score and weight parameter set. Another method is for terminal B to send its own voting score and weight parameter set to the on-site terminal, and the on-site terminal calculates terminal B's weighted voting score based on the voting score and weight parameter set sent by terminal B.

[0171] In step D10, for example, one approach is for terminal C to calculate a weighted voting score based on its own voting score and weight parameter set. Another approach is for terminal C to send its own voting score and weight parameter set to an on-site terminal, which then calculates terminal B's weighted voting score based on the voting score and weight parameter set sent by terminal C.

[0172] It should be noted that the execution order of step D9 and step D10 is not limited.

[0173] In step D11, for example, one approach is for terminal B to receive the weighted voting score sent by terminal C and, by combining its own weight parameter set with the weight parameter set of terminal C, calculate a comprehensive identification score. Alternatively, terminal B and terminal C each send their weighted voting scores to an on-site terminal, which aggregates and calculates the combined score to obtain a comprehensive identification score. The on-site terminal then sends this comprehensive identification score to terminal B.

[0174] In step D12, for example, one approach is for terminal C to receive the weighted voting score sent by terminal B and, by combining its own weight parameter set with that of terminal B, calculate a comprehensive identification score. Alternatively, terminal B and terminal C each send their weighted voting scores to an on-site terminal, which aggregates and calculates the combined score to obtain a comprehensive identification score. The on-site terminal then sends this comprehensive identification score to terminal C.

[0175] It should be noted that the execution order of step D11 and step D12 is not limited.

[0176] Steps D13 to D14 in this embodiment are similar to steps C11 to C12 in the embodiment shown in FIG. 4 , and their details are not repeated here.

[0177] The following describes the weighted voting method with specific examples. Assume that the weight parameter set includes device weight, environment weight, and preference weight. The device weight can be determined based on factors such as the terminal's performance and historical prediction accuracy. For example, the better the terminal's performance, the higher the device weight. The environment weight can be determined based on the terminal's on-site environment. For example, if the light intensity in the terminal's on-site environment is too strong or too weak, the environment weight will be higher, while if the light intensity is within the light intensity range, the environment weight will be lower. The preference weight can be determined based on the terminal's preference for different optimization algorithm parameters. For example, if the optimization algorithm parameters used by the terminal are consistent with the optimization algorithm parameters of the on-site terminal, the preference weight will be higher.

[0178] It is understandable that the adjustment method of device weight, environment weight and preference weight can be flexibly adjusted according to actual conditions. This is only an illustration and should not be understood as a limitation of this application.

[0179] Based on this, the comprehensive recognition scores for terminal B and terminal C can be calculated as follows: V weighted_average =(w B *V B *I B *P B +w C *V C *I C *P C ) / (w B *I B *P B +w C *I C *P C );

[0180] Among them, V weighted_average Indicates the comprehensive recognition score. B Indicates the device weight of terminal B. w CRepresents the device weight of terminal C. V B Indicates the voting score of terminal B, for example, 1 or 0. V C Indicates the voting score of terminal C. B Represents the environmental weight of terminal B. C Represents the environmental weight of terminal C. P B Represents the preference weight of terminal B. C represents the preference weight of terminal C. B *V B *I B *P B Indicates the voting weighted score of terminal B. C *V C *I C *P C Indicates the voting weighted score of terminal C.

[0181] Again, in an embodiment of the present application, another method for determining a comprehensive recognition score based on a voting score is provided. Through the above method, each terminal with an associated relationship votes on the fine-tuning result (i.e., the model adjustment parameter) respectively, and the weight parameter of each terminal is introduced to calculate the voting score, which is used as a basis for measuring whether to accept the fine-tuning method. It not only provides a specific and feasible way for the implementation of the solution, but also takes into account the differences in performance of different terminals, so that the fine-tuning results can be voted on more comprehensively.

[0182] Based on one or more embodiments corresponding to FIG. 3 above, in another optional embodiment provided by the embodiment of the present application, before sending the model adjustment parameters to T terminals, the following steps may also be included:

[0183] If the on-site terminal and at least one terminal are in the same location area, determining that the at least one terminal has an association relationship with the on-site terminal, and determining the at least one terminal as T terminals;

[0184] or,

[0185] If the on-site terminal and at least one terminal are set with the same binding object, determining that the at least one terminal has an association relationship with the on-site terminal, and determining the at least one terminal as T terminals;

[0186] or,

[0187] If the on-site terminal and the at least one terminal are connected to the same access point, it is determined that the at least one terminal has an association relationship with the on-site terminal, and the at least one terminal is determined as T terminals.

[0188] In one or more embodiments, three methods for establishing associations between terminals are described. As can be seen from the aforementioned embodiments, a field terminal is associated with T terminals, based on which the field terminal can communicate with each of the T terminals. The following examples illustrate the three methods for establishing associations between terminals.

[0189] Method 1: Based on geographic location;

[0190] Specifically, terminals within the same location area are considered to be associated with each other. It is understood that the same location area can specifically be an administrative area, such as a provincial-level administrative region, a prefecture-level administrative region, a county-level administrative region, or a township-level administrative region. It can also be a custom area, such as a residential community, a school, or an office building. It should be noted that the terminals have positioning capabilities, and users can establish associations between terminals located in the same geographical area.

[0191] For easier understanding, please refer to Figure 6, which is a schematic diagram illustrating establishing associations between terminals in an embodiment of the present application. As shown in the figure, a cell is considered a location area. In other words, terminals A, B, and C in cell A are associated with each other. Terminals D, E, and F in cell B are associated with each other. However, the terminals in cell A are not associated with the terminals in cell B. Therefore, assuming the on-site terminal is terminal A, the T terminals include terminal B and terminal C.

[0192] Method 2: Based on binding relationship;

[0193] Specifically, the user can customize the binding relationship between terminals. For example, each terminal can be set with a merchant ID, based on which terminals with the same merchant ID can be bound.

[0194] For easier understanding, please refer to Figure 7, which is another schematic diagram of establishing associations between terminals in an embodiment of the present application. As shown in the figure, taking Mall A as an example, assuming Mall A has branches in both City A and City B, then Terminals A, B, and C in Mall A in City A are associated with Terminals D, E, and F in Mall A in City B. Therefore, assuming the on-site terminal is Terminal A, the T terminals include Terminal B, Terminal C, Terminal D, Terminal E, and Terminal F.

[0195] Method 3: Based on network connection;

[0196] Specifically, an association relationship is established between terminals connected to the same access point, where the access point can be a wireless hotspot (eg, a mobile hotspot (wireless fidelity, WiFi), etc.) or a wired access point.

[0197] For easier understanding, please refer to Figure 8, which is another schematic diagram of establishing an association relationship between terminals in an embodiment of the present application. As shown in the figure, terminal A, terminal B, and terminal C are connected to the same access point, so these three terminals are associated with each other. Assuming that the field terminal is terminal A, then the T terminals include terminal B and terminal C.

[0198] Again, in the embodiments of the present application, three methods for establishing associations between terminals are provided. Through the above methods, associations between multiple terminals can be established according to actual needs, so that these terminals can perform corresponding processing (for example, voting, status monitoring, etc.) as a cluster. This increases the flexibility of the solution implementation.

[0199] Based on one or more embodiments corresponding to FIG. 3 above, in another optional embodiment provided by the embodiment of the present application, after determining that the local recognition model meets the model fine-tuning condition, the following steps may also be included:

[0200] Get the image to be tested;

[0201] Based on the image to be tested, obtaining a fifth prediction result through the local recognition model;

[0202] Obtain T sixth prediction results from T terminals, where each sixth prediction result is obtained by the terminal through the recognition model based on the image to be tested;

[0203] If it is determined based on the fifth prediction result and the T sixth prediction results that the local recognition model is in a stable model state, the corresponding business is executed through the local recognition model.

[0204] In one or more embodiments, a method for monitoring model status based on multiple associated terminals is described. As can be seen from the aforementioned embodiments, based on field terminals and T-terminals, the server can also monitor and evaluate the status of each terminal to determine whether these terminals have completed model fine-tuning and whether the fine-tuned model has stabilized.

[0205] For example, monitoring and evaluating the terminal's status mainly includes the fine-tuning status of the model and the terminal's operating status. That is, the terminal needs to be able to regularly send status reports to the server. The report content includes the current fine-tuning status (for example, fine-tuning in progress or completed), operating status (for example, normal operation or failure), etc. The server can understand the terminal's status based on these reports.

[0206] Evaluating whether the model has stabilized can usually be achieved by comparing the model's recognition results for the same image (i.e., the image to be tested). For example, the on-site terminal uses the image to be tested as the input of the local recognition model and obtains the fifth prediction result through the local recognition model. The other T terminals respectively use the image to be tested as the input of the recognition model and obtain the sixth prediction result through the recognition model. If the fifth prediction result and the T sixth prediction results are the same or similar, it is considered that the model is in a stable state. If the fifth prediction result and the T sixth prediction results are significantly different, it may be necessary to continue fine-tuning.

[0207] It is understood that in actual applications, the server can compare the fifth prediction result with the T sixth prediction results. Based on this, the server can determine whether the on-site terminal has completed fine-tuning based on the terminal's status, and determine whether the model is stable based on the fifth prediction result and the T sixth prediction results. If these two conditions are met, it is considered that the corresponding business has begun on-site. Otherwise, further model fine-tuning or terminal maintenance may be required.

[0208] It should be noted that if the fifth prediction result and the T sixth prediction results include the same prediction category, and the absolute value of the difference between the category scores is less than or equal to a certain threshold (for example, 0.2), the prediction results are considered to be similar.

[0209] For ease of understanding, please refer to FIG9 , which is a schematic diagram of an overall flow chart of the data processing method in an embodiment of the present application. As shown in the figure, specifically:

[0210] In step E1, the on-site terminal trains and deploys the model to be trained.

[0211] In step E2, the on-site terminal may obtain on-site environment information through a camera or a light sensor.

[0212] In step E3, the on-site terminal image acquisition device collects K images, and then uses the fine-tuning training set constructed by the K images to fine-tune the training model under the current on-site environment information to obtain a local recognition model.

[0213] In step E4, the prediction result of the local recognition model is compared with the prediction result of the image recognition model on the server side.

[0214] In step E5, if the prediction result of the local recognition model is the same as or similar to the prediction result of the image recognition model, the on-site terminal synchronizes the model adjustment parameters to T terminals in the same environment.

[0215] In step E6, T terminals in the same environment respectively use the model adjustment parameters to fine-tune the local model to be trained to obtain a recognition model.

[0216] In step E7, each of the T terminals votes based on the performance change of the fine-tuned model, and a comprehensive recognition score is calculated based on the votes. Based on the comprehensive recognition score, it is determined whether the T terminals accept the fine-tuning. If so, step E8 is executed. If not, step E9 is executed.

[0217] In step E8, if T terminals accept the fine-tuning, the on-site terminal reports the model adjustment parameters to the server.

[0218] In step E9, if the T terminals do not accept this fine-tuning, the existing model remains unchanged.

[0219] In step E10 , the server performs model training again based on the model adjustment parameters reported by the on-site terminal.

[0220] In step E11 , the server monitors and evaluates the status of each terminal and the performance of the fine-tuned model.

[0221] In step E12, the server determines whether the model is currently in a stable state. If so, step E13 is executed. If not, step E14 is executed.

[0222] In step E13 , if the model is in a stable state, the terminal and the server may perform corresponding daily services.

[0223] In step E14 , if the model is not in a stable state, the terminal and the server continue to fine-tune the model.

[0224] Furthermore, an embodiment of the present application provides a method for model status monitoring based on multiple associated terminals. Through this method, multiple associated terminals can also monitor and evaluate the status of various devices on-site to determine whether these devices have completed model fine-tuning and whether the fine-tuned model has stabilized. Based on this judgment, the normal operation of the system can be maintained. At the same time, it also facilitates the backend to understand the status of on-site terminals in real time, thereby helping the backend to promptly detect and handle abnormal issues.

[0225] Based on one or more embodiments corresponding to FIG. 3 above, another optional embodiment provided by the embodiment of the present application may further include: after determining the comprehensive identification score according to the voting score corresponding to each terminal, the following steps may be performed:

[0226] If the local recognition model does not meet the model fine-tuning condition, a model fine-tuning request is sent to a terminal among T terminals, so that the terminal updates the model parameters of the to-be-trained model according to the model fine-tuning request to obtain a recognition model, wherein the T terminals are associated with the on-site terminal, and T is an integer greater than or equal to 1;

[0227] receiving model adjustment parameters sent by the terminal;

[0228] The model parameters of the to-be-trained model at the on-site terminal are updated using the model adjustment parameters sent by the terminal.

[0229] In one or more embodiments, a training method for when the model fine-tuning conditions are not met is introduced. As can be seen from the aforementioned embodiments, when the local recognition model does not meet the model fine-tuning conditions, the on-site terminal can also randomly select a terminal from T terminals and send a model fine-tuning request to the terminal.

[0230] Assume that "Terminal A" receives a model fine-tuning request. Terminal A can then fine-tune the local model to be trained. Terminal A then sends the resulting model adjustment parameters to other terminals in the same location, including the on-site terminal. Based on this, the on-site terminal uses the model adjustment parameters sent by Terminal A to fine-tune the model to be trained again. Alternatively, the on-site terminal also uses the model adjustment parameters sent by Terminal A to continue fine-tuning the local recognition model.

[0231] Secondly, the present embodiment provides a training method for when the model fine-tuning conditions are not met. With this method, if the local recognition model does not meet the model fine-tuning conditions, another terminal in the same field can be selected for fine-tuning. Based on the model adjustment parameters obtained after fine-tuning on that terminal, fine-tuning can be continued on the other terminals in the same field. This allows for continuous model fine-tuning.

[0232] Based on one or more embodiments corresponding to FIG. 3 above, another optional embodiment provided by the embodiments of the present application may further include: updating the model parameters of the to-be-trained model based on the K second prediction results and the fine-tuning training set, and obtaining the local recognition model and the model adjustment parameters corresponding to the local recognition model;

[0233] When the local recognition model meets the model fine-tuning conditions, an image to be recognized is captured by an image acquisition device;

[0234] Obtaining a seventh prediction result based on the image to be recognized by using the local recognition model, wherein the seventh prediction result includes a predicted category and a category score;

[0235] If the category score in the seventh prediction result is greater than or equal to the category score threshold, it is determined that the image to be identified belongs to the predicted category in the seventh prediction result.

[0236] In one or more embodiments, a method for performing image recognition locally on a terminal is described. As can be seen from the aforementioned embodiments, if the local recognition model meets the model fine-tuning requirements, the on-site terminal can use this local recognition model for image recognition. The following description uses an on-site terminal as an example. In actual applications, other terminals can also perform image recognition in a similar manner, which is not detailed here.

[0237] Specifically, the on-site terminal captures several images using an image acquisition device and selects a high-quality image as the image to be identified. Taking palm print image recognition as an example, the captured images are subjected to image quality assessment, image enhancement, and precise positioning of the palm print area. For example, image processing techniques (e.g., edge detection, threshold segmentation, etc.) can be used to extract and locate the palm print area. At the same time, image clarity assessment methods (e.g., gradient methods, frequency domain analysis methods, etc.) are used to select images with high clarity and distinct palm print features for subsequent recognition processing.

[0238] After obtaining the image to be identified, it will be identified through the local recognition model to obtain the seventh prediction result, where the seventh prediction result includes the predicted category and the category score. The predicted category is the predicted category, and the category score represents the score predicted for the category. The higher the category score, the greater the probability that the image is predicted to be the predicted category. Based on this, if the category score is greater than or equal to the category score threshold (for example, 0.90), the on-site terminal can determine that the image to be identified belongs to the predicted category, thereby executing the corresponding business (for example, payment business, access control business, etc.).

[0239] For example, in one case, the local recognition model can output the category probability distribution corresponding to the image to be recognized. The category corresponding to the maximum probability value in the category probability distribution is used as the predicted category of the image to be recognized, and the maximum probability value is used as the category score. In another case, the local recognition model can extract the feature vector of the image to be recognized, and then match the extracted feature vector with the feature vector already in the database, for example, using the k-nearest neighbors (KNN) algorithm for feature matching. As a result, the category corresponding to the feature vector with the highest similarity is used as the predicted category of the image to be recognized, and the highest similarity is used as the category score. The similarity between feature vectors can be measured using cosine similarity.

[0240] Secondly, the embodiments of this application provide a method for performing image recognition locally on the terminal. Through this method, the terminal can use the local recognition model to recognize the collected image. This can reduce the data processing pressure on the server side and improve recognition efficiency, and it is not dependent on the network environment.

[0241] Based on one or more embodiments corresponding to FIG. 3 above, in another optional embodiment provided by the embodiment of the present application, after obtaining the seventh prediction result by using the local recognition model based on the image to be recognized, the following steps may also be included:

[0242] If the category score in the seventh prediction result is less than the category score threshold, sending the image to be recognized to the server, so that the server obtains an image recognition result based on the image to be recognized through the image recognition model;

[0243] Receive the image recognition results sent by the server.

[0244] In one or more embodiments, a server-side image recognition method is described. As can be seen from the aforementioned embodiments, after a terminal uses a local recognition model to recognize an image, if the resulting category score is less than a category score threshold (e.g., 0.90), the terminal may send the image to the server.

[0245] Specifically, after receiving the image to be identified, the server can call the image recognition model to identify the image to be identified, thereby obtaining an image recognition result. The server then sends the image recognition result to the on-site terminal, where the image recognition result includes the predicted category of the image to be identified and the category score. Based on this, the on-site terminal can execute the corresponding service (for example, payment service, access control service, etc.) according to the predicted category in the image recognition result.

[0246] For example, in one scenario, the image recognition model can output a probability distribution for the category of the image to be recognized. The category corresponding to the maximum probability value in the category probability distribution is used as the predicted category of the image to be recognized, and the maximum probability value is used as the category score of the predicted category. In another scenario, the image recognition model can extract feature vectors of the image to be recognized and then match the extracted feature vectors with feature vectors already in a database. The category corresponding to the feature vector with the highest similarity is used as the predicted category of the image to be recognized, and the highest similarity is used as the category score of the predicted category.

[0247] Furthermore, in an embodiment of the present application, a method for performing image recognition on the server side is provided. Through this method, if the terminal cannot locally predict the category of an image, the terminal can request the server to make a prediction for the image. Because the server-side model has better recognition capabilities, the success rate and accuracy of image recognition can be improved.

[0248] In combination with the above introduction, the data processing method in this application will be introduced from the perspective of the server. Please refer to Figure 10. The data processing method in the embodiment of this application can be completed by the server independently or by the server in cooperation with at least one terminal. The method of this application includes:

[0249] 310. Receive K images sent by the on-site terminal, where the K images are captured by the on-site terminal using a collection device under the current on-site environment, and K is an integer greater than or equal to 1;

[0250] In one or more embodiments, the on-site terminal calls the image acquisition device to capture a number of images of the current on-site environment to obtain K images. The server receives the K images sent by the on-site terminal,

[0251] 320. Based on the K images, obtain K first prediction results using the image recognition model;

[0252] In one or more embodiments, the server inputs each of the K images into an image recognition model, and outputs a first prediction result of each image through the image recognition model, thereby obtaining K first prediction results.

[0253] Step 320 in this embodiment is similar to step 220 in the embodiment shown in FIG. 3 , and will not be described in detail here.

[0254] 330. Send K first prediction results to the on-site terminal, so that the on-site terminal constructs a fine-tuning training set based on the K images and the K first prediction results, and uses the fine-tuning training set to fine-tune the model to be trained on the on-site terminal. During the fine-tuning of the model to be trained on the on-site terminal, based on the images included in each group of fine-tuning training data in the fine-tuning training set, obtain a second prediction result corresponding to each image using the model to be trained, and update model parameters of the model to be trained based on the second prediction result corresponding to each image and the first prediction result of the image in the fine-tuning training set, to obtain a local recognition model and model adjustment parameters corresponding to the local recognition model, wherein the fine-tuning training set includes K groups of fine-tuning training data, and each group of fine-tuning training data includes an image and the first prediction result of the image;

[0255] In one or more embodiments, the server sends K first prediction results to the on-site terminal, based on which the on-site terminal can use the K images and the K first prediction results to perform model training.

[0256] Step 330 in this embodiment is similar to steps 230 to 250 in the embodiment shown in FIG. 3 , and will not be described in detail here.

[0257] 340. If the local recognition model meets the model fine-tuning conditions, receive the model adjustment parameters sent by the on-site terminal;

[0258] In one or more embodiments, if the local recognition model satisfies the model fine-tuning condition, the on-site terminal sends the model adjustment parameters to the server.

[0259] 350. Update the model parameters of the image recognition model when a model adjustment parameter set is obtained from at least one terminal, wherein the model adjustment parameter set includes the model adjustment parameters.

[0260] In one or more embodiments, the server integrates the model adjustment parameters uploaded by different terminals to obtain a model adjustment parameter set. Based on this, the server uses the model adjustment parameter set to update the model parameters of the image recognition model, that is, to fine-tune the image recognition model.

[0261] Steps 340 to 350 in this embodiment are similar to step 260 in the embodiment shown in FIG. 3 , and are not described in detail here.

[0262] In an embodiment of the present application, a data processing method is provided. Using the above method, a terminal can fine-tune a local model based on images captured from a live environment and report the fine-tuned model adjustment parameters to a server. The server integrates the model adjustment parameter sets reported by each terminal and updates the image recognition model. This not only enables the image recognition model to be adapted to various specific live environments, thereby improving the accuracy of model recognition, but also conserves server processing resources and improves the efficiency of model learning.

[0263] In another optional embodiment provided in the present application, based on one or more embodiments corresponding to FIG. 10 , upon obtaining a model adjustment parameter set from at least one terminal, updating the model parameters of the image recognition model may specifically include:

[0264] Obtaining a set of model adjustment parameters from at least one terminal;

[0265] Performing weighted processing on the model adjustment parameter set according to the comprehensive recognition score corresponding to each terminal to obtain a weighted model adjustment parameter set;

[0266] The model parameters of the image recognition model are updated using the weighted model adjustment parameter set.

[0267] In one or more embodiments, a method for setting influence weights based on fine-tuning voting results is described. As can be seen from the aforementioned embodiments, the server receives model adjustment parameters sent by terminals from different sites, and then determines whether to accept the model adjustment parameters sent by each terminal based on the comprehensive recognition score corresponding to each terminal. It is understood that the method for calculating the comprehensive recognition score has been described in the aforementioned embodiments and will not be repeated here. If the server accepts the model adjustment parameters, the model adjustment parameters are integrated into the image recognition model for training.

[0268] In one possible implementation, the server opens an interface for receiving the model adjustment parameters and voting scores (or comprehensive identification scores) sent by the terminal. If the server receives the voting score, it further calculates the comprehensive identification score. When the comprehensive identification score is greater than or equal to the score threshold (for example, 0.5), it means that most terminals voted in favor, so the server also accepts the model adjustment parameters. When the comprehensive identification score is less than the score threshold, it means that most terminals voted against, so the server does not adopt the model adjustment parameters. Furthermore, if the server accepts the model adjustment parameters, the model adjustment parameters can also be weighted according to the comprehensive identification score.

[0269] For example, assume that "Terminal A" reports a set of model adjustment parameters, and, based on the voting scores of several terminals associated with "Terminal A", the calculated comprehensive recognition score is "0.8". "Terminal B" reports a set of model adjustment parameters, and, based on the voting scores of several terminals associated with "Terminal B", the calculated comprehensive recognition score is "0.3". "Terminal C" reports a set of model adjustment parameters, and, based on the voting scores of several terminals associated with "Terminal C", the calculated comprehensive recognition score is "0.9". Assuming the score threshold is 0.5, the model adjustment parameters reported by "Terminal B" will not be adopted. The model adjustment parameters reported by "Terminal A" have an influence of "0.8" on the fine-tuning of the image recognition model. "0.8" is used to weight the model adjustment parameters to obtain the weighted model adjustment parameters. The model adjustment parameters reported by Terminal C have an impact of 0.9 on fine-tuning the image recognition model. Using 0.9 to weight these model adjustment parameters yields the weighted model adjustment parameters. This indicates that the model adjustment parameters reported by Terminal C have a greater impact on fine-tuning the image recognition model.

[0270] Secondly, in an embodiment of the present application, a method for setting influence weights based on fine-tuning voting results is provided. With this method, since each terminal may have different perceptions of the on-site environment, the results of fine-tuning will also vary. Therefore, the server fine-tunes the image recognition model based on the comprehensive recognition scores and model adjustment parameters reported by terminals from different environments. This allows the server-side image recognition model to absorb different fine-tuning results, thereby improving and enhancing the model's recognition capabilities.

[0271] In another optional embodiment provided in the present application, based on one or more embodiments corresponding to FIG. 10 , upon obtaining a model adjustment parameter set from at least one terminal, updating the model parameters of the image recognition model may specifically include:

[0272] updating the model parameters of the image recognition model when a model parameter set is obtained from at least one terminal, wherein the model parameter set includes a model adjustment parameter, and the model adjustment parameter is a model parameter;

[0273] or,

[0274] updating model parameters of the image recognition model when a gradient set is obtained from at least one terminal, wherein the gradient set includes a model adjustment parameter, and the model adjustment parameter is a gradient;

[0275] or,

[0276] When an optimization algorithm parameter set is obtained from at least one terminal, the model parameters of the image recognition model are updated, wherein the optimization algorithm parameter set includes model adjustment parameters, and the model adjustment parameters are optimization algorithm parameters.

[0277] In one or more embodiments, three methods for training an image recognition model based on a set of model adjustment parameters are described. As can be seen from the aforementioned embodiments, model adjustment parameters include, but are not limited to, model parameters, gradients, and optimization algorithm parameters. Based on this, the server can fine-tune the image recognition model based on the model adjustment parameters.

[0278] 1. Based on model parameters;

[0279] The model adjustment parameter set can be a model parameter set. That is, each terminal from different environments can report model parameters to the server separately. For example, the server can adopt a model fusion strategy to fuse these model parameters and use the fused model parameters to fine-tune the image recognition model.

[0280] It should be noted that model fusion strategies include but are not limited to stacking strategies, boosting strategies, and bootstrap aggregating (Bagging).

[0281] 2. Gradient-based;

[0282] The model adjustment parameter set can be a gradient set. That is, each terminal from different environments can report gradients to the server. For example, the server can average these gradients to obtain a gradient mean. Based on this, the server can use the gradient mean to fine-tune the image recognition model.

[0283] 3. Based on optimization algorithm parameters;

[0284] The model adjustment parameter set may be an optimization algorithm parameter set, where the optimization algorithm parameters include, but are not limited to, the optimization algorithm, learning rate, and number of iterations. Terminals from different environments may report the optimization algorithm parameters to the server. For example, the server may aggregate these optimization algorithm parameters and use the most frequently occurring optimization algorithm parameters to fine-tune the image recognition model.

[0285] It is understood that the server can also select one of the fine-tuning methods reported by the terminal for training. For example, if the fine-tuning method is to modify model parameters, the updated model parameters can be directly used for training. For another example, if the fine-tuning method is to change the optimization algorithm parameters, the optimized algorithm parameters can be used when training the large model.

[0286] Secondly, the embodiments of this application provide a method for training an image recognition model based on a set of model adjustment parameters. Through this method, terminals from different environments can report model adjustment parameters to the server, which then fine-tunes the image recognition model based on the specific content of the model adjustment parameters. This improves the flexibility and diversity of model training methods.

[0287] In another optional embodiment provided in the embodiment of the present application, based on one or more embodiments corresponding to FIG. 10 , after obtaining a model adjustment parameter set from at least one terminal, updating the model parameters of the image recognition model may further include:

[0288] The model adjustment parameters of the image recognition model are sent to at least one terminal, so that each of the at least one terminal uses the model adjustment parameters of the image recognition model to update the model parameters of the recognition model.

[0289] In one or more embodiments, a method for updating other recognition models based on an image recognition model is described. As can be seen from the aforementioned embodiments, a server can send model adjustment parameters for an image recognition model to terminals in different environments, allowing these terminals to use the model adjustment parameters to fine-tune the recognition model. It is understood that the method for fine-tuning based on the model adjustment parameters can be found in the aforementioned embodiments and will not be further described here.

[0290] For ease of understanding, please refer to Figure 11, which is a schematic diagram of a framework for data processing between a terminal and a server in an embodiment of the present application. As shown in the figure, taking an on-site terminal as an example, the on-site terminal includes a local recognition module indicated by F1, an automatic fine-tuning module based on on-site environmental information indicated by F2, a multi-terminal status monitoring module indicated by F3, and a same-site multi-terminal model fine-tuning synchronization module indicated by F4. The server includes an on-site fine-tuning influence module indicated by F5 and an image recognition model indicated by F6.

[0291] During the model fine-tuning process, the local recognition module can send the on-site environment information to the automatic fine-tuning module based on the on-site environment information, and the automatic fine-tuning module based on the on-site environment information fine-tunes the local model to be trained according to the on-site environment information and the fine-tuning training set.

[0292] During the status monitoring process, the local recognition module may also send the terminal status of the field terminal to the multi-terminal status monitoring module, and the multi-terminal status monitoring module may feed back the model stability status to the local recognition module.

[0293] During the process of synchronizing fine-tuning results among multiple terminals, the automatic fine-tuning module based on on-site environmental information sends model adjustment parameters to the model fine-tuning synchronization module of multiple terminals on the same site, so that other terminals can use the model adjustment parameters to fine-tune the local model to be trained and feedback the corresponding fine-tuning results.

[0294] During the server-side fine-tuning process, the multi-terminal model fine-tuning synchronization module on-site feeds back the voting results of each terminal to the server-side fine-tuning influence module. Based on the voting results, the fine-tuning influence module generates a comprehensive recognition score. If the server determines to accept the model adjustment parameters reported by the on-site terminal based on the comprehensive recognition score, it can use the model adjustment parameters to fine-tune the image recognition model and then distribute the resulting model adjustment parameters to the on-site terminal.

[0295] Secondly, the embodiments of this application provide a method for updating other recognition models based on the image recognition model. Through this method, the server can also distribute the fine-tuned model adjustment parameters to each terminal, allowing the terminal to fine-tune the local model based on the model adjustment parameters. This achieves the effect of continuous model learning and optimization, and overall improves the accuracy and efficiency of image recognition.

[0296] The data processing device in the present application is described in detail below. Please refer to FIG12 , which is a schematic diagram of an embodiment of the data processing device in the present application. The data processing device 40 includes:

[0297] The shooting module 410 is configured to capture K images using an image acquisition device in the current scene environment, where K is an integer greater than or equal to 1;

[0298] A sending module 420 is configured to send K images to a server, so that the server obtains K first prediction results based on the K images using an image recognition model;

[0299] An acquisition module 430 is configured to construct a fine-tuning training set based on the K images and the K first prediction results sent by the server, wherein the fine-tuning training set includes K groups of fine-tuning training data, each group of fine-tuning training data including an image and a first prediction result of the image;

[0300] An updating module 440 is configured to fine-tune the model to be trained on the on-site terminal using the fine-tuning training set. During fine-tuning of the model to be trained on the on-site terminal, based on the images included in each set of fine-tuning training data in the fine-tuning training set, a second prediction result corresponding to each image is obtained by the model to be trained. Model parameters of the model to be trained are updated based on the second prediction result corresponding to each image and the first prediction result of the image in the fine-tuning training set, thereby obtaining a local recognition model and model adjustment parameters corresponding to the local recognition model.

[0301] The sending module 420 is also used to send model adjustment parameters to the server if the local recognition model meets the model fine-tuning conditions, so that the server updates the model parameters of the image recognition model according to the model adjustment parameter set from at least one terminal, wherein the model adjustment parameter set includes the model adjustment parameters.

[0302] On the basis of the embodiment corresponding to FIG. 12 , in another embodiment of the data processing device 40 provided in the embodiment of the present application, the data processing device 40 further includes a receiving module 450 ;

[0303] The sending module 420 is further configured to send a model training request to the server, so that the server determines a training data set to be applied to the field terminal according to the model training request;

[0304] A receiving module 450 is configured to receive an initial training set sent by a server, wherein the initial training set includes M groups of initial training data, each group of initial training data including an image and an image annotation result;

[0305] The acquisition module 430 is further configured to acquire M initial prediction results based on the M images included in the initial training set using the initial recognition model, wherein each initial prediction result includes a predicted category and a category score of the image;

[0306] The updating module 440 is further configured to update the model parameters of the initial recognition model according to the M initial prediction results and the M labeling results included in the initial training set to obtain a model to be trained.

[0307] On the basis of the embodiment corresponding to FIG. 12 , in another embodiment of the data processing device 40 provided in the embodiment of the present application, the data processing device 40 further includes a processing module 460 ;

[0308] The acquisition module 430 is further configured to acquire on-site environment information of the on-site terminal before capturing K images through the image acquisition device, wherein the on-site environment information includes at least one of light intensity and background noise;

[0309] a processing module 460 configured to adjust a first application parameter of the image acquisition device in response to a first adjustment operation on the image acquisition device if the light intensity included in the scene environment information is not within the light intensity range, wherein the first application parameter includes at least one of a shutter speed, a sensitivity parameter, and an exposure compensation parameter;

[0310] The processing module 460 is also used to adjust the second application parameter of the image acquisition device in response to a second adjustment operation on the image acquisition device if the background noise included in the scene environment information is greater than or equal to the background noise threshold, wherein the second application parameter includes at least one of a sharpness parameter, a sensitivity parameter and a noise reduction parameter.

[0311] On the basis of the embodiment corresponding to FIG. 12 , in another embodiment of the data processing device 40 provided in the embodiment of the present application, the data processing device 40 further includes a determination module 470 ;

[0312] The processing module 460 is further configured to obtain a recognition accuracy rate of the local recognition model for N images, where N is an integer greater than or equal to 1; and the N images are captured by the image acquisition device;

[0313] The determination module 470 is also used to determine that the local recognition model meets the model fine-tuning conditions if the recognition accuracy is greater than or equal to the accuracy threshold; or, the sending module 420 is also used to send model adjustment parameters to T terminals if the recognition accuracy is greater than or equal to the accuracy threshold, so that the T terminals respectively update the model parameters of their corresponding models to be trained according to the model adjustment parameters to obtain T recognition models, wherein the T terminals are associated with the on-site terminals, and T is an integer greater than or equal to 1; the acquisition module 430 is also used to obtain the voting score corresponding to each of the T terminals, wherein the voting score is determined based on the prediction result of the recognition model and the prediction result of the image recognition model; the determination module 470 is also used to determine the comprehensive recognition score based on the voting score corresponding to each terminal; the determination module 470 is also used to determine that the local recognition model meets the model fine-tuning conditions if the comprehensive recognition score is greater than or equal to the recognition score threshold.

[0314] Based on the embodiment corresponding to FIG. 12 above, in another embodiment of the data processing device 40 provided in the embodiment of the present application,

[0315] The sending module 420 is further configured to send N images captured by the image acquisition device to the server, so that the server obtains N third prediction results based on the N images using the image recognition model;

[0316] The receiving module 450 is further configured to receive N third prediction results sent by the server;

[0317] The acquisition module 430 is further configured to acquire N fourth prediction results based on the N images using the local recognition model;

[0318] The processing module 460 is further configured to verify the N fourth prediction results based on the N third prediction results to obtain recognition accuracy rates for the N images.

[0319] Based on the embodiment corresponding to FIG. 12 above, in another embodiment of the data processing device 40 provided in the embodiment of the present application,

[0320] The determination module 470 is specifically configured to sum the voting scores of the T terminals to obtain a total voting score;

[0321] The comprehensive recognition score is obtained based on the ratio between the total voting score and the T value.

[0322] Based on the embodiment corresponding to FIG. 12 above, in another embodiment of the data processing device 40 provided in the embodiment of the present application,

[0323] The determination module 470 is specifically configured to obtain a weight parameter set corresponding to each of the T terminals, wherein the weight parameter set includes at least one of a device weight, an environment weight, and a preference weight;

[0324] For each of the T terminals, the terminal's voting score is weighted using the terminal's weight parameter set to obtain the terminal's voting weighted score;

[0325] The comprehensive identification score is determined based on the voting weighted score corresponding to each terminal in the T terminals.

[0326] Based on the embodiment corresponding to FIG. 12 above, in another embodiment of the data processing device 40 provided in the embodiment of the present application,

[0327] The determining module 470 is further configured to, before sending the model adjustment parameters to the T terminals, if the on-site terminal and the at least one terminal are in the same location area, determine that the at least one terminal has an association relationship with the on-site terminal, and determine the at least one terminal as the T terminals;

[0328] or,

[0329] The determining module 470 is further configured to, before sending the model adjustment parameters to the T terminals, determine that the at least one terminal has an association relationship with the on-site terminal if the on-site terminal and at least one terminal are set with the same binding object, and determine the at least one terminal as the T terminals;

[0330] or,

[0331] The determination module 470 is further configured to, before sending the model adjustment parameters to the T terminals, determine that the at least one terminal is associated with the on-site terminal if the on-site terminal and at least one terminal are connected to the same access point, and determine the at least one terminal as the T terminals.

[0332] Based on the embodiment corresponding to FIG. 12 above, in another embodiment of the data processing device 40 provided in the embodiment of the present application,

[0333] The acquisition module 430 is further configured to acquire an image to be tested after determining that the local recognition model satisfies the model fine-tuning conditions;

[0334] The acquisition module 430 is further configured to acquire a fifth prediction result based on the image to be tested by using the local recognition model;

[0335] The acquisition module 430 is further configured to acquire T sixth prediction results from the T terminals, wherein each sixth prediction result is acquired by the terminal through the recognition model based on the image to be tested;

[0336] The processing module 460 is further configured to execute corresponding services through the local recognition model if it is determined that the local recognition model is in a stable state according to the fifth prediction result and the T sixth prediction results.

[0337] Based on the embodiment corresponding to FIG. 12 above, in another embodiment of the data processing device 40 provided in the embodiment of the present application,

[0338] The sending module 420 is further configured to, after determining the comprehensive recognition score based on the voting score corresponding to each terminal, send a model fine-tuning request to a terminal among T terminals if the local recognition model does not meet the model fine-tuning condition, so that the terminal updates the model parameters of the to-be-trained model according to the model fine-tuning request to obtain a recognition model, where the T terminals are associated with the on-site terminal, and T is an integer greater than or equal to 1.

[0339] The receiving module 450 is further configured to receive the model adjustment parameters sent by the terminal;

[0340] The updating module 440 is further configured to update the model parameters of the to-be-trained model of the on-site terminal using the model adjustment parameters sent by the terminal.

[0341] Based on the embodiment corresponding to FIG. 12 above, in another embodiment of the data processing device 40 provided in the embodiment of the present application,

[0342] The capturing module 410 is further configured to update the model parameters of the to-be-trained model based on the K second prediction results and the fine-tuning training set, obtain the local recognition model and the model adjustment parameters corresponding to the local recognition model, and, if the local recognition model satisfies the model fine-tuning conditions, capture the image to be recognized using the image acquisition device;

[0343] The acquisition module 430 is further configured to acquire a seventh prediction result based on the image to be recognized by using the local recognition model, wherein the seventh prediction result includes a predicted category and a category score;

[0344] The determination module 470 is further configured to determine that the image to be identified belongs to the predicted category in the seventh prediction result if the category score in the seventh prediction result is greater than or equal to the category score threshold.

[0345] Based on the embodiment corresponding to FIG. 12 above, in another embodiment of the data processing device 40 provided in the embodiment of the present application,

[0346] The sending module 420 is further configured to, after obtaining a seventh prediction result based on the image to be recognized by using the local recognition model, send the image to be recognized to the server if the category score in the seventh prediction result is less than the category score threshold, so that the server obtains an image recognition result based on the image to be recognized by using the image recognition model;

[0347] The receiving module 450 is further configured to receive the image recognition result sent by the server.

[0348] The data processing device in the present application is described in detail below. Please refer to FIG13 , which is a schematic diagram of an embodiment of the data processing device in the present application. The data processing device 50 includes:

[0349] The receiving module 510 is configured to receive K images sent by the on-site terminal, wherein the K images are captured by the on-site terminal through a collection device under the current on-site environment, and K is an integer greater than or equal to 1;

[0350] An acquisition module 520 is configured to acquire K first prediction results based on the K images using an image recognition model;

[0351] a sending module 530 configured to send K first prediction results to an on-site terminal, so that the on-site terminal constructs a fine-tuning training set based on the K images and the K first prediction results, and uses the fine-tuning training set to fine-tune the model to be trained on the on-site terminal. During fine-tuning of the model to be trained on the on-site terminal, based on the images included in each group of fine-tuning training data in the fine-tuning training set, a second prediction result corresponding to each image is obtained by the model to be trained, and model parameters of the model to be trained are updated based on the second prediction result corresponding to each image and the first prediction result of the image in the fine-tuning training set, thereby obtaining a local recognition model and model adjustment parameters corresponding to the local recognition model, wherein the fine-tuning training set includes K groups of fine-tuning training data, each group of fine-tuning training data including an image and the first prediction result of the image;

[0352] The receiving module 510 is further configured to receive the model adjustment parameters sent by the on-site terminal if the local recognition model satisfies the model fine-tuning condition;

[0353] The updating module 540 is configured to update the model parameters of the image recognition model upon obtaining a model adjustment parameter set from at least one terminal, wherein the model adjustment parameter set includes the model adjustment parameters.

[0354] Based on the embodiment corresponding to FIG. 13 above, in another embodiment of the data processing device 50 provided in the embodiment of the present application,

[0355] An updating module 540 is specifically configured to obtain a set of model adjustment parameters from at least one terminal;

[0356] Performing weighted processing on the model adjustment parameter set according to the comprehensive recognition score corresponding to each terminal to obtain a weighted model adjustment parameter set;

[0357] The model parameters of the image recognition model are updated using the weighted model adjustment parameter set.

[0358] Based on the embodiment corresponding to FIG. 13 above, in another embodiment of the data processing device 50 provided in the embodiment of the present application,

[0359] An updating module 540 is specifically configured to update model parameters of the image recognition model when a model parameter set is obtained from at least one terminal, wherein the model parameter set includes a model adjustment parameter, which is a model parameter;

[0360] or,

[0361] updating model parameters of the image recognition model when a gradient set is obtained from at least one terminal, wherein the gradient set includes a model adjustment parameter, and the model adjustment parameter is a gradient;

[0362] or,

[0363] When an optimization algorithm parameter set is obtained from at least one terminal, the model parameters of the image recognition model are updated, wherein the optimization algorithm parameter set includes model adjustment parameters, and the model adjustment parameters are optimization algorithm parameters.

[0364] Based on the embodiment corresponding to FIG. 13 above, in another embodiment of the data processing device 50 provided in the embodiment of the present application,

[0365] The sending module 530 is also used to send the model adjustment parameters of the image recognition model to at least one terminal after updating the model parameters of the image recognition model when a model adjustment parameter set is obtained from at least one terminal, so that each terminal in the at least one terminal uses the model adjustment parameters of the image recognition model to update the model parameters of the recognition model.

[0366] 14 is a schematic diagram of the structure of a computer device 60 according to an embodiment of the present application. The computer device 60 may include an input device 610, an output device 620, a processor 630, and a memory 640. The output device 620 in the embodiment of the present application may be a display device.

[0367] The memory 640 may include a read-only memory and a random access memory, and provides instructions and data to the processor 630. A portion of the memory 640 may also include a non-volatile random access memory (NVRAM).

[0368] The memory 640 stores the following elements, executable modules or data structures, or a subset or an extension thereof:

[0369] Operation instructions: including operation instructions, used to implement operations.

[0370] Operating system: includes system programs used to implement basic business and handle hardware-based tasks.

[0371] The processor 630 controls the computer device 60 to perform operations. The processor 630 may also be referred to as a central processing unit (CPU).

[0372] In a specific application, the various components of the computer device 60 are coupled together via a bus system 650. In addition to a data bus, the bus system 650 may also include a power bus, a control bus, and a status signal bus. For ease of description, various buses are labeled as the bus system 650 in the figure.

[0373] The methods provided in the embodiments of the present application can be applied to or implemented by the processor 630. The processor 630 may be an integrated circuit chip with signal processing capabilities. During implementation, the steps of the above method can be completed by hardware-based integrated logic circuits or software-based instructions in the processor 630. The processor 630 can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component, and can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of the present application. The general-purpose processor can be a microprocessor or other conventional processor. The steps of the methods disclosed in conjunction with the embodiments of the present application can be directly implemented by hardware in the processor, or by a combination of hardware and software modules in the processor. The software module can be located in a storage medium mature in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, etc. The storage medium is located in the memory 640 , and the processor 630 reads the information in the memory 640 and completes the steps of the above method in combination with its hardware.

[0374] The steps executed by the terminal or server in the above embodiment may be based on the computer device structure shown in FIG14 .

[0375] A computer-readable storage medium is also provided in an embodiment of the present application, on which a computer program is stored. When the computer program is executed by a processor, the steps of the methods described in the above embodiments are implemented.

[0376] A computer program product is also provided in an embodiment of the present application, including a computer program. When the computer program is executed by a processor, the steps of the methods described in the above embodiments are implemented.

[0377] It is understandable that in the specific implementation of this application, data related to palm print information, iris information, facial information, etc. are involved. When the above embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions.

[0378] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0379] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0380] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0381] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0382] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for enabling a computer device (which can be a server or terminal device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store computer programs, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0383] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A data processing method, the method being executed by a field terminal, comprising: Under the current on-site environment, K images are obtained by taking pictures with an image acquisition device, wherein K is an integer greater than or equal to 1; Sending the K images to a server, so that the server obtains K first prediction results through an image recognition model based on the K images; Constructing a fine-tuning training set according to the K images and the K first prediction results sent by the server, wherein the fine-tuning training set includes K groups of fine-tuning training data, each group of fine-tuning training data includes an image and the first prediction result of the image; The model to be trained on the on-site terminal is fine-tuned using the fine-tuning training set. In the process of fine-tuning the model to be trained on the on-site terminal, based on the images included in each group of fine-tuning training data in the fine-tuning training set, a second prediction result corresponding to each image is obtained through the model to be trained, and according to the second prediction result corresponding to each image and the first prediction result of the image in the fine-tuning training set, the model parameters of the model to be trained are updated to obtain a local recognition model and model adjustment parameters corresponding to the local recognition model; If the local recognition model meets the model fine-tuning conditions, the model adjustment parameters are sent to the server so that the server updates the model parameters of the image recognition model according to a model adjustment parameter set from at least one terminal, wherein the model adjustment parameter set includes the model adjustment parameters.

2. The method according to claim 1, further comprising: Sending a model training request to the server, so that the server determines a training data set applied to the field terminal according to the model training request; Receiving an initial training set sent by the server, wherein the initial training set includes M groups of initial training data, each group of initial training data includes an image and a labeling result of the image; Based on the M images included in the initial training set, M initial prediction results are obtained through the initial recognition model, wherein each initial prediction result includes a predicted category and a category score of the image; According to the M initial prediction results and the M annotation results included in the initial training set, the model parameters of the initial recognition model are updated to obtain the model to be trained.

3. The method according to claim 1, before obtaining K images by the image acquisition device under the current scene environment, the method further comprises: Acquiring on-site environment information of the on-site terminal, wherein the on-site environment information includes at least one of light intensity and background noise; If the light intensity included in the scene environment information is not within the light intensity range, in response to a first adjustment operation on the image acquisition device, adjusting a first application parameter of the image acquisition device, wherein the first application parameter includes at least one of a shutter speed, a sensitivity parameter, and an exposure compensation parameter; If the background noise included in the scene environment information is greater than or equal to the background noise threshold, then in response to a second adjustment operation on the image acquisition device, a second application parameter of the image acquisition device is adjusted, wherein the second application parameter includes at least one of a sharpness parameter, a sensitivity parameter, and a noise reduction parameter.

4. According to the method of claim 1, after updating the model parameters of the model to be trained according to the second prediction result corresponding to each of the images and the first prediction result of the image in the fine-tuning training set to obtain the local recognition model and the model adjustment parameters corresponding to the local recognition model, the method further comprises: Obtaining the recognition accuracy of the local recognition model for N images, wherein N is an integer greater than or equal to 1; the N images are captured by the image acquisition device; If the recognition accuracy is greater than or equal to the accuracy threshold, determining that the local recognition model meets the model fine-tuning condition; Alternatively, if the recognition accuracy is greater than or equal to the accuracy threshold, the model adjustment parameters are sent to T terminals, so that the T terminals respectively update the model parameters of the corresponding models to be trained according to the model adjustment parameters, and obtain T recognition models, wherein the T terminals are associated with the on-site terminals, and T is an integer greater than or equal to 1; the voting score corresponding to each of the T terminals is obtained, wherein the voting score is determined based on the prediction result of the recognition model and the prediction result of the image recognition model; a comprehensive recognition score is determined based on the voting score corresponding to each terminal; if the comprehensive recognition score is greater than or equal to the recognition score threshold, it is determined that the local recognition model meets the model fine-tuning condition.

5. According to the method of claim 4, the obtaining the recognition accuracy of the local recognition model for N images comprises: Sending the N images captured by the image acquisition device to the server, so that the server obtains N third prediction results through the image recognition model based on the N images; Receiving the N third prediction results sent by the server; Based on the N images, obtaining N fourth prediction results by using the local recognition model; The N fourth prediction results are verified according to the N third prediction results to obtain recognition accuracy rates for the N images.

6. The method according to claim 4, wherein determining the comprehensive identification score according to the voting score corresponding to each terminal comprises: Sum the voting scores of the T terminals to obtain a total voting score; The comprehensive recognition score is obtained according to the ratio between the total voting score and the T value.

7. The method according to claim 4, wherein determining the comprehensive identification score according to the voting score corresponding to each terminal comprises: Acquire a weight parameter set corresponding to each of the T terminals, wherein the weight parameter set includes at least one of a device weight, an environment weight, and a preference weight; For each terminal among the T terminals, weight the voting score of the terminal using the weight parameter set of the terminal to obtain the voting weighted score of the terminal; The comprehensive identification score is determined according to the voting weighted score corresponding to each terminal in the T terminals.

8. The method according to claim 4, before sending the model adjustment parameters to T terminals, the method further comprises: If the on-site terminal and at least one terminal are in the same location area, determining that the at least one terminal has an association relationship with the on-site terminal, and determining the at least one terminal as the T terminals; or, If the on-site terminal and at least one terminal are set with the same binding object, determining that the at least one terminal has an association relationship with the on-site terminal, and determining the at least one terminal as the T terminals; or, If the on-site terminal and at least one terminal are connected to the same access point, it is determined that the at least one terminal has an association relationship with the on-site terminal, and the at least one terminal is determined as the T terminals.

9. The method according to claim 4, after determining that the local recognition model satisfies the model fine-tuning condition, the method further comprises: Get the image to be tested; Based on the image to be tested, obtaining a fifth prediction result by using the local recognition model; Obtaining T sixth prediction results from the T terminals, wherein each sixth prediction result is obtained by the terminal through a recognition model based on the image to be tested; If it is determined based on the fifth prediction result and the T sixth prediction results that the local recognition model is in a model stable state, the corresponding business is performed through the local recognition model.

10. The method according to claim 1, further comprising: If the local recognition model does not meet the model fine-tuning condition, a model fine-tuning request is sent to a terminal among the T terminals, so that the terminal updates the model parameters of the to-be-trained model according to the model fine-tuning request to obtain a recognition model, wherein the T terminals are associated with the on-site terminal, and T is an integer greater than or equal to 1; Receiving a model adjustment parameter sent by the terminal; The model adjustment parameters sent by the terminal are used to update the model parameters of the model to be trained of the on-site terminal.

11. According to the method of any one of claims 1 to 10, after updating the model parameters of the model to be trained according to the K second prediction results and the fine-tuning training set to obtain the local recognition model and the model adjustment parameters corresponding to the local recognition model, the method further comprises: When the local recognition model satisfies the model fine-tuning condition, an image to be recognized is obtained by photographing the image acquisition device; Based on the image to be recognized, obtaining a seventh prediction result through the local recognition model, wherein the seventh prediction result includes a prediction category and a category score; If the category score in the seventh prediction result is greater than or equal to the category score threshold, it is determined that the image to be identified belongs to the predicted category in the seventh prediction result.

12. The method according to claim 11, after obtaining the seventh prediction result through the local recognition model based on the image to be recognized, the method further comprises: If the category score in the seventh prediction result is less than the category score threshold, sending the image to be recognized to the server, so that the server obtains an image recognition result through the image recognition model based on the image to be recognized; Receive the image recognition result sent by the server.

13. A data processing method, the method being executed by a server, comprising: Receiving K images sent by a field terminal, wherein the K images are taken by the field terminal through a collection device under a current field environment, and K is an integer greater than or equal to 1; Based on the K images, obtaining K first prediction results through an image recognition model; Sending the K first prediction results to the on-site terminal so that the on-site terminal constructs a fine-tuning training set according to the K images and the K first prediction results, and using the fine-tuning training set to fine-tune the model to be trained on the on-site terminal. In the process of fine-tuning the model to be trained on the on-site terminal, based on the images included in each group of fine-tuning training data in the fine-tuning training set, obtaining a second prediction result corresponding to each image through the model to be trained, and updating the model parameters of the model to be trained according to the second prediction result corresponding to each image and the first prediction result of the image in the fine-tuning training set, so as to obtain a local recognition model and a model adjustment parameter corresponding to the local recognition model, wherein the fine-tuning training set includes K groups of fine-tuning training data, and each group of fine-tuning training data includes an image and the first prediction result of the image; If the local recognition model satisfies the model fine-tuning condition, receiving the model adjustment parameter sent by the on-site terminal; In a case where a model adjustment parameter set is obtained from at least one terminal, the model parameters of the image recognition model are updated, wherein the model adjustment parameter set includes the model adjustment parameters.

14. The method according to claim 13, wherein in the case of obtaining a model adjustment parameter set from at least one terminal, updating the model parameters of the image recognition model comprises: Acquire a model adjustment parameter set from the at least one terminal; According to the comprehensive identification score corresponding to each terminal, the model adjustment parameter set is weighted to obtain a weighted model adjustment parameter set; The model parameters of the image recognition model are updated using the weighted model adjustment parameter set.

15. The method according to claim 13, wherein in the case of obtaining a model adjustment parameter set from at least one terminal, updating the model parameters of the image recognition model comprises: In the case of obtaining a model parameter set from at least one terminal, updating the model parameters of the image recognition model, wherein the model parameter set includes the model adjustment parameter, and the model adjustment parameter is a model parameter; or, In a case where a gradient set from at least one terminal is obtained, updating a model parameter of the image recognition model, wherein the gradient set includes the model adjustment parameter, and the model adjustment parameter is a gradient; or, In case of obtaining an optimization algorithm parameter set from at least one terminal, the model parameters of the image recognition model are updated, wherein the optimization algorithm parameter set includes the model adjustment parameters, and the model adjustment parameters are optimization algorithm parameters.

16. The method according to any one of claims 13 to 15, wherein after updating the model parameters of the image recognition model in the case of obtaining the model adjustment parameter set from at least one terminal, the method further comprises: The model adjustment parameters of the image recognition model are sent to at least one terminal, so that each of the at least one terminal uses the model adjustment parameters of the image recognition model to update the model parameters of the recognition model.

17. A data processing device, the device being deployed on a field terminal, comprising: A shooting module, used to obtain K images by shooting through an image acquisition device under the current scene environment, wherein K is an integer greater than or equal to 1; A sending module, used for sending the K images to a server, so that the server obtains K first prediction results through an image recognition model based on the K images; An acquisition module, configured to construct a fine-tuning training set according to the K images and the K first prediction results sent by the server, wherein the fine-tuning training set includes K groups of fine-tuning training data, each group of fine-tuning training data includes an image and the first prediction result of the image; An updating module, configured to use the fine-tuning training set to fine-tune the model to be trained on the on-site terminal. In the process of fine-tuning the model to be trained on the on-site terminal, based on the images included in each group of fine-tuning training data in the fine-tuning training set, a second prediction result corresponding to each image is obtained through the model to be trained, and according to the second prediction result corresponding to each image and the first prediction result of the image in the fine-tuning training set, a model parameter of the model to be trained is updated to obtain a local recognition model and a model adjustment parameter corresponding to the local recognition model; The sending module is also used to send the model adjustment parameters to the server if the local recognition model meets the model fine-tuning conditions, so that the server updates the model parameters of the image recognition model according to the model adjustment parameter set from at least one terminal, wherein the model adjustment parameter set includes the model adjustment parameters.

18. A data processing device, the device being deployed on a server, comprising: A receiving module, used to receive K images sent by a field terminal, wherein the K images are taken by the field terminal through a collection device under a current field environment, and K is an integer greater than or equal to 1; An acquisition module, used for acquiring K first prediction results based on the K images through an image recognition model; A sending module, used for sending the K first prediction results to the on-site terminal, so that the on-site terminal constructs a fine-tuning training set according to the K images and the K first prediction results, and uses the fine-tuning training set to fine-tune the model to be trained on the on-site terminal. In the process of fine-tuning the model to be trained on the on-site terminal, based on the images included in each group of fine-tuning training data in the fine-tuning training set, a second prediction result corresponding to each image is obtained through the model to be trained, and according to the second prediction result corresponding to each image and the first prediction result of the image in the fine-tuning training set, the model parameters of the model to be trained are updated to obtain a local recognition model and a model adjustment parameter corresponding to the local recognition model, wherein the fine-tuning training set includes K groups of fine-tuning training data, and each group of fine-tuning training data includes an image and the first prediction result of the image; The receiving module is further configured to receive the model adjustment parameters sent by the on-site terminal if the local recognition model meets the model fine-tuning condition; An updating module is used to update the model parameters of the image recognition model when a model adjustment parameter set is obtained from at least one terminal, wherein the model adjustment parameter set includes the model adjustment parameters.

19. A computer device comprising a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the processor implements the steps of the method according to any one of claims 1 to 12, or implements the steps of the method according to any one of claims 13 to 16.

20. A computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the computer program implements the steps of the method according to any one of claims 1 to 12, or implements the steps of the method according to any one of claims 13 to 16.

21. A computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements the steps of the method according to any one of claims 1 to 12, or the computer program implements the steps of the method according to any one of claims 13 to 16.

Images