Method and device for training and distributing gesture recognition models based on blockchain and images
By acquiring image feature data and gesture labels on edge computing devices, training domain adaptive models and inputting support vector machines, the problem of insufficient computing performance in the training of gesture recognition models of edge computing devices is solved, and high-precision gesture recognition and low-level gesture recognition are achieved Cost transmission ensures data privacy.
Patent Information
- Application Number
- CN202011620074.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-12-30
- Publication Date
- 2024-12-06
- Estimated Expiration
- 2040-12-30
AI Technical Summary
In the prior art, edge computing devices lack computational performance during gesture recognition model training, resulting in the need to send original image data to the server, which has problems such as low security, low recognition accuracy and high transmission cost.
By obtaining the image feature data and gesture tags of edge computing devices, inputting image feature data, training domain adaptive model for optimized feature description, and inputting the optimized feature description and gesture tags into support vector machine training, obtaining a gesture recognition classifier, so as to be at the edge Gesture recognition is realized on computing devices, only uploading image features and downloading recognition results, ensuring privacy and reducing transmission costs.
High-precision gesture recognition on edge computing devices is realized, data privacy is guaranteed, transmission costs are reduced, and recognition accuracy is improved through domain adaptive models.
Smart Images

Figure CN112801134B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of blockchain technology, and more specifically, to a method, system, device, computer equipment, and computer-readable storage medium for training and distributing a gesture recognition model based on blockchain and images. Background Art
[0002] Gesture recognition methods based on two-dimensional images have been widely studied and applied. However, to implement gesture recognition model training on edge computing devices, there is still a problem of insufficient computing performance. Therefore, it is necessary to send data to the server for model training, and then get the trained model from the server.
[0003] However, in the prior art, the transmission of original subject images and other information between edge computing devices and servers is prone to privacy leakage and has high transmission costs; because there is no appropriate training model, the gesture recognition accuracy is not high.
[0004] In summary, the prior art has technical problems such as low security, low recognition accuracy, high transmission cost, and easy abuse of servers. Summary of the invention
[0005] In order to solve the above technical problems, the present invention provides a method, device, system, computer equipment and computer-readable storage medium for training and distributing a gesture recognition model based on blockchain and images.
[0006] A method for training and distributing a gesture recognition model based on blockchain and images, comprising:
[0007] Acquire image feature data of an edge computing device, where the image feature data corresponds to a gesture tag;
[0008] Input the image feature data to train a domain adaptive model to optimize feature description of the image feature data;
[0009] The image feature data after the optimized feature description and the gesture label are input into a support vector machine for training to obtain a gesture recognition classifier.
[0010] A gesture recognition model training and distribution device based on blockchain and image, comprising:
[0011] An acquisition module, used to acquire image feature data of an edge computing device, wherein the image feature data corresponds to a gesture tag;
[0012] A model training module, used for inputting the image feature data to train a domain adaptive model and optimizing feature description of the image feature data;
[0013] The classifier training module is used to input the image feature data after the optimized feature description and the gesture label into a support vector machine for training to obtain a gesture recognition classifier.
[0014] A blockchain and image-based gesture recognition model training and distribution system, comprising: a plurality of edge computing devices and a server; the server runs an acquisition module, a model training module, and a classifier training module;
[0015] The acquisition module acquires image feature data of the edge computing device, where the image feature data corresponds to a gesture label;
[0016] The model training module inputs the image feature data to train a domain adaptive model, and performs optimized feature description on the image feature data;
[0017] The classifier training module inputs the image feature data after the optimized feature description and the gesture label into a support vector machine for training to obtain a gesture recognition classifier.
[0018] A computer device comprises a memory and a processor, wherein the memory stores a computer program and the computer program executes an implementable method in the processor.
[0019] A computer-readable storage medium stores a computer program, wherein the computer program can implement the above method when executed in a processor.
[0020] The present invention provides a method, device and system for training and distributing a gesture recognition model based on blockchain and images. The method obtains image feature data of an edge computing device, the image feature data corresponds to a gesture label, the image feature data is input to train a domain adaptive model, the image feature data is optimized for feature description, the image feature data after the optimized feature description and the gesture label are input to a support vector machine for training, and a gesture recognition classifier is obtained. Thus, only image features are uploaded and recognition results are downloaded between the edge computing device and the server, and the original image and other information are not transmitted, privacy is guaranteed, the amount of feature data is smaller than that of the original image, and the transmission cost is reduced. Training data on different edge computing devices is collected, the domain adaptive model is trained to improve the model, and the image feature data after the optimized feature description is used to train the support vector machine, so as to obtain a more accurate gesture recognition classifier. BRIEF DESCRIPTION OF THE DRAWINGS
[0021] Figure 1 A schematic diagram of a framework of a blockchain- and image-based gesture recognition model training and distribution system provided by an embodiment;
[0022] Figure 2 A flowchart of a method for training and distributing a gesture recognition model based on blockchain and images provided by an embodiment;
[0023] Figure 3 A schematic diagram of the architecture of a gesture recognition model training and distribution device based on blockchain and images provided by an embodiment;
[0024] Figure 4 A schematic diagram of a computer device provided in accordance with an embodiment of the present invention. DETAILED DESCRIPTION
[0025] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0026] The method for training and distributing a gesture recognition model based on blockchain and images provided in this embodiment can be applied in the following aspects: Figure 1 In the application environment, the edge computing device communicates with the server through the network, and the server runs in the blockchain system. The edge computing device can be, but is not limited to, various personal computers, laptops, smartphones, tablets, and portable wearable devices. The server can be implemented as an independent server or a server cluster consisting of multiple servers.
[0027] In one embodiment, if Figure 2 As shown in the figure, a method for training and distributing a gesture recognition model based on blockchain and image is provided, and the method is applied in Figure 1 The server in the example is used as an example to illustrate the process, including the following steps:
[0028] S1. Obtain image feature data of an edge computing device, where the image feature data corresponds to a gesture label;
[0029] S2, inputting image feature data to train a domain adaptation model to optimize feature description of the image feature data;
[0030] S3. Input the image feature data and gesture labels after the optimized feature description into a support vector machine for training to obtain a gesture recognition classifier.
[0031] In this embodiment, by obtaining image feature data of an edge computing device, the image feature data corresponds to a gesture label, and the image feature data is input to train a domain adaptive model to optimize the feature description of the image feature data. The image feature data and the gesture label after the optimized feature description are input into a support vector machine for training to obtain a gesture recognition classifier, so that only image features are uploaded and recognition results are downloaded between the edge computing device and the server, and the original image and other information are not transmitted, thereby ensuring privacy. The amount of feature data is smaller than that of the original image, reducing the transmission cost, collecting training data on different edge computing devices, training a domain adaptive model to improve the model, and using the image feature data after the optimized feature description to train a support vector machine to obtain a more accurate gesture recognition classifier.
[0032] In step S1, the edge computing device sends device information, such as the device serial number, to the server. The serial number is verified on the edge computing device information chain. Devices that pass the verification can request the server to distribute the gesture recognition model.
[0033] For example, when an edge computing device uploads valid data, it can send device information such as the device serial number, and the serial number generates an MD5 code on the server; the device information is uploaded to the blockchain after being verified by the PBFT consensus mechanism.
[0034] It should be noted that the edge computing device can upload device information to the blockchain through the server. The uploaded information includes but is not limited to device information and contributed data volume. In this way, when requesting the server to distribute the gesture recognition model, the server first initiates information verification to the blockchain, clarifies the identity and data volume contribution of the requesting device, and responds to or rejects the request of the requesting device. In this way, not only can the authenticity and privacy of the information on the chain be guaranteed through the blockchain, but also the server can be prevented from being abused.
[0035] It should also be noted that the image data provided by the edge computing device to the server is image feature data, which can be feature-extracted data that corresponds to the image data one-to-one. Compared with the original image data, it has the advantage of small data volume, thereby effectively reducing transmission costs during transmission. Among them, the image feature data corresponds to gesture labels, such as pre-defined different gesture categories such as clenched fist and open fist.
[0036] In a specific embodiment, the image feature data is extracted by a denoising autoencoder, which may be:
[0037] Given a set of image samples, X = [x1, x2, x3...x n ]∈R d ; n represents the number of samples, d represents the length of the original image vector; the autoencoder is used to reduce the reconstruction error: xi and are the original input and the reconstructed output respectively; Represented as x i Corrupted version, W:R d →R d A mapping representing the reconstruction of corrupted input;
[0038] The square of the reconstruction loss is defined as The total squared loss is defined as Represents the original input x i The j-th damaged version of , m is the number of layers;
[0039] X=[x1,x2,……,x n ]∈R d*n Represented as a data matrix, Represented as a version repeated m times, is represented as a corrupted version of x, The loss is reduced to Minimize the closed-form solution of ordinary least squares: W = VQ -1 ;in and
[0040] It is understandable that the data source provided by edge computing devices can use any camera to capture images, or directly input existing pictures, videos, etc.
[0041] It should be noted that the image feature data extracted by the denoising autoencoder is used as the subsequent training set and test set to train and test the model. Since the reconstruction error is reduced, the trained model can achieve more accurate recognition.
[0042] In step S2, by inputting the image feature data to train the domain adaptation model and optimizing the feature description of the image feature data, the features of the test set can be mapped to the space of the training set, so that the distribution of the test set and the training set are similar, thereby optimizing the training model.
[0043] It should be noted that training a domain adaptation model may refer to providing data with correct answers to obtain a domain adaptation model.
[0044] In a specific embodiment, a preferred training scheme for a domain adaptation model is provided.
[0045] The image feature data is represented as
[0046] By implementing low-rank representation (LRR)
[0047] Derive the correlation matrix used in the alignment process;
[0048] Given a eigenvector matrix X, the correlation between eigenvectors is calculated by solving the following formula:
[0049] Calculation formula min||Q0|| * +λ||E0|| 2,1 and Get the correlation between feature vectors, is the image feature data, ||||* represents the nuclear norm, and ||||2, 1 is used to describe l 2,1 The noise, E0 is used to ensure the penalty term of sparsity, and the parameter λ is used to balance the influence of the two parts.
[0050] L k =D k -W k The optimal solution (Regarding the Q0 variable) a similarity matrix is automatically formed, which represents the pairwise similarity between eigenvectors. Therefore, the similarity Skl between two eigenvectors xk and xl can be obtained by (·) lk Represents the (l, k) coordinate element of the matrix.
[0051] By computing the low-rank representation, the local geometric graphs of the source and target domains can be represented by adjacency matrices Ss and St. In order to learn the alignment relationship between them, the general framework of manifold alignment can be used. For each adjacency matrix, the diagonal matrix can be calculated K represents the source s or the target t. With Dk and Sk, the Laplacian matrix can be calculated as: L k =D k -W k Then, the overall structure of the source domain and the target domain can be expressed as:
[0052]
[0053] C=2 means aligning the two matrices in this application, It is a diagonal matrix and is calculated by the following formula:
[0054] Through the formula calculate.
[0055] To calculate the alignment relationship, the d-dimensional embedding result is calculated by the eigenvalue decomposition of the d smallest eigenvectors γ1,γ2,...,γd: Lγ=λDr.
[0056] Finally, we can find the relationship between Ss and St, let Ys be a part of [γ1,γ2,...,γd] (from the 1st row to the nth row). Let Yt be another part of [γ1,γ2,...,γd] (from the n+1th row to the 2nth row). Let Ys and Yt be directly compared in the same space. In this way, Xs and Xt can be mapped to the same space. At the same time, the cost function can be defined as:
[0057]
[0058] By minimizing the cost function, we can get a model that maps the test set to the training set.
[0059] In step S3, the image feature data and gesture labels after the optimized feature description are input into a support vector machine for training to obtain a gesture recognition classifier.
[0060] It should be noted that after the image feature data is optimized for feature description and then input into the support vector machine for training, a gesture recognition classifier with a recognition result closer to the real one can be obtained through training.
[0061] Furthermore, in a specific embodiment, the following data uplink operations may be performed:
[0062] Upload edge computing device information to the blockchain;
[0063] Upload the domain adaptation model and / or gesture recognition classifier to the blockchain;
[0064] Distribute the domain adaptation model and / or gesture recognition classifier to the verified edge computing devices.
[0065] Among them, the domain adaptive model on-chain can be:
[0066] Generate MD5 code through the binary data of the model;
[0067] The new model is put on the chain after being verified by the PBFT consensus mechanism;
[0068] The gesture recognition classifier on-chain can be:
[0069] Generate MD5 code through binary data of classifier;
[0070] The new classifier is put on the chain after verification by the PBFT consensus mechanism;
[0071] The edge computing device information uploaded to the chain can be:
[0072] When the edge computing device uploads valid data, it sends its own device serial number, which generates an MD5 code on the server.
[0073] The device information is uploaded to the chain after verification through the PBFT consensus mechanism.
[0074] It should be noted that uploading the above data to the blockchain can prevent the data (such as the model) from being abused by unverified devices, and the model can be distributed to the edge computing device after verification, so that the gesture recognition process can be implemented on the edge computing device.
[0075] It should be understood that the order of execution of the steps in the above embodiment does not necessarily mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention.
[0076] In one embodiment, see Figure 3 , a device for training and distributing a gesture recognition model based on blockchain and images is provided, which corresponds one-to-one with the method for training and distributing a gesture recognition model based on blockchain and images in the above-mentioned embodiment. Figure 3 As shown, the device includes an acquisition module 1, a model training module 2 and a classifier training module 3.
[0077] Acquisition module 1, used to acquire image feature data of the edge computing device, where the image feature data corresponds to the gesture label;
[0078] Model training module 2, used for inputting image feature data to train a domain adaptive model so as to optimize feature description of the image feature data;
[0079] The classifier training module 3 is used to input the image feature data and gesture labels after the optimized feature description into the support vector machine for training to obtain a gesture recognition classifier.
[0080] It should be noted that the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or modules is not necessarily limited to those steps or modules clearly listed, but may include other steps or modules that are not clearly listed or are inherent to these processes, methods, products or devices. The division of modules appearing in this application is merely a logical division, and there may be other division methods when implemented in actual applications.
[0081] It should also be noted that, for the specific limitations and descriptions of the above-mentioned device, please refer to the limitations and descriptions of the training and distribution method of the gesture recognition model based on blockchain and images above, which will not be repeated here. Each module in the above-mentioned device can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.
[0082] In one embodiment, see Figure 3 and Figure 4 , a computer device is provided, which may be a server, and the server runs an acquisition module 1, a model training module 2, and a classifier training module 3. The internal structure diagram of the computer device may be as follows Figure 4 As shown. The computer device includes a processor, a memory, a network interface and a database connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data involved in the training and distribution method of the gesture recognition model based on blockchain and images. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, the training and distribution method of the gesture recognition model based on blockchain and images is implemented.
[0083] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the method for training and distributing a gesture recognition model based on blockchain and images in the above embodiment are implemented, such as Figure 2 Steps 1 to 3 shown in the figure and other extensions of the method and extensions of related steps. Alternatively, when the processor executes the computer program, the functions of each module / unit of the gesture recognition model training and distribution device based on blockchain and image in the above embodiment are realized, for example Figure 3 The functions of modules 1 to 3 are shown in Figure 1. To avoid repetition, they will not be described here.
[0084] The processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc. The processor is the control center of a computer device, and uses various interfaces and lines to connect various parts of the entire computer device.
[0085] The memory can be used to store computer programs and / or modules. The processor implements various functions of the computer device by running or executing the computer programs and / or modules stored in the memory and calling the data stored in the memory. The memory can mainly include a program storage area and a data storage area. The program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area can store data created according to the use of the mobile phone (such as audio data, video data, etc.), etc.
[0086] The memory may be integrated into the processor or provided separately from the processor.
[0087] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the method for training and distributing a gesture recognition model based on blockchain and images in the above embodiment are implemented, such as Figure 2 Steps 1 to 3 shown in the figure and other extensions of the method and related steps. Alternatively, when the computer program is executed by the processor, the functions of each module / unit of the gesture recognition model training and distribution device based on blockchain and image in the above embodiment are realized, for example Figure 3 The functions of modules 1 to 3 are shown in Figure 1. To avoid repetition, they will not be described here.
[0088] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in the present application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).
[0089] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.
[0090] The embodiments described above are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included in the protection scope of the present invention.
Claims
1. A method for training and distributing a gesture recognition model based on blockchain and images, characterized in that: include: The server obtains image feature data of the edge computing device, where the image feature data corresponds to the gesture tag, and the server communicates with the edge computing device through a network; The server inputs the image feature data to train a domain adaptive model, and performs optimized feature description on the image feature data; The server inputs the image feature data after the optimized feature description and the gesture label into a support vector machine for training to obtain a gesture recognition classifier; The server receives the device serial number sent by the edge computing device, the device serial number generates an MD5 code on the server, the device information is uploaded to the blockchain after verification by the PBFT consensus mechanism, and the server runs in the blockchain system; The server uploads the domain adaptation model and / or the gesture recognition classifier to the blockchain; The edge computing device requests the server to distribute the gesture recognition model, and the server initiates information verification to the blockchain, clarifies the identity and data volume contribution of the requesting device, and responds to the request of the requesting device or rejects the request of the requesting device.
2. The method according to claim 1, characterized in that Also includes: The image feature data is extracted by a denoising autoencoder.
3. A gesture recognition model training and distribution system based on blockchain and images, characterized in that: include: Multiple edge computing devices and servers; The server runs an acquisition module, a model training module, a classifier training module and an upload module; The acquisition module acquires image feature data of the edge computing device, the image feature data corresponds to the gesture tag, and the server communicates with the edge computing device through a network; The model training module inputs the image feature data to train a domain adaptive model, and performs optimized feature description on the image feature data; The classifier training module inputs the image feature data after the optimized feature description and the gesture label into a support vector machine for training to obtain a gesture recognition classifier; The upload module receives the device serial number sent by the edge computing device, the device serial number generates an MD5 code on the server, and the device information is uploaded to the blockchain after verification by the PBFT consensus mechanism. The upload module is also used to upload the edge computing device information to the blockchain and the domain adaptation model and / or the gesture recognition classifier to the blockchain. The server runs in the blockchain system; The edge computing device requests the server to distribute the gesture recognition model, and the server initiates information verification to the blockchain, clarifies the identity and data volume contribution of the requesting device, and responds to the request of the requesting device or rejects the request of the requesting device.
4. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: The computer program is executed in the processor to implement any one of the methods in claims 1-2.
5. A computer-readable storage medium storing a computer program, characterized in that: The computer program is executed in a processor to implement any method in claims 1-2.
Citation Information
Patent Citations
Identity recognition system, client, server and method
CN110378092A
Machine learning model processing method based on blockchain network and node
CN110490305A
