Object recognition method, device, equipment and medium based on edge computing

Abstract

The present invention relates to the field of artificial intelligence technology, and provides an object recognition method, device, equipment and medium based on edge computing. The method comprises: obtaining an original model to be processed, performing a pruning operation on the original model based on the weights of all convolution kernels in each layer of the original model to obtain a pruned model, training the pruned model based on a first training sample set to obtain a model after pruning, performing mixed precision training on the pruned model based on a second training sample set, converting the trained model into a binary model file of a half-precision floating point number to obtain a target compression model, so that the client can load the target compression model to identify the object to be identified. The present invention can improve the timeliness of model recognition and reduce the risk of server downtime. The present invention also relates to the field of blockchain technology, and the above-mentioned target compression model can be stored in a node of a blockchain.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to an object recognition method, device, equipment and storage medium based on edge computing. Background Art

[0002] At present, large-scale image recognition models are often deployed on the server side. When users use them, they first need to upload the image to be recognized to the image library through the App, and then transfer it from the image library to the server to call the model interface, and finally get the recognition result and return it to the user and the database. This process requires three network transmissions. The waiting time from uploading the picture to getting the result is the accumulation of the network transmission time and the algorithm calculation time. Once the user is in an area with poor network signals, the waiting time will be extended indefinitely. And because the algorithm is uniformly deployed on the central server, if a large number of users call the model interface at the same time, the server will be under extremely high concurrency pressure, which is prone to downtime risks. Summary of the invention

[0003] In view of the above, the present invention provides an object recognition method, device, equipment and storage medium based on edge computing, which aims to solve the technical problems in the prior art that the model deployed on the server requires multiple data transmissions and easily causes server crashes.

[0004] To achieve the above object, the present invention provides an object recognition method based on edge computing, the method comprising:

[0005] Obtaining an original model to be processed, and performing a pruning operation on the original model based on the weights of all convolution kernels in each layer of the original model to obtain a pruned model;

[0006] Training the pruned model based on the first training sample set to obtain a pruned model;

[0007] Based on the second training sample set, mixed precision training is performed on the pruned model, and the model after mixed precision training is converted into a binary model file of half-precision floating point numbers to obtain a target compression model, so that the client can load the target compression model to identify the object to be identified.

[0008] Preferably, performing a pruning operation on the original model based on the weights of all convolution kernels in each layer of the original model includes:

[0009] The sum of the absolute values ​​of the weights of all convolution kernels in each layer of the original model is calculated, the convolution kernels whose absolute value sum is greater than a preset threshold are retained, and the convolution kernels whose absolute value is less than or equal to the preset threshold are discarded.

[0010] Preferably, performing a pruning operation on the original model based on the weights of all convolution kernels in each layer of the original model includes:

[0011] The sum of the absolute values ​​of the weights of all the convolution kernels in each layer of the original model is calculated, and the sums of the absolute values ​​of the convolution kernels in each layer are sorted from large to small, and a preset number of convolution kernels at the front of the sorting result are retained.

[0012] Preferably, the original model includes two layers, and the training of the pruned model based on the first training sample set to obtain a pruned model includes:

[0013] When the first layer pruning operation of the original model is completed, the learning rate of the model after the first layer pruning operation is completed is adjusted to a preset interval, and the model after the adjusted learning rate is trained using a first preset number of training samples in the first training sample set to obtain a trained first pruning model;

[0014] Performing a second-layer pruning operation on the first pruning model, and training the model after the second-layer pruning operation using the first preset number of training samples to obtain a trained second pruning model;

[0015] The second pruning model is trained using a second preset number of training samples in the first training sample set to obtain the pruned model.

[0016] Preferably, the training process of the model after adjusting the learning rate includes:

[0017] Input each sample in the first training sample set into the model after adjusting the learning rate, to obtain a prediction result corresponding to each sample in the sample set;

[0018] Based on the labeling information of each sample in the sample set, the real result pre-labeled for each sample in the first training sample set is read.

[0019] The structural parameters of the model after adjusting the learning rate are determined by minimizing the loss value between the predicted results and the true results, and the trained model is obtained.

[0020] Preferably, performing mixed precision training on the pruned model based on the second training sample set includes:

[0021] Dividing the second training sample set into multiple batches, and converting the single-precision weights in the model obtained by training the previous batch of samples into half-precision weights;

[0022] Based on the half-precision weights and the network structure parameters of the model, the training samples of the current batch are predicted to obtain output data, and the single-precision loss value is determined according to the output data and the annotation information in the training samples;

[0023] Multiplying the single-precision loss by the proportion of the current batch of samples in the second training sample set to obtain a half-precision loss value, and determining a half-precision gradient value based on a back propagation algorithm and the half-precision loss value;

[0024] The half-precision gradient value is divided by the ratio to obtain a single-precision gradient value, and the single-precision weight is updated according to the single-precision gradient value.

[0025] Preferably, the identifying the object to be identified includes:

[0026] An object recognition request from a user is received, the object to be recognized carried in the request is input into the target compression model, a recognition result is obtained and the recognition result is stored in a preset database.

[0027] To achieve the above object, the present invention further provides an object recognition device based on edge computing, the object recognition device based on edge computing comprising:

[0028] Pruning module: used to obtain the original model to be processed, and perform a pruning operation on the original model based on the weights of all convolution kernels in each layer of the original model to obtain a pruned model;

[0029] Training module: used to train the pruned model based on the first training sample set to obtain a pruned model;

[0030] Identification module: used to perform mixed precision training on the pruned model based on the second training sample set, convert the model after mixed precision training into a binary model file of half-precision floating point numbers, and obtain a target compression model for the client to load the target compression model to identify the object to be identified.

[0031] To achieve the above object, the present invention further provides an electronic device, the electronic device comprising:

[0032] at least one processor; and,

[0033] a memory communicatively connected to the at least one processor; wherein,

[0034] The memory stores a program that can be executed by the at least one processor, and the program is executed by the at least one processor so that the at least one processor can perform any step of the edge computing-based object recognition method as described above.

[0035] To achieve the above-mentioned objectives, the present invention also provides a computer-readable storage medium, which stores an edge computing-based object recognition program. When the edge computing-based object recognition program is executed by a processor, any step of the edge computing-based object recognition method as described above is implemented.

[0036] The object recognition method, device, equipment and storage medium based on edge computing proposed in the present invention can perform pruning operations on the model deployed on the server side, reduce the redundant information of the model, and then train the model to restore its performance. Finally, the model after pruning and training is deployed to the user side, avoiding the need for multiple network transmissions when the user uses the model, which affects the timeliness of recognition, and also avoiding the risk of server downtime when a large number of users call the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 A flowchart diagram of a preferred embodiment of the object recognition method based on edge computing of the present invention;

[0038] Figure 2 A schematic diagram of a module of a preferred embodiment of an object recognition device based on edge computing of the present invention;

[0039] Figure 3 A schematic diagram of a preferred embodiment of the electronic device of the present invention;

[0040] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0041] In order to make the purpose, technical scheme and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0042] The present invention provides an object recognition method based on edge computing. Figure 1 FIG. 1 is a flow chart of an embodiment of an object recognition method based on edge computing of the present invention. The method can be performed by an electronic device, and the electronic device can be implemented by software and / or hardware. The object recognition method based on edge computing includes:

[0043] Step S10: obtaining an original model to be processed, and performing a pruning operation on the original model based on the weights of all convolution kernels in each layer of the original model to obtain a pruned model.

[0044] In this embodiment, the original model to be processed may refer to a large-scale machine learning model such as an image recognition model or an image detection model deployed on the server side, for example, a recognition model for identifying prescription documents, medical settlement documents or medical record documents in the medical field. Since the model deployed on the server side requires multiple data transmissions when recognizing an image, that is, the user uploads the image to be recognized to the image library through the client, and then the image is transmitted from the image library to the server side to call the interface of the image recognition model, and finally the recognition result is obtained and returned to the user and the database. If the machine learning model deployed on the server side is compressed and deployed to the user side, multiple data transmissions can be avoided, the timeliness of model recognition is increased, and the risk of server downtime is reduced.

[0045] Since the computing unit on the server side is the GPU, and the computing unit on the user side (such as the mobile terminal) is the ARM CPU, the computing power, hardware architecture and instruction set system of the two are different. Directly migrating the model from the server side to the user side code will not be able to run, and the model deployed on the server side often occupies a large amount of memory. If the model is directly migrated to the mobile side, the user side CPU is mostly unable to support the huge amount of computing, so the server side model needs to be compressed.

[0046] Specifically, the original model to be compressed (e.g., image recognition model) is obtained, and a pruning operation is performed on the model according to the weight of each layer of the original model to be compressed to obtain a pruned model. It can be understood that the original model to be compressed is a model that is pre-trained and deployed on the server side, and the model can be directly used to recognize images uploaded by the user side. Since there is often a lot of redundant information in the model, the redundant information of the model to be compressed can be deleted through the pruning operation.

[0047] In one embodiment, performing a pruning operation on the original model based on the weights of all convolution kernels in each layer of the original model includes:

[0048] The sum of the absolute values ​​of the weights of all convolution kernels in each layer of the original model is calculated, the convolution kernels whose absolute value sum is greater than a preset threshold are retained, and the convolution kernels whose absolute value is less than or equal to the preset threshold are discarded.

[0049] Traverse each layer of the model, find out the convolution kernels of each layer, set a threshold N, retain the convolution kernels whose sum of absolute weights is greater than N, discard the remaining convolution kernel parameters, and obtain a pruned model. The specific formula is as follows:

[0050]

[0051] Among them, M represents the retained convolution kernel, n represents the number of convolution kernel weights, N represents the preset threshold, and w irepresents the i-th weight of the convolution kernel. argmax represents the value that makes Get the variable point corresponding to the maximum value.

[0052] In one embodiment, performing a pruning operation on the original model based on the weights of all convolution kernels in each layer of the original model includes:

[0053] The sum of the absolute values ​​of the weights of all the convolution kernels in each layer of the original model is calculated, and the sums of the absolute values ​​of the convolution kernels in each layer are sorted from large to small, and a preset number of convolution kernels at the front of the sorting result are retained.

[0054] By sorting the absolute values ​​of each convolution kernel in each layer and retaining a fixed number of convolution kernels, the complexity of the model can be effectively reduced without causing too much loss in the recognition performance of the model.

[0055] Step S20: training the pruned model based on the first training sample set to obtain a pruned model.

[0056] In this embodiment, since the pruned model may discard some useful convolution kernel parameters while reducing redundant information, resulting in a certain loss of model performance, the pruned model can be trained using the first training sample set after the model to be compressed is pruned to maintain the accuracy of the model. Specifically, the learning rate of the pruned model is adjusted to between 0.01 and 0.1 of the model learning rate before the pruning operation is performed, and the model with the adjusted learning rate is trained based on the training sample set to obtain a model after pruning.

[0057] In one embodiment, the original model includes two layers, and the training of the pruned model based on the first training sample set to obtain a pruned model includes:

[0058] When the first layer pruning operation of the original model is completed, the learning rate of the model after the first layer pruning operation is completed is adjusted to a preset interval, and the model after the adjusted learning rate is trained using a first preset number of training samples in the first training sample set to obtain a trained first pruning model;

[0059] Performing a second-layer pruning operation on the first pruning model, and training the model after the second-layer pruning operation using the first preset number of training samples to obtain a trained second pruning model;

[0060] The second pruning model is trained using a second preset number of training samples in the first training sample set to obtain the pruned model.

[0061] After the first layer of the original model to be compressed is pruned, a first preset number (e.g., 5000) of training sample sets in the first training sample set are used to restore part of the model's performance through training to obtain a trained first pruned model, and then the second layer of the first pruned model is pruned, and the model after the second layer pruning is trained using the training samples to obtain a second pruned model. After pruning each layer of the model, only a small number of training samples are used to restore the model's performance, which can effectively prevent the model from entering a local optimum. Then, the second pruned model is trained using a second preset number (e.g., 30,000) of training samples to obtain a model after pruning.

[0062] It can be understood that if the original model has Q layers (Q is a positive integer greater than 2), it is also trained with a small amount of sample data after pruning each layer of the model. After the Q layers of the original model have been pruned and trained, the overall performance of the model is restored through iterations of more training samples.

[0063] Furthermore, the training process of the model after adjusting the learning rate includes:

[0064] Input each sample in the first training sample set into the model after adjusting the learning rate, to obtain a prediction result corresponding to each sample in the sample set;

[0065] Based on the labeling information of each sample in the sample set, the real result pre-labeled for each sample in the first training sample set is read.

[0066] The structural parameters of the model after adjusting the learning rate are determined by minimizing the loss value between the predicted results and the true results, and the trained model is obtained.

[0067] The annotation information of each sample in the sample set marks the true result of each sample. The pruned model is trained using the training sample set to restore the overall performance of the model.

[0068] Step S30: performing mixed precision training on the pruned model based on the second training sample set, converting the model after the mixed precision training into a binary model file of half-precision floating point numbers, and obtaining a target compression model, so that the client can load the target compression model to identify the object to be identified.

[0069] In this embodiment, the model after pruning is subjected to mixed precision training using the second training sample set. It can be understood that the first training sample set and the second training sample set are different training sets, which helps to improve the generalization ability of the model. The model after mixed precision training is converted into a binary model file of half-precision (FP16) floating point numbers using the onnx and ncnn frameworks so that the user end can load the target compression model. The target compression model refers to a model that can be deployed on the user end to recognize images input by the user. Since the GPU architecture on the server side and the various ARM CPU architectures on the mobile end support running C++ code, all Python codes of the server-side model are rewritten in C++, and then specific cross-compilation is performed for different ARM CPU architectures so that the model can run on most models. The code is encapsulated into an SDK on both Android and IOS ends, and the interface is exposed and delivered to the App end for use.

[0070] In one embodiment, performing mixed precision training on the pruned model based on the second training sample set includes:

[0071] Dividing the second training sample set into multiple batches, and converting the single-precision weights in the model obtained by training the previous batch of samples into half-precision weights;

[0072] Based on the half-precision weights and the network structure parameters of the model, the training samples of the current batch are predicted to obtain output data, and the single-precision loss value is determined according to the output data and the annotation information in the training samples;

[0073] Multiplying the single-precision loss by the proportion of the current batch of samples in the second training sample set to obtain a half-precision loss value, and determining a half-precision gradient value based on a back propagation algorithm and the half-precision loss value;

[0074] The half-precision gradient value is divided by the ratio to obtain a single-precision gradient value, and the single-precision weight is updated according to the single-precision gradient value.

[0075] Using half-precision FP16 floating point numbers is half the model size and memory usage of single-precision FP32 floating point numbers, but runs twice as fast as single-precision FP32 floating point numbers.

[0076] In one embodiment, identifying the object to be identified includes:

[0077] An object recognition request from a user is received, the object to be recognized carried in the request is input into the target compression model, a recognition result is obtained and the recognition result is stored in a preset database.

[0078] After the model is deployed to the user end, the model recognition can be completed directly on the user end, and the recognition results can be fed back to the user and stored in the database. There is no need to transfer the object to be recognized to the image library first and then to the server to call the model interface for recognition, which reduces network transmission and improves the timeliness of recognition.

[0079] Reference Figure 2 , which is a schematic diagram of the functional modules of the object recognition device 100 based on edge computing of the present invention.

[0080] The object recognition device 100 based on edge computing of the present invention can be installed in an electronic device. According to the functions implemented, the object recognition device 100 based on edge computing may include a pruning module 110, a training module 120 and a recognition module 130. The module described in the present invention may also be referred to as a unit, which refers to a series of computer program segments that can be executed by an electronic device processor and can complete fixed functions, which are stored in the memory of the electronic device.

[0081] In this embodiment, the functions of each module / unit are as follows:

[0082] The pruning module 110 is used to obtain the original model to be processed, and perform a pruning operation on the original model based on the weights of all convolution kernels in each layer of the original model to obtain a pruned model.

[0083] In this embodiment, the original model to be compressed (e.g., image recognition model) is obtained, and a pruning operation is performed on the model according to the weight of each layer of the original model to be compressed to obtain a pruned model. It can be understood that the original model to be compressed is a model that is pre-trained and deployed on the server side, and the model can be directly used to recognize images uploaded by the user side. Since there is often a lot of redundant information in the model, the redundant information of the model to be compressed can be deleted through the pruning operation.

[0084] In one embodiment, performing a pruning operation on the original model based on the weights of all convolution kernels in each layer of the original model includes:

[0085] The sum of the absolute values ​​of the weights of all convolution kernels in each layer of the original model is calculated, the convolution kernels whose absolute value sum is greater than a preset threshold are retained, and the convolution kernels whose absolute value is less than or equal to the preset threshold are discarded.

[0086] Traverse each layer of the model, find out the convolution kernels of each layer, set a threshold N, retain the convolution kernels whose sum of absolute weights is greater than N, discard the remaining convolution kernel parameters, and obtain a pruned model. The specific formula is as follows:

[0087]

[0088] Among them, M represents the retained convolution kernel, n represents the number of convolution kernel weights, N represents the preset threshold, and w i represents the i-th weight of the convolution kernel. argmax represents the value that makes Get the variable point corresponding to the maximum value.

[0089] In one embodiment, performing a pruning operation on the original model based on the weights of all convolution kernels in each layer of the original model includes:

[0090] The sum of the absolute values ​​of the weights of all the convolution kernels in each layer of the original model is calculated, and the sums of the absolute values ​​of the convolution kernels in each layer are sorted from large to small, and a preset number of convolution kernels at the front of the sorting result are retained.

[0091] By sorting the absolute values ​​of each convolution kernel in each layer and retaining a fixed number of convolution kernels, the complexity of the model can be effectively reduced without causing too much loss in the recognition performance of the model.

[0092] Training module 120: used to train the pruned model based on the first training sample set to obtain a pruned model.

[0093] In this embodiment, since the pruned model may discard some useful convolution kernel parameters while reducing redundant information, resulting in a certain loss of model performance, the pruned model can be trained using the training sample set after the model to be compressed is pruned to maintain the accuracy of the model. Specifically, the learning rate of the pruned model is adjusted to between 0.01 and 0.1 of the model learning rate before the pruning operation is performed, and the model with the adjusted learning rate is trained based on the training sample set to obtain a model after pruning.

[0094] In one embodiment, the original model includes two layers, and the training of the pruned model based on the first training sample set to obtain a pruned model includes:

[0095] When the first layer pruning operation of the original model is completed, the learning rate of the model after the first layer pruning operation is completed is adjusted to a preset interval, and the model after the adjusted learning rate is trained using a first preset number of training samples in the first training sample set to obtain a trained first pruning model;

[0096] Performing a second-layer pruning operation on the first pruning model, and training the model after the second-layer pruning operation using the first preset number of training samples to obtain a trained second pruning model;

[0097] The second pruning model is trained using a second preset number of training samples in the first training sample set to obtain the pruned model.

[0098] After the first layer of the original model to be compressed is pruned, a first preset number (e.g., 5000) of training sample sets in the first training sample set are used to restore part of the model's performance through training to obtain a trained first pruned model, and then the second layer of the first pruned model is pruned, and the model after the second layer pruning is trained using the training samples to obtain a second pruned model. After pruning each layer of the model, only a small number of training samples are used to restore the model's performance, which can effectively prevent the model from entering a local optimum. Then, the second pruned model is trained using a second preset number (e.g., 30,000) of training samples to obtain a model after pruning.

[0099] It can be understood that if the original model has Q layers (Q is a positive integer greater than 2), it is also trained with a small amount of sample data after pruning each layer of the model. After the Q layers of the original model have been pruned and trained, the overall performance of the model is restored through iterations of more training samples.

[0100] Furthermore, the training process of the model after adjusting the learning rate includes:

[0101] Input each sample in the first training sample set into the model after adjusting the learning rate, to obtain a prediction result corresponding to each sample in the sample set;

[0102] Based on the labeling information of each sample in the sample set, the real result pre-labeled for each sample in the first training sample set is read.

[0103] The structural parameters of the model after adjusting the learning rate are determined by minimizing the loss value between the predicted results and the true results, and the trained model is obtained.

[0104] The annotation information of each sample in the sample set marks the true result of each sample. The pruned model is trained using the training sample set to restore the overall performance of the model.

[0105] The recognition module 130 is used to perform mixed precision training on the pruned model based on the second training sample set, convert the model after the mixed precision training into a binary model file of half-precision floating point numbers, and obtain a target compression model for the client to load the target compression model to identify the object to be identified.

[0106] In this embodiment, the model after pruning is subjected to mixed precision training using the second training sample set. It can be understood that the first training sample set and the second training sample set are different training sets, which helps to improve the generalization ability of the model. The model after mixed precision training is converted into a binary model file of half-precision (FP16) floating point numbers using the onnx and ncnn frameworks so that the user end can load the target compression model. The target compression model refers to a model that can be deployed on the user end to recognize images input by the user. Since the GPU architecture on the server side and the various ARM CPU architectures on the mobile end support running C++ code, all Python codes of the server-side model are rewritten in C++, and then specific cross-compilation is performed for different ARM CPU architectures so that the model can run on most models. The code is encapsulated into an SDK on both Android and IOS ends, and the interface is exposed and delivered to the App end for use.

[0107] In one embodiment, performing mixed precision training on the pruned model based on the second training sample set includes:

[0108] Dividing the second training sample set into multiple batches, and converting the single-precision weights in the model obtained by training the previous batch of samples into half-precision weights;

[0109] Based on the half-precision weights and the network structure parameters of the model, the training samples of the current batch are predicted to obtain output data, and the single-precision loss value is determined according to the output data and the annotation information in the training samples;

[0110] Multiplying the single-precision loss by the proportion of the current batch of samples in the second training sample set to obtain a half-precision loss value, and determining a half-precision gradient value based on a back propagation algorithm and the half-precision loss value;

[0111] The half-precision gradient value is divided by the ratio to obtain a single-precision gradient value, and the single-precision weight is updated according to the single-precision gradient value.

[0112] Using half-precision FP16 floating point numbers is half the model size and memory usage of single-precision FP32 floating point numbers, but runs twice as fast as single-precision FP32 floating point numbers.

[0113] In one embodiment, identifying the object to be identified includes:

[0114] An object recognition request from a user is received, the object to be recognized carried in the request is input into the target compression model, a recognition result is obtained and the recognition result is stored in a preset database.

[0115] After the model is deployed to the user end, the model recognition can be completed directly on the user end, and the recognition results can be fed back to the user and stored in the database. There is no need to transfer the object to be recognized to the image library first and then to the server to call the model interface for recognition, which reduces network transmission and improves the timeliness of recognition.

[0116] Reference Figure 3 FIG. 1 is a schematic diagram of a preferred embodiment of an electronic device 1 of the present invention.

[0117] The electronic device 1 includes but is not limited to: a memory 11, a processor 12, a display 13 and a network interface 14. The electronic device 1 is connected to a network through the network interface 14 to obtain raw data. The network may be a wireless or wired network such as an intranet, the Internet, a global system of mobile communications (GSM), a wideband code division multiple access (WCDMA), a 4G network, a 5G network, Bluetooth, Wi-Fi, a call network, etc.

[0118] Among them, the memory 11 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, etc. In some embodiments, the memory 11 can be an internal storage unit of the electronic device 1, such as a hard disk or memory of the electronic device 1. In other embodiments, the memory 11 can also be an external storage device of the electronic device 1, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped with the electronic device 1. Of course, the memory 11 can also include both the internal storage unit of the electronic device 1 and its external storage device. In this embodiment, the memory 11 is generally used to store the operating system and various application software installed on the electronic device 1, such as the program code of the object recognition program 10 based on edge computing, etc. In addition, the memory 11 can also be used to temporarily store various types of data that have been output or are to be output.

[0119] In some embodiments, the processor 12 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chip. The processor 12 is generally used to control the overall operation of the electronic device 1, such as performing control and processing related to data interaction or communication. In this embodiment, the processor 12 is used to run the program code stored in the memory 11 or process data, such as running the program code of the object recognition program 10 based on edge computing.

[0120] The display 13 may be referred to as a display screen or a display unit. In some embodiments, the display 13 may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, and an organic light-emitting diode (OLED) touch device. The display 13 is used to display information processed in the electronic device 1 and to display a visual working interface, such as displaying the results of data statistics.

[0121] The network interface 14 may optionally include a standard wired interface, a wireless interface (such as a WI-FI interface), and the network interface 14 is generally used to establish a communication connection between the electronic device 1 and other electronic devices.

[0122] Figure 3 Only the electronic device 1 having components 11 - 14 and the edge computing-based object recognition program 10 is shown, but it should be understood that it is not required to implement all the components shown, and more or fewer components may be implemented instead.

[0123] Optionally, the electronic device 1 may further include a user interface, which may include a display (Display), an input unit such as a keyboard (Keyboard), and the optional user interface may also include a standard wired interface and a wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, and an organic light-emitting diode (Organic Light-Emitting Diode, OLED) touch device, etc. Among them, the display may also be appropriately referred to as a display screen or a display unit, which is used to display information processed in the electronic device 1 and to display a visual user interface.

[0124] The electronic device 1 may further include a radio frequency (RF) circuit, a sensor, an audio circuit, etc., which will not be described in detail here.

[0125] In the above embodiment, the processor 12 may implement the following steps when executing the edge computing-based object recognition program 10 stored in the memory 11:

[0126] Obtaining an original model to be processed, and performing a pruning operation on the original model based on the weights of all convolution kernels in each layer of the original model to obtain a pruned model;

[0127] Training the pruned model based on the first training sample set to obtain a pruned model;

[0128] Based on the second training sample set, mixed precision training is performed on the pruned model, and the model after mixed precision training is converted into a binary model file of half-precision floating point numbers to obtain a target compression model, so that the client can load the target compression model to identify the object to be identified.

[0129] The storage device may be the memory 11 of the electronic device 1 , or may be another storage device that is communicatively connected to the electronic device 1 .

[0130] For a detailed description of the above steps, please refer to the above Figure 2 Functional module diagram of an embodiment of an object recognition device 100 based on edge computing and Figure 1 Description of a flowchart of an embodiment of an object recognition method based on edge computing.

[0131] In addition, an embodiment of the present invention further proposes a computer-readable storage medium, which may be non-volatile or volatile. The computer-readable storage medium may be any one or any combination of a hard disk, a multimedia card, an SD card, a flash memory card, an SMC, a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a portable compact disk read-only memory (CD-ROM), a USB memory, etc. The computer-readable storage medium includes a data storage area and a program storage area, the data storage area stores data created according to the use of the blockchain node, and the program storage area stores an object recognition program 10 based on edge computing, and the object recognition program 10 based on edge computing is executed by the processor to implement the following operations:

[0132] Obtaining an original model to be processed, and performing a pruning operation on the original model based on the weights of all convolution kernels in each layer of the original model to obtain a pruned model;

[0133] Training the pruned model based on the first training sample set to obtain a pruned model;

[0134] Based on the second training sample set, mixed precision training is performed on the pruned model, and the model after mixed precision training is converted into a binary model file of half-precision floating point numbers to obtain a target compression model, so that the client can load the target compression model to identify the object to be identified.

[0135] The specific implementation of the computer-readable storage medium of the present invention is roughly the same as the specific implementation of the above-mentioned edge computing-based object recognition method, and will not be repeated here.

[0136] The present invention can acquire and process relevant data based on artificial intelligence technology. Artificial Intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0137] AI basic technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics, etc. AI software technologies mainly include computer vision technology, robotics technology, biometrics technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0138] In another embodiment, the object recognition method based on edge computing provided by the present invention can further ensure the privacy and security of all the above data, and all the above data can also be stored in a blockchain node. For example, the target compression model and recognition results, etc., can all be stored in the blockchain node.

[0139] It should be noted that the blockchain referred to in the present invention is a new application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanism, encryption algorithm, etc. Blockchain is essentially a decentralized database, a string of data blocks generated by cryptographic methods. Each data block contains a batch of network transaction information, which is used to verify the validity of its information (anti-counterfeiting) and generate the next block. Blockchain can include the blockchain underlying platform, platform product service layer, and application service layer.

[0140] It should be noted that the serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages and disadvantages of the embodiments. And the terms "including", "comprising" or any other variants thereof in this article are intended to cover non-exclusive inclusion, so that a process, device, article or method including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, device, article or method. In the absence of further restrictions, an element defined by the sentence "including a ..." does not exclude the presence of other identical elements in the process, device, article or method including the element.

[0141] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, including a number of instructions for enabling a terminal device (which can be a mobile phone, a computer, an electronic device, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0142] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. An object recognition method based on edge computing, characterized in that: The method comprises: Obtain an original model to be processed, where the original model to be processed is an image processing machine learning model deployed on a server side, and perform a pruning operation on the original model based on the weights of all convolution kernels in each layer of the original model to obtain a pruned model, wherein the pruning operation on the original model based on the weights of all convolution kernels in each layer of the original model includes: calculating the sum of the absolute values ​​of the weights of all convolution kernels in each layer of the original model, retaining convolution kernels whose absolute value sums are greater than a preset threshold, and discarding convolution kernels whose absolute values ​​are less than or equal to the preset threshold, or calculating the sum of the absolute values ​​of the weights of all convolution kernels in each layer of the original model, and sorting the sums of the absolute values ​​of each convolution kernel in each layer from large to small, and retaining a preset number of convolution kernels at the front of the sorting result; Adjusting the learning rate of the pruned model to a preset numerical range of the learning rate of the original model, training the pruned model based on the first training sample set, and obtaining a pruned model; Based on the second training sample set, mixed precision training is performed on the pruned model, and the model after mixed precision training is converted into a binary model file of half-precision floating point numbers to obtain a target compression model, so that the client can load the target compression model to identify the object to be identified.

2. The object recognition method based on edge computing according to claim 1, characterized in that: The original model includes two layers, and the pruned model is trained based on the first training sample set to obtain a pruned model, including: When the first layer pruning operation of the original model is completed, the learning rate of the model after the first layer pruning operation is completed is adjusted to a preset numerical range of the learning rate of the original model, and the model after the adjusted learning rate is trained using a first preset number of training samples in the first training sample set to obtain a trained first pruning model; Performing a second-layer pruning operation on the first pruning model, and training the model after the second-layer pruning operation using the first preset number of training samples to obtain a trained second pruning model; The second pruning model is trained using a second preset number of training samples in the first training sample set to obtain the pruned model.

3. The object recognition method based on edge computing according to claim 1, characterized in that: The step of training the pruned model based on the first training sample set to obtain a pruned model includes: Input each sample in the first training sample set into the model after adjusting the learning rate, to obtain a prediction result corresponding to each sample in the sample set; Reading the real result pre-labeled for each sample in the first training sample set based on the labeling information of each sample in the sample set; The structural parameters of the model after adjusting the learning rate are determined by minimizing the loss value between the predicted results and the true results, and the trained model is obtained.

4. The object recognition method based on edge computing according to claim 1, characterized in that: The performing mixed precision training on the pruned model based on the second training sample set includes: Dividing the second training sample set into multiple batches, and converting the single-precision weights in the model obtained by training the previous batch of samples into half-precision weights; Based on the half-precision weights and the network structure parameters of the model, the training samples of the current batch are predicted to obtain output data, and the single-precision loss value is determined according to the output data and the annotation information in the training samples; Multiplying the single-precision loss by the proportion of the current batch of samples in the second training sample set to obtain a half-precision loss value, and determining a half-precision gradient value based on a back propagation algorithm and the half-precision loss value; The half-precision gradient value is divided by the ratio to obtain a single-precision gradient value, and the single-precision weight is updated according to the single-precision gradient value.

5. The object recognition method based on edge computing according to any one of claims 1 to 4, characterized in that: The identifying the object to be identified includes: An object recognition request from a user is received, the object to be recognized carried in the request is input into the target compression model, a recognition result is obtained and the recognition result is stored in a preset database.

6. An object recognition device based on edge computing, characterized in that: The device comprises: Pruning module: used to obtain the original model to be processed, which is an image processing machine learning model deployed on the server side, and perform a pruning operation on the original model based on the weights of all convolution kernels in each layer of the original model to obtain a pruned model, wherein the pruning operation on the original model based on the weights of all convolution kernels in each layer of the original model includes: calculating the sum of the absolute values ​​of the weights of all convolution kernels in each layer of the original model, retaining the convolution kernels whose absolute value sum is greater than a preset threshold, and discarding the convolution kernels whose absolute value is less than or equal to the preset threshold, or calculating the sum of the absolute values ​​of the weights of all convolution kernels in each layer of the original model, and sorting the sums of the absolute values ​​of each convolution kernel in each layer from large to small, and retaining a preset number of convolution kernels at the front of the sorting result; Training module: used for adjusting the learning rate of the pruned model to a preset numerical range of the learning rate of the original model, training the pruned model based on the first training sample set, and obtaining a pruned model; Identification module: used to perform mixed precision training on the pruned model based on the second training sample set, convert the model after mixed precision training into a binary model file of half-precision floating point numbers, and obtain a target compression model for the client to load the target compression model to identify the object to be identified.

7. An electronic device, characterized in that: The electronic device comprises: at least one processor; and, a memory communicatively connected to the at least one processor; wherein, The memory stores a program that can be executed by the at least one processor, and the program is executed by the at least one processor so that the at least one processor can execute the object recognition method based on edge computing as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores an edge computing-based object recognition program, and when the edge computing-based object recognition program is executed by a processor, the steps of the edge computing-based object recognition method as described in any one of claims 1 to 5 are implemented.

Images