Image recognition method and apparatus
By jointly training the YOLOv7-x and YOLOv7-E6E models and optimizing the network layers of YOLOv7-x, the problem of deep learning models being difficult to run efficiently on small devices is solved, and efficient image recognition is achieved on handheld and portable devices.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- SHENHUA BAOSHEN RAILWAY GRP
- Filing Date
- 2022-12-22
- Publication Date
- 2026-07-10
AI Technical Summary
In existing technologies, deep learning models are large in scale, making them difficult to run efficiently on handheld and portable devices.
The YOLOv7-x recognition model and the high-precision but low-efficiency YOLOv7-E6E model are jointly trained. The original image is converted from the RGB color space to the LAB color space, and the network layers of the YOLOv7-x model are optimized to reduce the model complexity and adapt it to small devices.
While ensuring recognition accuracy, recognition efficiency has been improved, enabling image recognition to operate efficiently on handheld and portable devices.
Smart Images
Figure CN115965852B_ABST
Abstract
Description
Technical Field
[0001] This disclosure relates to the field of image recognition technology, and in particular to image recognition methods and apparatus. Background Technology
[0002] The latest and most efficient method for analyzing contact wire inspection images is to use deep learning technology to identify and locate components in contact wire inspection images. However, traditional deep learning technology uses recognition models that are generally large in scale to ensure model generalization ability, with model parameters reaching tens of millions to hundreds of millions. In order to load and compute the inference model, the required computing equipment is large in scale, making it difficult to run efficiently on small devices such as handheld devices and portable devices. Summary of the Invention
[0003] In view of this, the present disclosure provides an image recognition method and apparatus to solve the problem that the large size of the recognition model in the prior art makes it difficult to operate efficiently on small devices such as handheld devices and portable devices.
[0004] A first aspect of this disclosure provides an image recognition method, comprising:
[0005] Obtain the original image containing the target to be identified;
[0006] The original image is converted from the RGB color space to the LAB color space to obtain a LAB image;
[0007] The LAB image is imported into the trained YOLOv7-x recognition model for recognition, and a recognition result is obtained for the target to be recognized. The YOLOv7-x recognition model is determined by training based on a preset combination of YOLOv7-E6E recognition models. The model complexity of the YOLOv7-E6E recognition model is higher than that of the YOLOv7-x recognition model.
[0008] In some embodiments, the step of determining the yolov7-x recognition model by training a preset yolov7-E6E recognition model combination includes:
[0009] Obtain a training image set, which includes multiple training images containing the target to be identified;
[0010] The training image set is imported into the YOLOv7-E6E recognition model to obtain the baseline recognition results;
[0011] The training image set is imported into the untrained YOLOv7-x recognition model and the combined model consisting of the YOLOv7-x recognition model and the YOLOv7-E6E recognition model for iterative training until the combined loss data of the combined model meets the preset requirements, thus obtaining the trained YOLOv7-x recognition model.
[0012] In some embodiments, the combined loss data is the output of the last layer of the yolov7-x recognition model and the yolov7-E6E recognition model, the combined recognition result after weighting based on preset weighting parameters, and the difference between the combined recognition result and the benchmark recognition result.
[0013] In some embodiments, the step of iteratively training the training image set by importing it into an untrained YOLOv7-x recognition model and a combined model consisting of the YOLOv7-x recognition model and the YOLOv7-E6E recognition model, until the combined loss data of the combined model meets preset requirements, thereby obtaining the trained YOLOv7-x recognition model, includes:
[0014] When the current training is completed, the ratio of each data point in the parameter matrix of the current YOLOv7-x recognition model to the corresponding data point in the parameter matrix of the YOLOv7-x recognition model when the previous training was completed is updated to the preset update rate matrix.
[0015] If the improvement rate of both the target loss data of the YOLOv7-x recognition model and the combined loss data of the current time is greater than a preset first threshold, the parameter matrix of the YOLOv7-x recognition model of the current time will be updated. Here, the target loss data refers to the difference between the recognition result of the YOLOv7-x recognition model and the benchmark recognition result.
[0016] The YOLOv7-x recognition model is optimized based on the update rate matrix.
[0017] If the optimization is successful, the optimized yolov7-x recognition model will be updated to the current yolov7-x recognition model.
[0018] Repeat the training until the combined loss data of the combined model meets the preset requirements, and then determine the current YOLOv7-x recognition model as the trained YOLOv7-x recognition model.
[0019] In some embodiments, optimizing the YOLOv7-x recognition model based on the update rate matrix includes:
[0020] Optimized update rate sets with values less than a preset second threshold are selected from the update rate matrix;
[0021] Randomly delete the connection branches in the network layer of the YOLOv7-x recognition model corresponding to a first preset number of update rates in the optimized update rate set, wherein the first preset number is a positive number;
[0022] If the difference between the target loss data of the YOLOv7-x recognition model after randomly deleting connection branches and the target loss data before randomly deleting connection branches is less than the preset third threshold, the optimization is considered successful.
[0023] Otherwise, after reducing the first preset quantity by a preset pruning step size, the step of randomly deleting the connection branches in the network layer of the YOLOv7-x recognition model corresponding to the first preset quantity update rate in the optimized update rate set is repeated until optimization is successful, or optimization fails when the first preset quantity is less than or equal to the pruning step size, wherein the pruning step size is a positive number less than the initial first preset quantity.
[0024] In some embodiments, the step of updating the optimized YOLOv7-x recognition model to the current YOLOv7-x recognition model if the optimization is successful includes:
[0025] If the optimization fails, the YOLOv7-x recognition model will not be optimized for that iteration.
[0026] In some embodiments, the combined loss data of the combined model meeting the preset requirements means that in the most recent second preset number of training sessions, the variance of the changes in the combined loss data is less than a preset fourth threshold.
[0027] A second aspect of this disclosure provides an image recognition device, comprising:
[0028] The acquisition module is used to acquire the original image containing the target to be identified;
[0029] The conversion module is used to convert the original image from the RGB color gamut space to the LAB color gamut space to obtain a LAB image;
[0030] The recognition module is used to import the LAB image into a trained yolov7-x recognition model for recognition, and obtain the recognition result for the target to be recognized. The yolov7-x recognition model is determined by training based on a preset yolov7-E6E recognition model combination. The model complexity of the yolov7-E6E recognition model is higher than that of the yolov7-x recognition model.
[0031] In some embodiments, the step of determining the yolov7-x recognition model by training a preset yolov7-E6E recognition model combination includes:
[0032] Obtain a training image set, which includes multiple training images containing the target to be identified;
[0033] The training image set is imported into the YOLOv7-E6E recognition model to obtain the baseline recognition results;
[0034] The training image set is imported into the untrained YOLOv7-x recognition model and the combined model consisting of the YOLOv7-x recognition model and the YOLOv7-E6E recognition model for iterative training until the combined loss data of the combined model meets the preset requirements, thus obtaining the trained YOLOv7-x recognition model.
[0035] In some embodiments, the combined loss data is the output of the last layer of the yolov7-x recognition model and the yolov7-E6E recognition model, the combined recognition result after weighting based on preset weighting parameters, and the difference between the combined recognition result and the benchmark recognition result.
[0036] In some embodiments, the step of iteratively training the training image set by importing it into an untrained YOLOv7-x recognition model and a combined model consisting of the YOLOv7-x recognition model and the YOLOv7-E6E recognition model, until the combined loss data of the combined model meets preset requirements, thereby obtaining the trained YOLOv7-x recognition model, includes:
[0037] When the current training is completed, the ratio of each data point in the parameter matrix of the current YOLOv7-x recognition model to the corresponding data point in the parameter matrix of the YOLOv7-x recognition model when the previous training was completed is updated to the preset update rate matrix.
[0038] If the improvement rate of both the target loss data of the YOLOv7-x recognition model and the combined loss data of the current time is greater than a preset first threshold, the parameter matrix of the YOLOv7-x recognition model of the current time will be updated. Here, the target loss data refers to the difference between the recognition result of the YOLOv7-x recognition model and the benchmark recognition result.
[0039] The YOLOv7-x recognition model is optimized based on the update rate matrix.
[0040] If the optimization is successful, the optimized yolov7-x recognition model will be updated to the current yolov7-x recognition model.
[0041] Repeat the training until the combined loss data of the combined model meets the preset requirements, and then determine the current YOLOv7-x recognition model as the trained YOLOv7-x recognition model.
[0042] In some embodiments, optimizing the YOLOv7-x recognition model based on the update rate matrix includes:
[0043] Optimized update rate sets with values less than a preset second threshold are selected from the update rate matrix;
[0044] Randomly delete the connection branches in the network layer of the YOLOv7-x recognition model corresponding to a first preset number of update rates in the optimized update rate set, wherein the first preset number is a positive number;
[0045] If the difference between the target loss data of the YOLOv7-x recognition model after randomly deleting connection branches and the target loss data before randomly deleting connection branches is less than the preset third threshold, the optimization is considered successful.
[0046] Otherwise, after reducing the first preset quantity by a preset pruning step size, the step of randomly deleting the connection branches in the network layer of the YOLOv7-x recognition model corresponding to the first preset quantity update rate in the optimized update rate set is repeated until optimization is successful, or optimization fails when the first preset quantity is less than or equal to the pruning step size, wherein the pruning step size is a positive number less than the initial first preset quantity.
[0047] In some embodiments, the step of updating the optimized YOLOv7-x recognition model to the current YOLOv7-x recognition model if the optimization is successful includes:
[0048] If the optimization fails, the YOLOv7-x recognition model will not be optimized for that iteration.
[0049] In some embodiments, the combined loss data of the combined model meeting the preset requirements means that in the most recent second preset number of training sessions, the variance of the changes in the combined loss data is less than a preset fourth threshold.
[0050] A third aspect of this disclosure provides an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the method described above.
[0051] A fourth aspect of this disclosure provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the above-described method.
[0052] Beneficial effects
[0053] The beneficial effects of the embodiments disclosed herein compared with the prior art include at least the following: by jointly training the low-precision, high-efficiency yolov7-x recognition model and the high-precision, low-efficiency yolov7-E6E model, a balance between speed and efficiency is achieved, and the recognition accuracy is greatly improved while reducing the size of the recognition model, so that it can be run efficiently on small devices such as handheld devices and portable devices while meeting the accuracy requirements. Attached Figure Description
[0054] To more clearly illustrate the technical solutions in the embodiments of this disclosure, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0055] Figure 1 This is a schematic diagram of an application scenario of the image recognition method provided according to the embodiments of this disclosure;
[0056] Figure 2 This is a flowchart of some embodiments of an image recognition method provided according to the present disclosure;
[0057] Figure 3 This is a flowchart of some other embodiments of another image recognition method provided according to the embodiments of this disclosure;
[0058] Figure 4 This is a simplified structural diagram of an image recognition device provided according to an embodiment of the present disclosure;
[0059] Figure 5 This is a schematic diagram of an electronic device provided according to an embodiment of the present disclosure. Detailed Implementation
[0060] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.
[0061] It should also be noted that, for ease of description, only the parts relevant to this disclosure are shown in the accompanying drawings. Unless otherwise specified, the embodiments and features described in this disclosure can be combined with each other.
[0062] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are used only to distinguish different systems, devices, modules or units, and are not used to limit the order of functions performed by these systems, devices, modules or units or their interdependencies.
[0063] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".
[0064] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.
[0065] This disclosure will now be described in detail with reference to the accompanying drawings and embodiments.
[0066] Figure 1 This is a schematic diagram of an application scenario of an image recognition method according to some embodiments of the present disclosure.
[0067] exist Figure 1 In the application scenario, firstly, the computing device 101 can acquire the original image 102 containing the target to be identified.
[0068] Secondly, the computing device 101 can convert the original image 102 from the RGB color gamut space to the LAB color gamut space to obtain the LAB image 103.
[0069] Finally, the computing device 101 can import the LAB image 103 into the trained yolov7-x recognition model 104 for recognition, and obtain the recognition result 105 for the target to be recognized. The yolov7-x recognition model is determined by training based on a preset yolov7-E6E recognition model combination, and the model complexity of the yolov7-E6E recognition model is higher than that of the yolov7-x recognition model.
[0070] It should be noted that the aforementioned computing device 101 can be either hardware or software. When the computing device is hardware, it can be implemented as a distributed cluster consisting of multiple servers or terminal devices, or as a single server or a single terminal device. When the computing device is software, it can be installed within the hardware devices listed above. It can be implemented as, for example, multiple software programs or software modules used to provide distributed services, or as a single software program or software module. No specific limitations are made here.
[0071] It should be understood that Figure 1 The number of computing devices shown is merely illustrative. Any number of computing devices can be used depending on implementation needs.
[0072] Continue to refer to Figure 2 The diagram illustrates a flow 200 of some embodiments of the image recognition method according to this disclosure. This method can be... Figure 1 The image recognition method is performed by a computing device 101. The method includes the following steps:
[0073] Step 201: Obtain the original image containing the target to be identified.
[0074] In some embodiments, the entity executing the image recognition method (e.g. Figure 1 The computing device 101 shown can be connected to the target device via a wired or wireless connection, and then acquire the original image containing the target to be identified.
[0075] The original image can refer to an image containing the target to be identified (such as an insulator). This original image can be captured by other devices and transmitted in real time. For example, the target to be identified can be an insulator, and the original image can be an image obtained by a railway-specific inspection vehicle imaging a high-speed railway catenary support device (the image contains the insulator). Alternatively, the original image can also be an image obtained from a pre-defined storage structure (such as a database). The original image can also be from any other source; no specific limitations are imposed here.
[0076] It should be noted that the aforementioned wireless connection methods may include, but are not limited to, 3G / 4G / 5G connections, WiFi connections, Bluetooth connections, WiMAX connections, Zigbee connections, UWB (ultra wideband) connections, and other currently known or future wireless connection methods.
[0077] Step 202: Convert the original image from the RGB color gamut space to the LAB color gamut space to obtain a LAB image.
[0078] In some embodiments, the aforementioned executing entity can convert the original image from the RGB color gamut space to the LAB color gamut space to obtain a LAB image.
[0079] Because the Lab color gamut has a wide color range, encompassing not only all the color ranges of RGB and CMY, but also colors that they cannot represent, and because it is about as fast as RGB but much faster than CMYK when working in the Lab color gamut, converting to the LAB color gamut can significantly improve the color quality of an image.
[0080] Step 203: Import the LAB image into the trained yolov7-x recognition model for recognition to obtain the recognition result for the target to be recognized. The yolov7-x recognition model is determined by training based on a preset combination of yolov7-E6E recognition models. The model complexity of the yolov7-E6E recognition model is higher than that of the yolov7-x recognition model.
[0081] In some embodiments, the aforementioned execution entity can import the LAB image into a trained YOLOv7-x recognition model for recognition to obtain a recognition result for the target to be recognized. The YOLOv7-x recognition model is determined by training based on a preset combination of YOLOv7-E6E recognition models, and the model complexity of the YOLOv7-E6E recognition model is higher than that of the YOLOv7-x recognition model.
[0082] YOLOv7-x and YOLOv7-E6E are obtained by transforming the YOLOv7 model into different modes. YOLOv7 is an object detection model that supports both mobile GPUs and GPU devices from the edge to the cloud. This model is existing technology and will not be discussed in detail here.
[0083] It should be noted that the YOLOv7-E6E model has high accuracy but low recognition efficiency, while YOLOv7-x has lower accuracy but higher efficiency. Therefore, this disclosure optimizes YOLOv7-x to improve recognition accuracy while maintaining recognition efficiency. The pre-set YOLOv7-E6E recognition model refers to a pre-trained model capable of recognizing the target from an image. Combined training refers to combining YOLOv7-E6E and YOLOv7-x for integrated training.
[0084] In some embodiments, the step of determining the YOLOv7-x recognition model by training a combination of preset YOLOv7-E6E recognition models includes: acquiring a training image set, which includes multiple training images containing the target to be recognized; importing the training image set into the YOLOv7-E6E recognition model to obtain a baseline recognition result; and iteratively training the training image set into an untrained YOLOv7-x recognition model and a combined model consisting of the YOLOv7-x recognition model and the YOLOv7-E6E recognition model, respectively, until the combined loss data of the combined model meets preset requirements, thereby obtaining the trained YOLOv7-x recognition model.
[0085] Training images can refer to images that are determined to contain the target to be identified. An untrained YOLOv7-x recognition model can refer to a YOLOv7-x recognition model whose parameters have not been optimized through training; the recognition accuracy of an untrained YOLOv7-x recognition model is obviously low. Since the YOLOv7-E6E recognition model has high recognition accuracy, its recognition results are used as a benchmark. When the recognition accuracy of the YOLOv7-x recognition model is close to that of the YOLOv7-E6E recognition model, or meets a certain proportion of the accuracy of the YOLOv7-E6E recognition model, the YOLOv7-x recognition model can be considered successfully trained. Combined loss data can refer to a single value, a combination of multiple data points, a data matrix, or other data formats; no specific restrictions are placed here.
[0086] In some optional implementations of some embodiments, the combined model loss data meeting the preset requirements means that in the most recent second preset number of training iterations, the variance of the changes in the combined loss data is less than a preset fourth threshold.
[0087] The second preset quantity can refer to a pre-set limit value for determining the number of training iterations. The fourth threshold can refer to a limit value used to restrict the variance. As an example, "the combined model's combined loss data meets the preset requirements" can mean that training ends when the variance of the combined loss data change is less than 10 in the last 20 training iterations, at which point the model training is considered to have reached stability. Alternatively, "the combined model's combined loss data meets the preset requirements" can also be other commonly used stopping indicators, set as needed, without specific limitations here.
[0088] In some embodiments, the combined loss data is the output of the last layer of the yolov7-x recognition model and the yolov7-E6E recognition model, the combined recognition result after weighting based on preset weighting parameters, and the difference between the combined recognition result and the benchmark recognition result.
[0089] The preset weighting parameter refers to the parameter used to limit the proportion of recognition results from the YOLOv7-x recognition model and the YOLOv7-E6E recognition model. Obviously, the higher the proportion of the YOLOv7-x recognition model's results, the lower the recognition accuracy; conversely, the lower the proportion of the YOLOv7-x recognition model's results, the higher the recognition accuracy. This weighting parameter can refer to two positive numbers that sum to 1. As an example, the weighting parameter for the YOLOv7-x recognition model can be 0.3, then the weighting parameter for the YOLOv7-E6E recognition model is 1 - 0.3 = 0.7. This weighting parameter can be a manually specified value or a value obtained through multiple training iterations; no specific restrictions are placed here.
[0090] In some embodiments, the step of iteratively training the training image set into an untrained YOLOv7-x recognition model and a combined model consisting of the YOLOv7-x recognition model and the YOLOv7-E6E recognition model until the combined loss data of the combined model meets preset requirements, thereby obtaining the trained YOLOv7-x recognition model, includes: upon completion of the current training iteration, updating a preset update rate matrix with the ratio of each data point in the parameter matrix of the current YOLOv7-x recognition model to the corresponding data point in the parameter matrix of the YOLOv7-x recognition model at the end of the previous training iteration; if the current YOLOv7-x recognition model... If the improvement rate of both the target loss data and the combined loss data in the current iteration is greater than a preset first threshold, the parameter matrix of the YOLOv7-x recognition model for that iteration is updated. Here, the target loss data refers to the difference between the recognition result of the YOLOv7-x recognition model and the baseline recognition result. The YOLOv7-x recognition model is then optimized based on the update rate matrix. If the optimization is successful, the optimized YOLOv7-x recognition model is updated to the current YOLOv7-x recognition model. Training is repeated until the combined loss data of the combined model meets preset requirements, at which point the current YOLOv7-x recognition model is determined as the trained YOLOv7-x recognition model.
[0091] The parameter matrix can refer to the matrix composed of the various parameters in the YOLOv7-x recognition model. Since the YOLOv7-x recognition model includes multiple computational layers, each layer has multiple computational nodes, and each node corresponds to a computational parameter (or weighting coefficient), these multiple computational parameters can form the parameter matrix. The update rate matrix has the same structure as the parameter matrix, but each data point in the update rate matrix is the ratio of each data point in the parameter matrix of the current YOLOv7-x recognition model to the corresponding data point in the parameter matrix of the YOLOv7-x recognition model at the end of the previous training. The first threshold can refer to a limit value used to constrain the improvement rate of the target loss data. The improvement rate can refer to the degree to which the target loss data decreases. As an example, if the target loss data after the nth training iteration is 0.3, and the target loss data after the (n+1)th training iteration is 0.25, then 0.25 / 0.3 = 8.33%, and the improvement rate of the target loss data after the (n+1)th training iteration is 8.33%.
[0092] In some alternative implementations of some embodiments, the first threshold is 20%.
[0093] It should be noted that updating the parameter matrix of the YOLOv7-x recognition model for this training iteration means updating and saving the parameters of the YOLOv7-x recognition model after this training. In other words, if the improvement rate of the target loss data after this training is not greater than the first threshold, the parameters of the YOLOv7-x recognition model will not be updated. Only if the improvement rate of the target loss data after this training is greater than the first threshold will the parameters of the YOLOv7-x recognition model not be updated.
[0094] Furthermore, the target loss data refers to the difference between the recognition result of the YOLOv7-x recognition model and the baseline recognition result. This difference can be an exponential value, a percentage of the difference, or other data formats for the difference; no specific restrictions are imposed here.
[0095] Optimizing the YOLOv7-x recognition model means attempting to remove computational nodes (or connection branches) that perform poorly in the YOLOv7-x recognition model.
[0096] In some optional implementations of certain embodiments, optimizing the YOLOv7-x recognition model based on the update rate matrix includes:
[0097] The first step is to select an optimized update rate set from the update rate matrix that is less than a preset second threshold.
[0098] The second threshold can refer to a limit value used to filter out computational nodes (or connection branches) with smaller impact factors. As an example, this second threshold could be 40%.
[0099] The second step is to randomly delete the connection branches in the network layer of the YOLOv7-x recognition model corresponding to the first preset number of update rates in the optimized update rate set, wherein the first preset number is a positive number.
[0100] The first preset quantity can refer to the number of items to be randomly deleted initially. This first preset quantity can be a specific numerical value or a percentage. For example, the first preset quantity could be 30%.
[0101] The third step involves training the YOLOv7-x recognition model after randomly deleting connection branches. If the difference between the target loss data of the YOLOv7-x recognition model after randomly deleting connection branches and the target loss data before randomly deleting connection branches is less than a preset third threshold, the optimization is successful. Otherwise, the first preset number is reduced by a preset pruning step size, and the step of randomly deleting connection branches in the network layer of the YOLOv7-x recognition model corresponding to the first preset number of update rates in the optimization update rate set is repeated until the optimization is successful, or the first preset number is less than or equal to the pruning step size, indicating that the optimization has failed. The pruning step size is a positive number less than the initial first preset number.
[0102] The third threshold can refer to a limit value for the difference in target loss data. If the difference between the target loss data of the YOLOv7-x recognition model after randomly deleting connection branches and the target loss data before randomly deleting connection branches is less than the preset third threshold, it indicates that the deletion has little impact on recognition accuracy, and the optimization is successful. Otherwise, the first preset number is reduced by a preset step size, and the model is retrained and the difference in target loss data before and after randomly deleting connection branches is calculated. The pruning step size can refer to a preset value used to reduce the first preset number, which is a positive number less than the first preset number.
[0103] As a concrete example, firstly, network connection branches with less than 40% of the update rate matrix are selected to establish an optimization candidate set. Then, connection branches corresponding to the data in the optimization candidate set are randomly deleted at a rate of 30%. The difference in target loss data of the YOLOv7-x recognition model after deletion is judged. If the difference is less than 10%, the optimization is considered effective; otherwise, the above steps are repeated after 40%-5% deletion, until the first preset number is less than or equal to the pruning step size. At this point, the optimization is considered to have failed, and no further branch deletion is performed. By randomly deleting some computational nodes (or connection branches) with less impact, the size of the YOLOv7-x recognition model can be further reduced, thereby improving the computational efficiency of the YOLOv7-x recognition model.
[0104] The beneficial effects of one of the above embodiments of this disclosure include at least the following: by jointly training a low-precision, high-efficiency yolov7-x recognition model and a high-precision, low-efficiency yolov7-E6E model, a balance between speed and efficiency is achieved, and the recognition accuracy is greatly improved while reducing the size of the recognition model, so that it can run efficiently on small devices such as handheld devices and portable devices while meeting the accuracy requirements.
[0105] Continue to refer to Figure 3 The flowchart 300 illustrates another embodiment of the image recognition method according to the present disclosure, which can be performed by... Figure 1 The image recognition method is executed by the computing device 101.
[0106] Step 301: Obtain the original image containing the target to be identified.
[0107] Step 302: Convert the original image from the RGB color gamut space to the LAB color gamut space to obtain a LAB image.
[0108] Step 303: Import the LAB image into the trained yolov7-x recognition model for recognition to obtain the recognition result for the target to be recognized. The yolov7-x recognition model is determined by training based on a preset combination of yolov7-E6E recognition models. The model complexity of the yolov7-E6E recognition model is higher than that of the yolov7-x recognition model.
[0109] Step 304, and the following steps 305-311, represent the steps of determining the yolov7-x recognition model based on the preset yolov7-E6E recognition model combination training.
[0110] Step 305: Obtain a training image set, which includes multiple training images containing the target to be identified.
[0111] Step 306: Import the training image set into the yolov7-E6E recognition model to obtain the benchmark recognition result.
[0112] Step 307: When the current training is completed, update the preset update rate matrix with the ratio of each data point in the parameter matrix of the current YOLOv7-x recognition model to the corresponding data point in the parameter matrix of the YOLOv7-x recognition model when the previous training was completed.
[0113] Step 308: If the improvement rate of both the target loss data of the YOLOv7-x recognition model and the combined loss data of the current time is greater than a preset first threshold, update the parameter matrix of the YOLOv7-x recognition model for the current time. Here, the target loss data refers to the difference between the recognition result of the YOLOv7-x recognition model and the benchmark recognition result.
[0114] Step 309: Optimize the yolov7-x recognition model based on the update rate matrix.
[0115] Step 310: If the optimization is successful, update the optimized yolov7-x recognition model to the current yolov7-x recognition model.
[0116] Step 311: Repeat training until the combined loss data of the combined model meets the preset requirements, and determine the current yolov7-x recognition model as the trained yolov7-x recognition model.
[0117] In some embodiments, the specific implementation of steps 301-311 and the resulting technical effects can be found in [reference needed]. Figure 2 The steps in those corresponding embodiments will not be repeated here.
[0118] All of the above-mentioned optional technical solutions can be combined in any way to form the optional embodiments of this application, and will not be described in detail here.
[0119] The following are embodiments of the apparatus disclosed herein, which can be used to execute embodiments of the method disclosed herein. For details not disclosed in the apparatus embodiments of this disclosure, please refer to the embodiments of the method disclosed herein.
[0120] Further reference Figure 4 As an implementation of the above figures and methods, this disclosure provides some embodiments of an image recognition device, which are similar to... Figure 2 The above-described method embodiments correspond to these.
[0121] like Figure 4 As shown, the image recognition device 400 in some embodiments includes:
[0122] The acquisition module 401 is used to acquire the original image containing the target to be identified;
[0123] Conversion module 402 is used to convert the original image from RGB color space to LAB color space to obtain a LAB image;
[0124] The recognition module 403 is used to import the LAB image into the trained yolov7-x recognition model for recognition, and obtain the recognition result for the target to be recognized. The yolov7-x recognition model is determined by training based on a preset combination of yolov7-E6E recognition models. The model complexity of the yolov7-E6E recognition model is higher than that of the yolov7-x recognition model.
[0125] In some optional implementations of certain embodiments, the step of determining the YOLOv7-x recognition model based on a preset YOLOv7-E6E recognition model combination training includes: acquiring a training image set, the training image set including multiple training images containing the target to be recognized; importing the training image set into the YOLOv7-E6E recognition model to obtain a baseline recognition result; iteratively training the training image set into an untrained YOLOv7-x recognition model and a combined model consisting of the YOLOv7-x recognition model and the YOLOv7-E6E recognition model, respectively, until the combined loss data of the combined model meets preset requirements, thereby obtaining the trained YOLOv7-x recognition model.
[0126] In some optional implementations of certain embodiments, the combined loss data is the difference between the combined recognition result after weighted combination based on preset weighting parameters and the benchmark recognition result, which is the output of the last layer of the yolov7-x recognition model and the yolov7-E6E recognition model.
[0127] In some optional implementations of certain embodiments, the step of iteratively training the training image set into an untrained YOLOv7-x recognition model and a combined model consisting of the YOLOv7-x recognition model and the YOLOv7-E6E recognition model until the combined loss data of the combined model meets preset requirements, thereby obtaining the trained YOLOv7-x recognition model, includes: upon completion of the current training iteration, updating a preset update rate matrix with the ratio of each data point in the parameter matrix of the current YOLOv7-x recognition model to the corresponding data point in the parameter matrix of the YOLOv7-x recognition model at the end of the previous training iteration; if the current YOLOv7-x... If the improvement rate of both the target loss data of the recognition model and the combined loss data of the current iteration is greater than a preset first threshold, the parameter matrix of the YOLOv7-x recognition model for the current iteration is updated. Here, the target loss data refers to the difference between the recognition result of the YOLOv7-x recognition model and the benchmark recognition result. The YOLOv7-x recognition model is then optimized based on the update rate matrix. If the optimization is successful, the optimized YOLOv7-x recognition model is updated to the YOLOv7-x recognition model for the current iteration. Training is repeated until the combined loss data of the combined model meets preset requirements, at which point the YOLOv7-x recognition model for the current iteration is determined as the trained YOLOv7-x recognition model.
[0128] In some optional implementations of certain embodiments, optimizing the YOLOv7-x recognition model based on the update rate matrix includes: selecting an optimized update rate set from the update rate matrix that is less than a preset second threshold; randomly deleting connection branches in the network layer of the YOLOv7-x recognition model corresponding to a first preset number of update rates in the optimized update rate set, wherein the first preset number is a positive number; training the YOLOv7-x recognition model after randomly deleting connection branches; if the difference between the target loss data of the YOLOv7-x recognition model after randomly deleting connection branches and the target loss data before randomly deleting connection branches is less than a preset third threshold, the optimization is successful; otherwise, after reducing the first preset number by a preset pruning step size, the step of randomly deleting connection branches in the network layer of the YOLOv7-x recognition model corresponding to the first preset number of update rates in the optimized update rate set is repeated until the optimization is successful, or the first preset number is less than or equal to the pruning step size, indicating optimization failure, wherein the pruning step size is a positive number less than the initial first preset number.
[0129] In some optional implementations of some embodiments, the step of updating the optimized yolov7-x recognition model to the current yolov7-x recognition model if the optimization is successful includes: if the optimization fails, not optimizing the current yolov7-x recognition model.
[0130] In some optional implementations of some embodiments, the combined model loss data meeting the preset requirements means that in the most recent second preset number of training iterations, the variance of the changes in the combined loss data is less than a preset fourth threshold.
[0131] It is understandable that the modules described in the device 400 are similar to those in the reference. Figure 2 The steps in the described method correspond accordingly. Therefore, the operations, features, and beneficial effects described above for the method also apply to device 400 and the modules contained therein, and will not be repeated here.
[0132] like Figure 5 As shown, the electronic device 500 may include a processing unit (e.g., a central processing unit, a graphics processor, etc.) 501, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 502 or a program loaded from a storage device 508 into a random access memory (RAM) 503. The RAM 503 also stores various programs and data required for the operation of the electronic device 500. The processing unit 501, ROM 502, and RAM 503 are interconnected via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.
[0133] Typically, the following devices can be connected to I / O interface 505: input devices 506 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 507 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 508 including, for example, magnetic tapes, hard disks, etc.; and communication devices 509. Communication device 509 allows electronic device 500 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 5 An electronic device 500 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively. Figure 5 Each box shown can represent a device or multiple devices as needed.
[0134] In particular, according to some embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, some embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 509, or installed from a storage device 508, or installed from a ROM 502. When the computer program is executed by the processing device 501, it performs the functions defined in the methods of some embodiments of this disclosure.
[0135] It should be noted that, in some embodiments of this disclosure, the computer-readable medium described above may be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium may be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In some embodiments of this disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In some embodiments of this disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.
[0136] In some implementations, clients and servers can communicate using any currently known or future-developed network protocol such as HTTP (Hypertext Transfer Protocol) and can interconnect with digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), the Internet (e.g., the Internet of Things), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future-developed networks.
[0137] The aforementioned computer-readable medium may be included in the aforementioned device; or it may exist independently and not assembled into the electronic device. The aforementioned computer-readable medium carries one or more programs that, when executed by the electronic device, cause the electronic device to: acquire an original image containing the target to be identified; convert the original image from the RGB color space to the LAB color space to obtain a LAB image; import the LAB image into a trained YOLOv7-x recognition model for recognition, thereby obtaining a recognition result for the target to be identified, wherein the YOLOv7-x recognition model is determined by training a combination of preset YOLOv7-E6E recognition models, and the model complexity of the YOLOv7-E6E recognition model is higher than that of the YOLOv7-x recognition model.
[0138] Computer program code for performing operations of some embodiments of this disclosure can be written in one or more programming languages or a combination thereof, including object-oriented programming languages such as Java, Smalltalk, and C++, and conventional procedural programming languages such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).
[0139] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.
[0140] The modules described in some embodiments of this disclosure can be implemented in software or hardware. The described modules can also be located in a processor, for example, and can be described as:
[0141] The module consists of an acquisition module, a conversion module, and a recognition module. For example, the acquisition module can also be described as "a module that acquires the original image containing the target to be recognized."
[0142] The functions described above in this document can be performed, at least in part, by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: Field Programmable Gate Arrays (FPGAs), Application-Specific Integrated Circuits (ASICs), Application Standard Products (ASSPs), System-on-Chip (SoCs), Complex Programmable Logic Devices (CPLDs), and so on.
[0143] The above description is merely a selection of preferred embodiments of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in the embodiments of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described inventive concept. For example, technical solutions formed by substituting the above-described features with (but not limited to) technical features with similar functions disclosed in the embodiments of this disclosure.
Claims
1. An image recognition method, characterized in that, include: Obtain the original image containing the target to be identified; The original image is converted from the RGB color space to the LAB color space to obtain a LAB image; The LAB image is imported into the trained yolov7-x recognition model for recognition, and the recognition result for the target to be recognized is obtained. The yolov7-x recognition model is determined by training based on a preset yolov7-E6E recognition model. The model complexity of the yolov7-E6E recognition model is higher than that of the yolov7-x recognition model. The step of determining the YOLOv7-x recognition model by training a combination of preset YOLOv7-E6E recognition models includes: acquiring a training image set, which includes multiple training images containing the target to be recognized; importing the training image set into the YOLOv7-E6E recognition model to obtain a baseline recognition result; and iteratively training the training image set into an untrained YOLOv7-x recognition model and a combined model consisting of the YOLOv7-x recognition model and the YOLOv7-E6E recognition model, respectively, until the combined loss data of the combined model meets preset requirements, thereby obtaining the trained YOLOv7-x recognition model. The step of iteratively training the training image set into an untrained YOLOv7-x recognition model and a combined model consisting of the YOLOv7-x recognition model and the YOLOv7-E6E recognition model until the combined loss data of the combined model meets preset requirements, thereby obtaining the trained YOLOv7-x recognition model, includes: upon completion of the current training iteration, updating a preset update rate matrix with the ratio of each data point in the parameter matrix of the current YOLOv7-x recognition model to the corresponding data point in the parameter matrix of the YOLOv7-x recognition model at the end of the previous training iteration; if the target loss of the current YOLOv7-x recognition model is... If the improvement rate of the data and the combined loss data in the current iteration both exceed a preset first threshold, the parameter matrix of the YOLOv7-x recognition model for that iteration is updated. Here, the target loss data refers to the difference between the recognition result of the YOLOv7-x recognition model and the baseline recognition result. The YOLOv7-x recognition model is then optimized based on the update rate matrix. If the optimization is successful, the optimized YOLOv7-x recognition model is updated to the current YOLOv7-x recognition model. Training is repeated until the combined loss data of the combined model meets preset requirements, at which point the current YOLOv7-x recognition model is determined as the trained YOLOv7-x recognition model.
2. The method according to claim 1, characterized in that, The combined loss data is the output of the last layer of the yolov7-x recognition model and the yolov7-E6E recognition model, the combined recognition result after weighting based on preset weighting parameters, and the difference between the combined recognition result and the benchmark recognition result.
3. The method according to claim 1, characterized in that, The optimization of the YOLOv7-x recognition model based on the update rate matrix includes: Optimized update rate sets with values less than a preset second threshold are selected from the update rate matrix; Randomly delete the connection branches in the network layer of the YOLOv7-x recognition model corresponding to a first preset number of update rates in the optimized update rate set, wherein the first preset number is a positive number; If the difference between the target loss data of the YOLOv7-x recognition model after randomly deleting connection branches and the target loss data before randomly deleting connection branches is less than the preset third threshold, the optimization is considered successful. Otherwise, after reducing the first preset quantity by a preset pruning step size, the step of randomly deleting the connection branches in the network layer of the YOLOv7-x recognition model corresponding to the first preset quantity update rate in the optimized update rate set is repeated until optimization is successful, or optimization fails when the first preset quantity is less than or equal to the pruning step size, wherein the pruning step size is a positive number less than the initial first preset quantity.
4. The method according to claim 1, characterized in that, If the optimization is successful, the process of updating the optimized YOLOv7-x recognition model to the current YOLOv7-x recognition model includes: If the optimization fails, the YOLOv7-x recognition model will not be optimized for that iteration.
5. The method according to claim 1, characterized in that, The combined loss data of the combined model meets the preset requirements, meaning that in the most recent second preset number of training sessions, the variance of the changes in the combined loss data is less than a preset fourth threshold.
6. An image recognition device, characterized in that, include: The acquisition module is used to acquire the original image containing the target to be identified; The conversion module is used to convert the original image from the RGB color gamut space to the LAB color gamut space to obtain a LAB image; The recognition module is used to import the LAB image into the trained yolov7-x recognition model for recognition, and obtain the recognition result for the target to be recognized. The yolov7-x recognition model is determined by training based on a preset yolov7-E6E recognition model. The model complexity of the yolov7-E6E recognition model is higher than that of the yolov7-x recognition model. The step of determining the YOLOv7-x recognition model by training a combination of preset YOLOv7-E6E recognition models includes: acquiring a training image set, which includes multiple training images containing the target to be recognized; importing the training image set into the YOLOv7-E6E recognition model to obtain a baseline recognition result; and iteratively training the training image set into an untrained YOLOv7-x recognition model and a combined model consisting of the YOLOv7-x recognition model and the YOLOv7-E6E recognition model, respectively, until the combined loss data of the combined model meets preset requirements, thereby obtaining the trained YOLOv7-x recognition model. The step of iteratively training the training image set into an untrained YOLOv7-x recognition model and a combined model consisting of the YOLOv7-x recognition model and the YOLOv7-E6E recognition model until the combined loss data of the combined model meets preset requirements, thereby obtaining the trained YOLOv7-x recognition model, includes: upon completion of the current training iteration, updating a preset update rate matrix with the ratio of each data point in the parameter matrix of the current YOLOv7-x recognition model to the corresponding data point in the parameter matrix of the YOLOv7-x recognition model at the end of the previous training iteration; if the target loss of the current YOLOv7-x recognition model is... If the improvement rate of the data and the combined loss data in the current iteration both exceed a preset first threshold, the parameter matrix of the YOLOv7-x recognition model for that iteration is updated. Here, the target loss data refers to the difference between the recognition result of the YOLOv7-x recognition model and the baseline recognition result. The YOLOv7-x recognition model is then optimized based on the update rate matrix. If the optimization is successful, the optimized YOLOv7-x recognition model is updated to the current YOLOv7-x recognition model. Training is repeated until the combined loss data of the combined model meets preset requirements, at which point the current YOLOv7-x recognition model is determined as the trained YOLOv7-x recognition model.
7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method as described in any one of claims 1 to 5.
8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1 to 5.