Image recognition method, device, electronic device, and storage medium

Abstract

The present application discloses an image recognition method, device, electronic device, and storage medium, which obtain an image to be recognized; perform linear processing on the image to be recognized to obtain linear image data; wherein the data type of the linear image data is a floating-point type; perform preprocessing on the linear image data to obtain preprocessed image data; wherein the preprocessed image data is within a preprocessing threshold range; perform a first quantization processing on the preprocessed image data according to an image quantization coefficient to obtain first image data; wherein the data type of the first image data is an integer type; perform a second quantization processing on the first image data to obtain second image data; wherein the data type of the second image data is an integer type; perform recognition processing on the second image data to obtain a recognition result of the image to be recognized. In this way, during the image recognition process, the image data to be processed is converted into integer data, which reduces the processing difficulty and the amount of calculation, thereby effectively improving the processing efficiency of the neural network model and promoting the further application of neural network technology in the field of computer vision.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and more specifically, to an image recognition method, device, electronic device, and storage medium. Background Art

[0002] With the development of artificial intelligence technology, neural network models have been widely studied in the field of computer vision. In fields such as autonomous driving, neural network models can be used for image recognition to assist vehicles in road condition identification, parking space identification, etc., and are playing an increasingly important role.

[0003] However, as the requirements for image recognition target detection, target classification and other accuracy become higher and higher, the structure of the neural network model becomes more and more complex, and the difficulty of data processing by the neural network model also increases, which seriously affects the processing speed of the neural network model and limits the further application of the neural network model in the field of computer vision. Summary of the Invention

[0004] In view of the above problems, the present invention proposes an image recognition method, device, electronic device and storage medium to improve the above problems.

[0005] In a first aspect, an embodiment of the present application provides an image recognition method, which includes: obtaining an image to be recognized; performing linear processing on the image to be recognized to obtain linear image data; the data type of the linear image data is a floating-point type; preprocessing the linear image data to obtain preprocessed image data; the preprocessed image data is within a preprocessing threshold range; performing a first quantization processing on the preprocessed image data according to an image quantization coefficient to obtain first image data; the data type of the first image data is an integer type; performing a second quantization processing on the first image data to obtain second image data; the data type of the second image data is an integer type; performing recognition processing on the second image data to obtain a recognition result of the image to be recognized.

[0006] In a second aspect, an embodiment of the present application further provides an image recognition device, comprising: an image acquisition module, a linear processing module, a preprocessing module, a first quantization module, and a second quantization module. The image acquisition module is used to acquire an image to be recognized; the linear processing module is used to perform linear processing on the image to be recognized to obtain linear image data; the data type of the linear image data is a floating-point type; the preprocessing module is used to preprocess the linear image data to obtain preprocessed image data; the preprocessed image data is within a preprocessing threshold range; the first quantization module is used to perform a first quantization processing on the preprocessed image data according to an image quantization coefficient to obtain first image data; the data type of the first image data is an integer type; the second quantization module is used to perform a second quantization processing on the first image data to obtain second image data; the data type of the second image data is an integer type; and the recognition module is used to perform recognition processing on the second image data to obtain a recognition result of the image to be recognized.

[0007] In a third aspect, an embodiment of the present application further provides an electronic device comprising one or more processors, a memory, and one or more application programs, wherein the one or more application programs are stored in the memory and configured to be executed by the processor to implement the above-mentioned image recognition method.

[0008] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, in which program code is stored, wherein the above-mentioned image recognition method is executed when the program code is run by a processor.

[0009] The technical solution provided by the present invention comprises the following steps: obtaining an image to be recognized; performing linear processing on the image to be recognized to obtain linear image data; the data type of the linear image data is a floating-point type; preprocessing the linear image data to obtain preprocessed image data; the preprocessed image data is within a preprocessing threshold range; performing a first quantization process on the preprocessed image data according to an image quantization coefficient to obtain first image data; the data type of the first image data is an integer type; performing a second quantization process on the first image data to obtain second image data; the data type of the second image data is an integer type; and performing recognition processing on the second image data to obtain a recognition result of the image to be recognized. Thus, during the image recognition process, the image data to be processed is converted into integer data, which reduces the processing difficulty and the amount of computation, thereby effectively improving the processing efficiency of the neural network model and promoting the further application of neural network technology in the field of computer vision. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] In order to more clearly illustrate the technical solutions in the embodiments of this application, the following is a brief introduction to the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments and drawings obtained by ordinary technicians in this field without creative work are within the scope of protection of this invention.

[0011] Figure 1 A flow chart of an image recognition method proposed in an embodiment of the present application is shown.

[0012] Figure 2 The figure shows a structural block diagram of an image recognition device proposed in an embodiment of the present application.

[0013] Figure 3 A structural block diagram of an electronic device proposed in an embodiment of the present application is shown.

[0014] Figure 4 A structural block diagram of a computer-readable storage medium proposed in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0015] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.

[0016] The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. The components of the embodiments of the present application generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work fall within the scope of protection of the present application.

[0017] With the development of artificial intelligence technology, neural network models have been widely studied in the field of computer vision. In fields such as autonomous driving, neural network models can be used for image recognition to assist vehicles in road condition identification, parking space identification, etc., and are playing an increasingly important role.

[0018] However, as the requirements for image recognition, target detection, and target classification accuracy become increasingly higher, the structure of the neural network model becomes more complex, and the difficulty of data processing by the neural network model also increases. The neural network model involves floating-point numerical calculations and requires a large amount of computationally intensive and complex calculations, which seriously affects the processing speed of the neural network model and limits its further application in the field of computer vision.

[0019] In order to improve the above-mentioned problems, the inventors of the present application have proposed the image recognition method, device, electronic device and storage medium provided by the present application, which are as follows: obtaining an image to be recognized; performing linear processing on the image to be recognized to obtain linear image data; the data type of the linear image data is a floating-point type; preprocessing the linear image data to obtain preprocessed image data; the preprocessed data is within the preprocessing threshold range; performing a first quantization processing on the preprocessed image data according to the image quantization coefficient to obtain first image data; the data type of the first image data is an integer type; performing a second quantization processing on the first image data to obtain second image data; the data type of the second image data is an integer type; performing recognition processing on the second image data to obtain a recognition result of the image to be recognized. In this way, during the image recognition process, the data to be processed is converted into integer type data, which reduces the processing difficulty and the amount of calculation, thereby effectively improving the processing efficiency of the neural network model and promoting the further application of neural network technology.

[0020] The embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0021] See also Figure 1 An embodiment of the present application provides an image recognition method that can be applied to electronic devices. This embodiment describes the step flow on the computer device side. The method may include steps S110 to S160.

[0022] Step S110: Acquire an image to be recognized.

[0023] In an embodiment of the present application, an electronic device may acquire an image to be recognized. The image to be recognized may be an image acquired during vehicle operation, such as an image containing elements such as road conditions and parking spaces. The electronic device performs image recognition on the image to identify the road conditions, parking spaces, etc. in the image, thereby assisting the vehicle in autonomous driving.

[0024] In some embodiments, the electronic device may be a server or a terminal, and the electronic device has computing capabilities and may include a storage unit, a processor, and the like.

[0025] Step S120: performing linear processing on the image to be recognized to obtain linear image data.

[0026] In an embodiment of the present application, an electronic device may perform image recognition on an image to be recognized using a neural network model. The neural network module may include a linear structure to linearly process the image to be recognized using the linear structure to obtain linear image data. The data type of the linear image data is a floating point type.

[0027] In some embodiments, the type of the linear image data may be one of float16, float32, and float64, etc. In the embodiment of the present application, the type of the data to be processed is float32 as an example for description.

[0028] In some embodiments, the linear structure may include convolutional layers, normalization layers, pooling layers, and the like. For example, the convolutional layer can be used to extract key features required for image recognition in an image to be recognized, such as parking space corner features. The specific configuration can be based on actual usage needs and is not limited in this application.

[0029] Step S130: pre-process the linear image data to obtain pre-processed image data.

[0030] In the embodiments of the present application, in order to solve the defect of insufficient expression ability of linear structures, it is necessary to activate linear image data through activation structures to increase nonlinear factors and enhance the ability of neural network models to learn and understand complex and nonlinear functions.

[0031] However, activation structures require large amounts of memory and computation, resulting in slower inference speeds. This is illustrated by using the GELU activation function as an example. The GELU activation function is an activation function structure used in neural network models. Compared to other activation structures, the GELU activation function can significantly improve the accuracy of neural network models in object detection and classification. In the field of computer vision, in particular, neural network models based on the Transformer module often use the GELU activation function as their activation structure.

[0032] However, the GELU activation function essentially adds the idea of ​​random regularization to the RELU activation function. Specifically, after linearly activating the neuron input, it multiplies it by a Gaussian normal distribution factor. The specific formula for the GELU activation function is shown in Formula 1.

[0033] xP(X≤x)=xφ(x) (Formula 1)

[0034] Where φ(x) represents the probability distribution function of the Gaussian normal distribution of x. The full expansion of φ(x) is shown in Formula 2.

[0035]

[0036] Among them, simplifying formula 2 gives formula 3.

[0037]

[0038] In Formulas 1 to 3 above, x represents the input received by the GELU activation function from the neuron in the previous layer. Since φ(x) follows a normal distribution, the value of φ(x) ranges from [0, 1]. As the value of φ(x) approaches 0, the value of xφ(x) (i.e., the output of the GELU activation function) approaches 0. As the value of φ(x) approaches 0, the value of xφ(x) approaches x, that is, the output of the activation structure corresponding to the GELU activation function is more likely to retain the value of x.

[0039] From the above formula, we can know that in the GELU activation function, since the value range of φ(x) is between [0, 1], all the numerical calculations involved are floating-point types, and the calculation of the Gaussian distribution in φ(x) includes complex calculations such as integration, square root, and square. The amount of calculation is large and complex, which seriously affects the reasoning speed and limits the further application of neural network technology in the field of computer vision.

[0040] In the embodiments of the present application, the linear image data is of floating-point type. If it is directly processed using the existing activation structure, the calculation is complex and the processing efficiency is low. Therefore, the image recognition method provided in the embodiments of the present application can be used to process the linear image data to improve the running speed of the neural network model.

[0041] Below, the specific process of the image recognition method provided by the embodiment of the present application will be described by taking the activation structure using the GELU activation function as an example.

[0042] In order to reduce errors, in an embodiment of the present application, the linear image data is preprocessed to obtain preprocessed image data, so as to limit the numerical value of the linear image data to within a preprocessing threshold range, thereby ensuring the calculation accuracy.

[0043] In some embodiments, the minimum value of the pre-processing threshold range is the first processing value, and the maximum value of the pre-processing threshold range is the second processing value.

[0044] In some implementations, in order to pre-process the linear image data, step S130 may include the following steps.

[0045] (1) If the linear image data is within a preset processing threshold range, the linear image data is used as pre-processed image data.

[0046] (2) Otherwise, if the linear image data is less than the first processing value, the value of the pre-processed image data is set to the first processing value.

[0047] (3) If the linear image data is greater than the second processing value, the value of the pre-processed image data is set to the second processing value.

[0048] It can be understood that the preprocessing threshold range can be set according to actual usage needs, and can be specifically determined according to the subsequent required integer type (the data type is subsequently converted from floating point type to integer type). This application does not impose any restrictions on this. For example, when the subsequent required integer type is Int8 type, the preprocessing threshold range can be -2.93 to 2.93.

[0049] For example, a preprocessing threshold value range of -2.93 to 2.93 is used for illustration, i.e., the first processed value is -2.93 and the second processed value is 2.93. If the linear image data value is 2, i.e., linear image data 2 is within the preprocessing threshold value range, then the preprocessed image data corresponding to linear image data 2 is 2. If the linear image data value is -3, i.e., linear image data -3 is less than the first processed value -2.93, then the preprocessed image data corresponding to linear image data -3 is -2.93. If the linear image data value is 3, i.e., linear image data 3 is greater than the second processed value 2.93, then the preprocessed image data corresponding to linear image data 3 is 2.93.

[0050] Step S140 : performing a first quantization process on the pre-processed image data according to the image quantization coefficient to obtain first image data.

[0051] In an embodiment of the present application, the electronic device performs a first quantization process on the pre-processed image data according to the image quantization coefficient to convert the pre-processed image data into first image data of an integer type, so as to reduce the complexity of subsequent further calculations.

[0052] In some embodiments, the integer type may be one of int8, int10, int16, bf16, etc. The embodiment of the present application is described with the integer type being int8.

[0053] In an embodiment of the present application, the image quantization coefficient is used to convert linear image data from a floating-point type to an integer type.

[0054] In some implementations, the image quantization coefficient may be a specified value, that is, set according to actual usage requirements.

[0055] In some implementations, before step S140 , the image quantization coefficients may also be calculated through the following steps.

[0056] (1) Determine the absolute value of the linear image data to obtain process image data; wherein the number of the linear image data is at least one.

[0057] (2) The data with the largest value in the process image data is used as the target process image data.

[0058] (3) Determine an image quantization scale coefficient according to a preset threshold and target process image data, wherein the image quantization scale coefficient is not greater than the preset threshold.

[0059] In an embodiment of the present application, the number of linear image data is one or more. When the number of linear image data is multiple, the absolute value of each linear image data is determined one by one to obtain the process image data corresponding to each linear image data.

[0060] Among them, the embodiment of the present application sets a preset threshold to limit the linear image data within the preset threshold, so that the floating-point linear image data can be evenly divided into a preset number of data spaces to avoid errors caused by uneven data distribution due to excessive data distribution.

[0061] In the embodiment of the present application, the preset threshold value may be in the range of 4.5 to 5.5. Optionally, the preset threshold value may be set to 4.5, 5, 5.5, etc., and may be set specifically according to actual needs. In order to obtain higher accuracy, preferably, the preset threshold value may be set to 5, that is, the preset threshold value is quantized to an integer, so that the image recognition method of the embodiment of the present application can achieve higher accuracy.

[0062] In some implementations, if the process image data is less than a preset threshold, the process image data is used as the preset threshold.

[0063] In some implementations, if the process image data is greater than or equal to a preset threshold, the preset threshold is used as the image quantization coefficient.

[0064] In some implementations, if a specified image quantization coefficient exists, subsequent calculations are performed based on the specified image quantization coefficient.

[0065] In some embodiments, if it is determined that there is no designated image quantization coefficient, the image quantization coefficient is determined according to the above method.

[0066] In some implementations, in order to perform the first quantization process on the pre-processed image data, step S140 includes the following steps.

[0067] (1) Determine the quantization threshold range based on the image quantization coefficient.

[0068] In the embodiment of the present application, according to the image quantization scale coefficient a_max (a_max>0), the quantization threshold range can be determined to be -a_max to a_max, wherein the first quantization value is -a_max and the second quantization value is a_max.

[0069] (2) Determine first data based on the preprocessed image data and the quantization threshold range; wherein the first data is within the quantization threshold range.

[0070] In the embodiment of the present application, in order to improve calculation accuracy, it is necessary to limit the pre-processed image data to within a quantization threshold range.

[0071] In some embodiments, if the pre-processed image data is within the quantization threshold range, the pre-processed image data is used as the first data.

[0072] In some embodiments, if the pre-processed image data is less than the first quantization value, the first quantization value is used as the first data.

[0073] In some embodiments, if the pre-processed image data is greater than the second quantized value, the second quantized value is used as the first data.

[0074] (3) Obtain second data based on the first data and the scale factor.

[0075] In an embodiment of the present application, the scale factor may be determined according to the image quantization coefficient and the subsequently required integer type.

[0076] Take the required integer type int8 as an example. Since int8 occupies 1 byte, that is, 8 binary bits, each binary bit can store two numbers 0 and 1, and 8 binary bits can have 2 8 = 256 combinations, that is, 256 numbers can be stored, and the scale factor sclae can be obtained by the following formula 4.

[0077]

[0078] Among them, sclae represents the scale coefficient, a_max represents the quantization scale coefficient; Int represents the integer coefficient.

[0079] In the embodiments of the present application, to partition the first data into the data space corresponding to the integer type, the first data can be divided by the scale factor to obtain the second data. For example, if the integer type is int8, the int8 type has 256 data spaces corresponding to it, so Int is 256. Similarly, if the integer type is int10, the int10 type has 1024 data spaces corresponding to it, so Int is 1024. The same applies to other sorting types and will not be further described here.

[0080] (4) The second data is clipped to obtain third data, wherein the third data is within a clipping threshold range.

[0081] Among them, the clipping threshold range can be determined according to the required integer type. For example, the required integer type is int8 type. Since the int8 integer type corresponds to 256 data spaces, that is, it can correspond to 256 numerical values, the clipping threshold range can be set according to the corresponding 256 data to limit the second data to within the range of 256 data.

[0082] For example, 256 data can be divided into -128 to 127, including 128 negative integers, 0, and 127 positive integers. The clipping threshold range can be set to -128 to 127, where the first clipping value is -128 and the second clipping value is 127. It is understood that in other implementations, the clipping threshold range can also be set to other ranges, such as -127 to 128.

[0083] In some implementations, if the second data is within the clipping threshold range, the second data is used as the third data.

[0084] In some implementations, if the second data is smaller than the first clipping value, the first clipping value is used as the third data.

[0085] In some implementations, if the second data is greater than the second clipping value, the second clipping value is used as the third data.

[0086] For example, taking the clipping threshold range of -128 to 127 as an example, the first clipping value is -128 and the second clipping value is 127. If the second data is 10, that is, the second data 10 is within the clipping threshold range, then the third data corresponding to the second data 10 is 10; if the second data is -130, that is, the second data 10 is less than the first clipping value -128, then the third data corresponding to the second data -130 is -128; if the second data is 130, that is, the second data 130 is greater than the second clipping value 127, then the third data corresponding to the second data 130 is 127.

[0087] (5) Rounding the third data to obtain the first image data.

[0088] In an embodiment of the present application, the third data is further converted into first image data of integer type by rounding, wherein the first image data is within the cropping threshold range.

[0089] Optionally, the decimal part of the third data may be discarded to obtain the first image data. For example, if the third data is 10.02, the first quantized data corresponding to the third data 10.02 is 10.

[0090] Optionally, the third number may be rounded to obtain the first image data. For example, if the third data is 120.6, the first image data corresponding to the third data 120.6 is 121. For another example, if the third data is 120.1, the first image data corresponding to the third data 120.1 is 120.

[0091] Step S150: Perform a second quantization process on the first image data to obtain second image data.

[0092] In an embodiment of the present application, to obtain data after linear image data activation processing, the first image data may be subjected to a second quantization process to obtain second image data, wherein the first image data and the second image data are integer types. The integer type may be, for example, int8, int10, int16, etc., and the present embodiment is described using the integer type int8.

[0093] In some implementations, in order to perform the second quantization process on the first image data, step S150 may include the following steps.

[0094] (1) Determine the output address based on the first image data and the preset first address.

[0095] (2) According to the output address and the preset storage space, the data corresponding to the output address in the preset address corresponding table is searched as the second image data.

[0096] In the embodiment of the present application, the required integer type is int8 type for illustration, and the first image data can be obtained according to formulas 6 to 7.

[0097]

[0098] Among them, q i represents the first image data, q i is an integer, q i Within the clipping threshold range; Result i Indicates q i The corresponding second image data; 1≤i≤Int, i is an integer; scale_lut represents the activation coefficient; a_max represents the quantization scale coefficient; Int represents the integer coefficient; the Round function is used to round the data.

[0099] In an embodiment of the present application, when the integer type is Int8, Int is 256, and when the integer type is Int10, Int is 1024. The specific setting can be made according to the required integer type, and the present application does not impose any restrictions on this.

[0100] In an embodiment of the present application, the clipping threshold range can be set according to the integer type. For example, when the integer type is Int8, the clipping threshold range can be set to -128 to 127.

[0101] By using formulas 6 and 7, the second image data corresponding to each value within the cropping threshold range can be determined, as shown in Table 1.

[0102] <![CDATA[q i ]]> <![CDATA[Result i ]]> <![CDATA[q1=-128]]> 0 <![CDATA[q 129 =0]]> 129 …… …… <![CDATA[q 256 =127]]> 256

[0103] Table 1

[0104] In order to improve the operation speed, a continuous preset address space can be allocated in advance in the memory of the electronic device. Taking the integer type of int8 as an example, the size of the preset address space is 256, and the preset first address offset of the address space is determined. i Corresponding second image data Result i Sequentially stored in the preset address space, wherein each second image data Result i The corresponding storage address is Q i , as shown in Table 2.

[0105] <![CDATA[q i ]]> <![CDATA[Q i =offset+q i ]]> <![CDATA[Result i ]]> <![CDATA[q1=-128]]> offset-128 0 <![CDATA[q 129 =0]]> offset 129 …… …… …… <![CDATA[q 256 =127]]> offset+127 256

[0106] Table 2

[0107] Therefore, the output address can be determined according to the first image data and the preset first address. Specifically, the output address can be determined according to Formula 8.

[0108] Q i =q i +offset (Formula 8)

[0109] Among them, q i Indicates the first image data, offset indicates the preset first address, Q i Indicates the output address.

[0110] For example, if the first image data is -127 and the preset first address is 0x2127, the corresponding output address is 0x2000. The data corresponding to the output address in the preset address correspondence table can then be searched and used as the second image data, effectively reducing the amount of computation and improving computational efficiency. It is understood that the preset first address can be set based on actual needs. This is merely an example and is not intended to limit this application.

[0111] Step S160: perform recognition processing on the second image data to obtain a recognition result of the image to be recognized.

[0112] In an embodiment of the present application, the neural network model performs recognition processing based on the second image data to obtain a recognition result corresponding to the image to be recognized. For example, the recognition result may include but is not limited to the coordinates of key points of the parking space, traffic light status, obstacles, etc., and the vehicle can then assist in automatic driving based on the recognition result.

[0113] Through the above embodiments, the image recognition method provided in the embodiments of the present application can realize accelerated processing of linear image data in the neural network model to improve computing efficiency. Since in the above embodiments, the type of linear image data involved is converted from floating-point type to integer type, the conversion process may cause errors in the processing results due to quantization errors.

[0114] In order to reduce the error caused by the conversion process, in some embodiments, the image recognition method of the embodiment of the present application may further include: performing inverse quantization processing on the second image data to obtain third image data, wherein the type of the third image data is a floating point type.

[0115] In some implementations, taking the integer type of int8 as an example, the third image data can be determined by the following formula nine.

[0116]

[0117] Among them, dequant i Represents the third image data; Result i represents the second image data, dea_min represents the first inverse quantization scale coefficient; dea_max represents the second inverse quantization scale coefficient; Int represents an integer coefficient.

[0118] In an embodiment of the present application, the first inverse quantization scale coefficient may be set to 0, and the second inverse quantization scale coefficient may be set to 1.

[0119] In an embodiment of the present application, the second image data is dequantized, and the third image data participates in the training of the neural network model, so that the operation of traininig aware quantization can be performed during the training of the neural network model, so that the neural network model can learn the quantization loss caused by the quantization conversion process, so that the integer data accuracy and floating-point data accuracy on the finally trained neural network model are consistent.

[0120] In order to better illustrate the effect of the hardware acceleration method of the embodiment of the present application, multiple sets of linear image data will be randomly obtained below. The first result obtained by using a traditional activation structure (such as a GELU function as an activation function) for the linear image data is compared with the second result obtained by using the image recognition method of the embodiment of the present application for the linear image data. The similarity between the two is determined (for example, using a cosine similarity method, etc.) to compare the processing effects of the two, as shown in Table 3 below.

[0121] Data to be processed Similarity [435,1,2048] 0.99 [1,435,2048] 1.00 [2048,435,1] 1.00 [100,50,2048] 1.00 [3,224,224] 1.00

[0122] Table 3

[0123] It should be noted that the closer the similarity is to 1, the higher the similarity between the first result and the second result, which means that the image recognition accuracy of the embodiment of the present application is higher. As can be seen from Table 3, the image recognition method used in the embodiment of the present application has a higher similarity in calculation results compared to the traditional activation structure. In other words, the image recognition method used in the embodiment of the present application has a smaller impact on the results due to the quantization error caused by the quantization conversion, which further proves that the image recognition method used in the embodiment of the present application has a higher accuracy.

[0124] The embodiment of the present application provides a hardware acceleration method, which comprises the following steps: obtaining an image to be recognized; performing linear processing on the image to be recognized to obtain linear image data; wherein the data type of the linear image data is a floating-point type; preprocessing the linear image data to obtain preprocessed image data; wherein the preprocessed image data is within a preprocessing threshold range; performing a first quantization processing on the preprocessed image data according to an image quantization coefficient to obtain first image data; wherein the data type of the first image data is an integer type; performing a second quantization processing on the first image data to obtain second image data; wherein the data type of the second image data is an integer type; and performing recognition processing on the second image data to obtain a recognition result of the image to be recognized. In this way, during the image recognition process, the data to be processed is converted into integer data, which reduces the processing difficulty and the amount of calculation, thereby effectively improving the processing efficiency of the neural network model and promoting the further application of neural network technology.

[0125] See also Figure 2 , which shows an image recognition device 200 provided by an embodiment of the present invention, the image recognition device 200 includes: an image acquisition module 210, a linear processing module 220, a preprocessing module 230, a first quantization module 240, a second quantization module 250 and a recognition module 260.

[0126] The image acquisition module 210 is used to acquire an image to be identified.

[0127] The linear processing module 220 is used to perform linear processing on the image to be recognized to obtain linear image data; wherein the data type of the linear image data is a floating point type.

[0128] The preprocessing module 230 is used to preprocess the linear image data to obtain preprocessed image data, and the preprocessed image data is within a preprocessing threshold range.

[0129] The first quantization module 240 is used to perform a first quantization process on the pre-processed image data according to the image quantization coefficient to obtain first image data, wherein the type of the first image data is an integer type.

[0130] The second quantization module 250 is used to perform a second quantization process on the first image data to obtain second image data; wherein the type of the second image data is an integer type.

[0131] In some embodiments, the image recognition device 200 further includes a dequantization module, which is configured to perform dequantization processing on the second image data to obtain third image data, wherein the third image data is of floating-point type.

[0132] In some embodiments, the minimum value of the pre-processing threshold range is a first processing value, and the maximum value of the pre-processing threshold range is a second processing value. The pre-processing module 230 is further configured to use the linear image data as pre-processed image data if the linear image data is within the preset processing threshold range; otherwise, use the first processing value as pre-processed image data if the linear image data is less than the first processing value; and use the second processing value as pre-processed image data if the data to be processed is greater than the second processing value.

[0133] In some embodiments, the first quantization module 240 includes a quantization threshold determination unit, a first data determination unit, a second data determination unit, a third data determination unit, and a first quantization unit.

[0134] The quantization threshold determination unit is used to determine the quantization threshold range according to the image quantization coefficient.

[0135] The first data determination unit is configured to determine first data according to the preprocessed image data and a quantization threshold range; wherein the first data is within the quantization threshold range.

[0136] The second data determination unit is used to obtain second data according to the first data and the scale factor.

[0137] The third data determining unit is configured to perform clipping processing on the second data to obtain third data, wherein the third data is within a clipping threshold range.

[0138] The first quantization unit is used to round the third data to obtain the first image data.

[0139] In some embodiments, the second quantization module 250 includes an output address determination unit and a second quantization unit.

[0140] The output address determining unit is used to determine the output address according to the first image data and a preset first address.

[0141] The second quantization unit is used for searching the preset address correspondence table for data corresponding to the output address as the second image data according to the output address and the preset storage space.

[0142] In some embodiments, the image recognition device 200 further includes: an absolute value determination module, a target determination module, and an image quantization coefficient determination module.

[0143] The absolute value determination module is used to determine the absolute value of the linear image data to obtain process image data. The number of the linear image data is at least one.

[0144] The target determination module is used to take the data with the largest value in the process image data as the target process image data.

[0145] The image quantization coefficient determination module is used to determine the image quantization coefficient according to a preset threshold and target process image data, wherein the image quantization coefficient is not greater than the preset threshold.

[0146] In some embodiments, the absolute value determination module is configured to determine the absolute value of the linear image data if it is determined that there is no designated quantization scale coefficient.

[0147] It should be noted that the various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to in detail. For the device embodiments, since they are basically similar to the method embodiments, the description is relatively simple. For the relevant parts, refer to the partial description of the method embodiments. Any processing method described in the method embodiment can be implemented by the corresponding processing module in the device embodiment, and will not be repeated in detail in the device embodiment.

[0148] In addition, each functional module in each embodiment of the present application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The above-mentioned integrated modules can be implemented in the form of hardware or software functional modules.

[0149] See also Figure 3Based on the above hardware acceleration method, the embodiment of the present application also provides another electronic device 300 including a processor 310 capable of executing the above hardware acceleration method. The electronic device 300 also includes one or more processors 310, a memory 320, and one or more application programs. The memory 320 stores a program capable of executing the content of the above embodiment, and the processor 310 can execute the program stored in the memory 320.

[0150] The processor 310 may include one or more cores for processing data and a message matrix unit. The processor 310 utilizes various interfaces and circuits to connect various components within the electronic device 300. It executes instructions, programs, code sets, or instruction sets stored in the memory 320, as well as accesses data stored in the memory 320, to perform various functions of the electronic device 300 and process data. Optionally, the processor 310 may be implemented using at least one of the following hardware forms: a digital signal processing (DSP), a field-programmable gate array (FPGA), or a programmable logic array (PLA). The processor 310 may integrate one or a combination of a central processing unit (CPU), a graphics processing unit (GPU), and a modem. The CPU primarily handles the operating system, user interface, and application programs; the GPU is responsible for rendering and drawing display content; and the modem handles wireless communications. It is understood that the modem may not be integrated into the processor 310 and may be implemented separately via a communication chip.

[0151] The memory 320 may include a random access memory 320 (RAM) or a read-only memory 320 (Read-Only Memory). The memory 320 may be used to store instructions, programs, codes, code sets, or instruction sets. The memory 320 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for implementing at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the various method embodiments described below, etc. The data storage area may also store data created by the terminal during use (such as linear image data, pre-processed image data, first image data, second image data), etc.

[0152] Please refer to Figure 4, which shows a structural block diagram of a computer-readable storage medium 400 provided in an embodiment of the present application. The computer-readable storage medium stores program code 410, which can be called by a processor to execute the method described in the above method embodiment.

[0153] The computer-readable storage medium 400 can be an electronic memory such as a flash memory, an EEPROM (Electrically Erasable Programmable Read-Only Memory), an EPROM, a hard disk, or a ROM. Alternatively, the computer-readable storage medium 400 includes a non-transitory computer-readable storage medium. The computer-readable storage medium 400 has storage space for program code 410 for executing any of the method steps in the above method. These program codes 410 can be read from or written to one or more computer program products. The program code 410 can be compressed, for example, in a suitable form.

[0154] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. An image recognition method, characterized in that: The method comprises: Obtain the image to be recognized; Performing linear processing on the image to be recognized to obtain linear image data; the data type of the linear image data is a floating point type; Preprocessing the linear image data to obtain preprocessed image data; the preprocessed image data is within a preprocessing threshold range; Determine a quantization threshold range according to the image quantization coefficient; determining first data according to the preprocessed image data and the quantization threshold range; wherein the first data is within the quantization threshold range; Obtaining second data according to the first data and a scale factor; Clipping the second data to obtain third data, wherein the third data is within a clipping threshold range; Rounding the third data to obtain first image data; the data type of the first image data is an integer type; determining an output address according to the first image data and a preset first address; According to the output address and the preset storage space, searching the preset storage space for data corresponding to the output address as second image data; the data type of the second image data is an integer type; Recognition processing is performed on the second image data to obtain a recognition result of the image to be recognized.

2. The method according to claim 1, characterized in that The performing recognition processing on the second image data to obtain a recognition result of the image to be recognized includes: Performing inverse quantization processing on the second image data to obtain third image data, wherein a data type of the third image data is a floating point type; Recognition processing is performed on the third image data to obtain a recognition result of the image to be recognized.

3. The method according to claim 1, characterized in that The minimum value of the preprocessing threshold range is the first processing value, and the maximum value of the preprocessing threshold range is the second processing value; The preprocessing of the linear image data to obtain preprocessed image data includes: If the linear image data is within the preprocessing threshold range, using the linear image data as preprocessing image data; otherwise, if the linear image data is less than the first processing value, setting the value of the pre-processed image data to the first processing value; If the linear image data is greater than the second processing value, the value of the pre-processed image data is set to the second processing value.

4. The method according to claim 1, wherein Before determining the quantization threshold range according to the image quantization coefficient, the method further includes: Determine the absolute value of the linear image data to obtain process image data; wherein the number of the linear image data is at least one; determining the largest process image data among the process image data as target process image data; An image quantization scale coefficient is determined according to a preset threshold and the target process image data, wherein the image quantization scale coefficient is not greater than the preset threshold.

5. The method according to claim 4, characterized in that Determining the absolute value of the linear image data includes: If it is determined that there is no designated image quantization coefficient, the absolute value of the linear image data is determined.

6. An image recognition device, characterized in that: The device comprises: An image acquisition module, used to acquire an image to be identified; A linear processing module, configured to perform linear processing on the image to be identified to obtain linear image data; the data type of the linear image data is a floating point type; A preprocessing module, configured to preprocess the linear image data to obtain preprocessed image data; the preprocessed image data is within a preprocessing threshold range; a first quantization module, configured to determine a quantization threshold range based on an image quantization coefficient; determine first data based on the preprocessed image data and the quantization threshold range, wherein the first data is within the quantization threshold range; obtain second data based on the first data and a scale coefficient; crop the second data to obtain third data, wherein the third data is within the cropping threshold range; and round the third data to obtain first image data; wherein the data type of the first image data is an integer type; a second quantization module, configured to determine an output address based on the first image data and a preset first address; and, based on the output address and a preset storage space, search the preset storage space for data corresponding to the output address as second image data; wherein the data type of the second image data is an integer type; The recognition module is used to perform recognition processing on the second image data to obtain a recognition result of the image to be recognized.

7. An electronic device, characterized in that: include: one or more processors; Memory; One or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the one or more processors, and the one or more applications are configured to execute the image recognition method according to any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores program code, and the program code can be called by a processor to execute the image recognition method according to any one of claims 1 to 5.

Images