Image format conversion method and conversion module, image recognition device and recognition method

Abstract

The present invention discloses an image format conversion method and conversion module, and an image recognition device and recognition method based on the image format conversion, wherein the image format conversion method includes saving the bitmap data entity of a bmp format file as a 16-bit unsigned integer x; normalizing the 16-bit unsigned integer x using a first conversion formula to obtain a normalized integer x', wherein max(x) represents the maximum value of the 16-bit unsigned integer x; converting the normalized integer x' into an 8-bit unsigned integer y, and performing a nonlinear conversion on the 8-bit unsigned integer y using a second conversion formula, and saving the conversion result z as a png format file. The bitmap data entity of the bmp format file is normalized and nonlinearly converted, and saved as a png format file, so as to avoid the problem of image display, and a complete picture can be displayed using conventional equipment, thereby improving the accuracy of target detection.

Description

Technical Field

[0001] The present invention belongs to the technical field of video image data processing, and more specifically, relates to an image format conversion method and conversion module, an image recognition device and a recognition method. Background Art

[0002] Intelligent detection and recognition of targets can reduce manpower consumption and improve processing efficiency for image data processing on the one hand; on the other hand, it can provide a basis and method support for subsequent target positioning, key events and automatic detection and recognition of states. Since the image display of 16-bit bmp format files has problems, and there are often multiple targets in the image, and the targets are of different sizes, the existing target detection and recognition generally uses the YOLO (You Only LookOnce) model, which is large in size and has a deep network layer. After multiple pooling, the features of small targets are submerged. Therefore, the accuracy of image target detection for bmp format files needs to be improved. Summary of the invention

[0003] In view of the above defects or improvement needs of the prior art, the present invention provides an image format conversion method and conversion module, an image recognition device and recognition method, the purpose of which is to extract and identify the target area in the grayscale image and improve the accuracy when analyzing and mining the target image.

[0004] To achieve the above object, according to a first aspect of the present invention, a method for image format conversion is provided, which comprises:

[0005] Save the bitmap data entity of the bmp format file as a 16-bit unsigned integer x;

[0006] Using the first conversion formula Normalizing the 16-bit unsigned integer x to obtain a normalized integer x', wherein max(x) represents the maximum value of the 16-bit unsigned integer x;

[0007] The normalized integer x' is converted into an 8-bit unsigned integer y, and the second conversion formula is used The 8-bit unsigned integer y is nonlinearly converted, and the conversion result z is saved as a png format file.

[0008] Preferably, the bmp format file includes a file header, a bitmap information header and a bitmap data entity, wherein the file header occupies 14 bits, the bitmap information header occupies 40 bits, and the bitmap data entity is data read from the 55th bit of the bmp format file.

[0009] Preferably, the bitmap data entity occupies 14 bits.

[0010] According to a second aspect of the present invention, there is provided an image format conversion module, comprising:

[0011] A bitmap entity data storage unit is used to obtain the bitmap data entity of a bmp format file and save it as a 16-bit unsigned integer x;

[0012] A first conversion unit is used to use a first conversion formula Normalizing the 16-bit unsigned integer x to obtain a normalized integer x', wherein max(x) represents the maximum value of the 16-bit unsigned integer x;

[0013] The second conversion unit is used to convert the normalized integer x' into an 8-bit unsigned integer y, and use the second conversion formula The 8-bit unsigned integer y is nonlinearly converted, and the conversion result z is saved as a png format file.

[0014] Preferably, the bitmap data entity occupies 14 bits.

[0015] According to a third aspect of the present invention, there is provided an image recognition device, comprising:

[0016] An image format conversion module, used for receiving a bmp format file and converting the bmp format file into a png format file, wherein the image format conversion module is the above-mentioned image format conversion module;

[0017] The target detection module is used to obtain the png format file and output a significant feature map S;

[0018] A target region interception module is used to intercept the target region in the salient feature map S as a recognition image;

[0019] The target recognition module is used to obtain the recognition image and recognize the target, and output the recognition result.

[0020] Preferably, the target detection module comprises a neural network structure:

[0021] The first convolutional network is used to receive png format files and output three-way images X in three convolutional layers respectively. i , i=1,2,3;

[0022] Focus on the network, image X i Connected to the i-th attention network, image X i After the deconvolution layer and convolution layer of the corresponding attention network, the output salient feature map A i , significant feature map A iAfter the pooling layer of the corresponding attention network, the salient feature map M is output i , the attention network is used to calculate the image Y i =X i *M i +X i ;

[0023] First joint network, image Y i After passing through the deconvolution layer of the first joint network, we get image Z i ; Image Z i After the convolution layer, we get the image S i , the first joint network integrates each image Z through the feature joint network i After that, the convolutional layer outputs the image C1;

[0024] The second joint network is used to i Upsample to get B i , the second joint network integrates each image Z through the feature joint network i , image C1 and each image B i , and outputs image C2 through the convolutional layer. It is also used to add images B1, B2, B3, C1 and C2 to output the salient feature map S.

[0025] Preferably, the target area interception module is used to locate the maximum values ​​of the pixel points in four directions in the significant feature map S, draw a rectangle with the maximum value as the boundary, and crop it and save it as the recognition picture.

[0026] Preferably, the target recognition module comprises a neural network structure:

[0027] The second convolutional network includes three convolutional blocks. The recognition image passes through the three convolutional blocks in sequence and outputs the convolution results. The first two convolutional blocks are composed of a convolutional layer, an activation layer, and a pooling layer, and the third convolutional block is composed of a convolutional layer and an activation layer.

[0028] Two fully connected layers. The convolution result output by the second convolutional network is sequentially reduced to the required classification value after two fully connected layers.

[0029] Softmax layer: The output of the fully connected layer passes through the softmax layer to obtain the predicted probability of each category.

[0030] According to a fourth aspect of the present invention, there is provided an image detection and recognition method, which comprises obtaining a bmp format file and inputting the bmp format file into any of the above-mentioned image recognition devices to obtain an image recognition result.

[0031] Since 16-bit bmp format files cannot be displayed normally on ordinary display devices, especially for infrared imaging, the contrast between image pixels is not enough, and it is difficult to see the details of the target clearly. In the image format conversion method and image format conversion model of the present application, the 16-bit bitmap data entity of the bmp format file is normalized and non-linearly converted, and saved as an 8-bit png format file, and the normalization and non-linear conversion process can effectively improve the image balance, avoid image display problems, and use conventional equipment to fully display the content of the 16-bit bmp picture for human eye observation. At the same time, the image recognition device in the present application performs target detection and recognition based on the converted png format picture, which can also improve the accuracy of detection. And the target detection module used in the image recognition device solves the problem of difficult target detection and recognition due to small targets and strong background noise by integrating multi-channel features, and further improves the target detection and recognition accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 is a flowchart of the steps of the image format conversion method in one embodiment of the present application;

[0033] Figure 2 is a structural diagram of an image format conversion module in one embodiment of the present application;

[0034] Figure 3 is a structural schematic diagram of an image recognition device in one embodiment of the present application;

[0035] Figure 4 is a network architecture diagram of a target detection module in an embodiment of the present application;

[0036] Figure 5 This is a network architecture diagram of a target identification module in an embodiment of the present application;

[0037] Figure 6 It is a data processing flow chart of the image recognition method in one embodiment of the present application. DETAILED DESCRIPTION

[0038] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0039] The present application relates to a method for image format conversion, such as Figure 1 The figure is a flowchart of a method for image format conversion in an embodiment of the present application. The method for image format conversion includes the following steps:

[0040] Step S110: Save the bitmap data entity of the bmp format file as a 16-bit unsigned integer x.

[0041] In one embodiment, the bmp file structure is analyzed, the file header occupies 14 bits, the bitmap information header occupies 40 bits, and the palette information is ignored, so the bitmap data entity starts from the 55th bit of the bmp data. The bmp file is unpacked to obtain the required information, and then the specific image data of the bitmap is read according to the width and height of the bitmap. After successfully reading the image data entity x in the bmp file, the 14-bit data is saved as a 16-bit unsigned integer closest to its bit number.

[0042] Step S120: Using the first conversion formula The 16-bit unsigned integer number x is normalized to obtain the normalized integer number x', where max(x) represents the maximum value of the 16-bit unsigned integer number x.

[0043] Through normalization, the 16-bit unsigned integer is normalized to between (0, 1].

[0044] Step S130: Convert the normalized integer x' into an 8-bit unsigned integer y, and use the second conversion formula Perform nonlinear transformation on the 8-bit unsigned integer y and save the transformation result z as a png format file.

[0045] Each sampled pixel value of an 8-bit image is between [0, 255], that is, it contains 256 levels of gray. The 8-bit data is nonlinearly transformed to enhance the visual effect of the target.

[0046] Accordingly, the present application also relates to an image format conversion module, such as Figure 2 The figure shows a schematic diagram of an image format conversion module in one embodiment of the present application. The image format conversion module includes a bitmap entity data storage unit, a first conversion unit and a second conversion unit.

[0047] The bitmap entity data storage unit is used to obtain the bitmap data entity of the bmp format file and save it as a 16-bit unsigned integer x. Specifically, in the bmp file structure, the file header occupies 14 bits, the bitmap information header occupies 40 bits, and the bitmap entity data storage unit obtains the 14-bit bitmap data entity starting from the 55th bit of the bmp data and saves it as a 16-bit unsigned integer x.

[0048] The first conversion unit is used to use the first conversion formula The 16-bit unsigned integer x is normalized to obtain a normalized integer x', wherein max(x) represents the maximum value of the 16-bit unsigned integer x; through the normalization process, the 16-bit unsigned integer is normalized to between (0, 1].

[0049] The second conversion unit is used to convert the normalized integer x' into an 8-bit unsigned integer y, and use the second conversion formula Perform nonlinear transformation on the 8-bit unsigned integer y to enhance the visual effect of the target, and save the transformation result z as a png format file.

[0050] The present application also relates to an image recognition device, such as Figure 3 The figure is a schematic diagram of the structure of an image recognition device in one embodiment of the present application, wherein the image recognition device includes an image format conversion module, a target detection module, a target area capture module and a target recognition module.

[0051] Among them, the image format conversion module is the image format conversion module introduced above. The specific structure is referred to the above introduction and will not be repeated here.

[0052] Among them, the target detection module is used to obtain png format files and output the saliency feature map S.

[0053] The target region capture module is used to capture the target region in the salient feature map S as a recognition image.

[0054] Among them, the target recognition module is used to obtain the recognition picture and recognize the target, and output the recognition result.

[0055] In one embodiment, if Figure 4 As shown, the target detection module includes a neural network structure, which specifically includes a first convolutional network, three attention networks, a first joint network and a second joint network.

[0056] Among them, the first convolution network is used to receive png format files. Specifically, after the first five convolution layers, the network is divided into three paths, each of which extracts the features of the target at different resolutions and outputs three images X1, image X2 and image X3. In a specific embodiment, the input image is a monochrome image, and the resolution of the read image needs to be adjusted to 384 in height and 480 in width, that is, the ratio of height to width is 0.8. The input image outputs image X1 in conv5-3, image X2 in conv5-4, and image X3 in conv5-5.

[0057] Among them, the three attention networks correspond to the three outputs one by one, and each attention network has a deconvolution layer, a convolution layer, and a pooling layer. For image X1, after two 4×4 deconvolution networks of the first attention network, in order to obtain the initial salient feature map A1 with a spatial scale of 96×120, a 1×1 nonlinear Sigmoid convolution layer is used after the last deconvolution layer, and then a pooling operation with a step size of 4 is applied to A1 to obtain the salient map M1 of the same spatial size. Then, the first attention network calculates the above features to obtain the image Y1=X1*M1+X1. Image X2 obtains the salient feature map A1 and the salient feature map M1 and Y2=X2*M2+X2 in the same way; image X3 obtains the salient feature map A3 and the salient feature map M3 and Y3=X3*M3+X3 in the same way.

[0058] The first joint network includes a deconvolution network, a feature joint layer, and a convolution layer. Image Y1 passes through the first deconvolution network of the first joint network to obtain image Z1, and image Z1 passes through the convolution layer to obtain image S1; image Y2 passes through the second deconvolution network of the first joint network to obtain image Z2, and image Z2 passes through the convolution layer to obtain image S2; image Y3 passes through the third deconvolution network of the first joint network to obtain image Z3, and image Z3 passes through the convolution layer to obtain image S3. The first joint network integrates images Z1, Z2, and Z3 through the feature joint layer, and outputs image C1 through the convolution layer, C1 = f 1×1 (concat[Z1,Z2,Z3]).

[0059] The second joint network includes a feature joint network, a convolution layer, and an activation layer. The second joint network upsamples images A1, A2, and A2 respectively to obtain corresponding B1, B2, and B3. The second joint network integrates each image Z through the feature joint network. i , image C1 and each image B i , and outputs image C2 through the convolution layer. It is also used to add images B1, B2, B3, C1 and C2, output the salient feature map S, and obtain the pixel values ​​of the salient area where the target is located.

[0060] It should be noted that the neural network of the target detection module needs to be trained multiple times so that the value of the loss function between the output salient area pixel value and the actual pixel value is within a tolerable range.

[0061] In one embodiment, the target region interception module obtains the output saliency feature map S, finds the maximum value points in the upper, lower, left and right directions among the pixels in the saliency region, and the rectangle drawn with the maximum value as the boundary is the local image block of the target region in the image. The absolute coordinates of the local image block are written into a text file with the same name as the original image, including the upper boundary, the left boundary, the right boundary and the lower boundary, and saved in the label_position folder. At the same time, the local image block is cropped and stored in a png format with the same name as the original image, and saved in the patch folder as the recognition image of the input target recognition module.

[0062] In one embodiment, if Figure 5 As shown in the figure, the target recognition module includes a neural network structure, which includes a second convolutional network, two fully connected layers and a softmax layer. Specifically, the first two convolutional layers consist of convolution, activation and pooling operations, and the last convolutional layer consists of convolution and activation operations. After the recognition image (32×64) passes through three convolutional layers, a 16×32×64 feature is obtained, which can be expanded to obtain a 32,768-dimensional vector. According to the number of categories required for the task, after passing through two fully connected layers, the dimensions are reduced to the values ​​of the required categories. A softmax layer is added to the end of the network, and the predicted probability of each category is obtained after passing through the softmax layer.

[0063] It should be noted that the neural network of the target recognition module needs to be trained multiple times so that the cross entropy loss between the output prediction probability and the actual true label is within a tolerable range. Among them, the training set used to train the target recognition module and the target detection module is a file converted from a bmp format file to a png format file using the above method. The training of the target recognition module and the training of the target detection module can be performed separately.

[0064] The present application also relates to an image detection and recognition method, which only needs to obtain a bmp format file and input the bmp format file into the above-mentioned image recognition device to obtain an image recognition result. Figure 6 As shown in the figure, after the image recognition device receives the bmp format file, it converts the bmp format file into a png format file through the image format conversion module, and then performs target detection through the target detection module, outputs the significant feature map S, and then uses the target area capture module to capture the target area in the significant feature map S as the recognition image, and finally recognizes the target through the target recognition module and outputs the recognition result.

[0065] In summary, in the image format conversion method and image format conversion model of the present application, the 16-bit bitmap data entity of the bmp format file is normalized and nonlinearly converted, and saved as an 8-bit png format file, and the normalization and nonlinear conversion process can effectively improve the balance of the image, avoid the problem of image display, and use conventional equipment to fully display the content of the 16-bit bmp picture for human eye observation. At the same time, the image recognition device in the present application performs target detection and recognition based on the converted png format picture, which can also improve the detection accuracy. And the target detection module used in the image recognition device solves the problem of difficult target detection and recognition due to small targets and strong background noise by integrating multi-channel features, and further improves the target detection and recognition accuracy.

[0066] It will be easily understood by those skilled in the art that the above are only preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for image format conversion, characterized in that: include: Save the bitmap data entity of the bmp format file as a 16-bit unsigned integer x; Using the first conversion formula Normalizing the 16-bit unsigned integer x to obtain a normalized integer x', wherein max(x) represents the maximum value of the 16-bit unsigned integer x; The normalized integer x' is converted into an 8-bit unsigned integer y, and the second conversion formula is used The 8-bit unsigned integer y is nonlinearly converted, and the conversion result z is saved as an 8-bit png format file.

2. The image format conversion method according to claim 1, characterized in that: The bmp format file includes a file header, a bitmap information header and a bitmap data entity, wherein the file header occupies 14 bits, the bitmap information header occupies 40 bits, and the bitmap data entity is data read from the 55th bit of the bmp format file.

3. The image format conversion method according to claim 2, characterized in that: The bitmap data entity occupies 14 bits.

4. An image format conversion module, characterized in that: include: A bitmap entity data storage unit is used to obtain the bitmap data entity of a bmp format file and save it as a 16-bit unsigned integer x; A first conversion unit is used to use a first conversion formula Normalizing the 16-bit unsigned integer x to obtain a normalized integer x', wherein max(x) represents the maximum value of the 16-bit unsigned integer x; The second conversion unit is used to convert the normalized integer x' into an 8-bit unsigned integer y, and use the second conversion formula The 8-bit unsigned integer y is nonlinearly converted, and the conversion result z is saved as a png format file.

5. The image format conversion module according to claim 4, characterized in that: The bitmap data entity occupies 14 bits.

6. An image recognition device, characterized in that: include: An image format conversion module, used for receiving a bmp format file and converting the bmp format file into a png format file, wherein the image format conversion module is the image format conversion module according to claim 4 or 5; The target detection module is used to obtain the png format file and output a significant feature map S; A target region interception module is used to intercept the target region in the salient feature map S as a recognition image; The target recognition module is used to obtain the recognition image and recognize the target, and output the recognition result.

7. The image recognition device according to claim 6, characterized in that: The target detection module includes a neural network structure: The first convolutional network is used to receive png format files and output three-way images X in three convolutional layers respectively. i , i=1,2,3; Focus on the network, image X i Connected to the i-th attention network, image X i After the deconvolution layer and convolution layer of the corresponding attention network, the output salient feature map A i , significant feature map A i After the pooling layer of the corresponding attention network, the salient feature map M is output i , the attention network is used to calculate the image Y i =X i *M i +X i ; First joint network, image Y i After passing through the deconvolution layer of the first joint network, we get image Z i ; Image Z i After the convolution layer, we get the image S i , the first joint network integrates each image Z through the feature joint network i After that, the convolutional layer outputs the image C1; The second joint network is used to i Upsample to get B i , the second joint network integrates each image Z through the feature joint network i , image C1 and each image B i , and outputs image C2 through the convolutional layer. It is also used to add images B1, B2, B3, C1 and C2 to output the salient feature map S.

8. The image recognition device according to claim 6, characterized in that: The target area interception module is used to locate the maximum values ​​of the pixel points in four directions in the significant feature map S, draw a rectangle with the maximum value as the boundary, and cut and save it as a recognition picture.

9. The image recognition device according to claim 6, characterized in that: The target recognition module includes a neural network structure: The second convolutional network includes three convolutional blocks. The recognition image passes through the three convolutional blocks in sequence and outputs the convolution results. The first two convolutional blocks are composed of a convolutional layer, an activation layer, and a pooling layer, and the third convolutional block is composed of a convolutional layer and an activation layer. Two fully connected layers. The convolution result output by the second convolutional network is sequentially reduced to the required classification value after two fully connected layers. Softmax layer,The output of the fully connected layer passes through the softmax layer to obtain the predicted probability of each category.

10. An image detection and recognition method, characterized in that: The method comprises obtaining a bmp format file and inputting the bmp format file into the image recognition device described in any one of claims 6 to 9 to obtain an image recognition result.

Images