Multi-category target recognition method based on multi-scale prediction CNN and Loongson chip
By building a convolutional neural network on multi-scale background prediction CNN and Loongson chips, combined with Gaussian weighted non-maximum suppression algorithm, the accuracy and speed problems in multi-category target recognition are solved, and a more efficient recognition effect is achieved.
Patent Information
- Application Number
- CN202010806204.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-08-12
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2040-08-12
AI Technical Summary
Existing multi-objective recognition algorithms have accuracy and speed problems when identifying multi-category targets, especially when there are mutual occlusion and scale changes in complex situations, false detection or missed detection is prone to occur.
A multi-category object recognition method based on multi-scale prediction CNN and Loongson chip was designed. By building a multi-scale background prediction convolutional neural network, combining Gaussian weighted non-maximum suppression algorithm, training using COCO data sets is optimized to optimize the network structure. To improve identification accuracy and speed.
The accuracy and speed of multi-category target recognition has been improved, and the problem of low accuracy and slow recognition speed of multi-category target recognition is solved. It is highly portable.
Smart Images

Figure CN111950451B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of multi-category target recognition and deep learning, and in particular to a multi-category target recognition method based on multi-scale prediction CNN and Loongson chip. Background Art
[0002] With the continuous development of computer vision, deep learning has been continuously expanding in the field of multi-target recognition in visual processing. However, some multi-target recognition algorithms still have problems such as inaccurate target recognition and slow recognition speed. In order to meet the requirements of multi-category target recognition in the visual field and the needs of transplanting deep learning models into miniaturized embedded systems, it is of great significance to study fast and accurate multi-category target recognition technology based on domestic Loongson chips.
[0003] At present, there are two main methods for multi-target recognition. One is through the template matching method in image processing, that is, after selecting a template, search for the sub-image with the highest degree of matching with the template in the image to be searched, and then perform similarity matching between the template and the sub-image to complete the recognition. This matching method has a large amount of calculation and is difficult to apply to the field of real-time multi-category target recognition that requires high target recognition speed. The second is through the deep learning method, using a neural network model to complete the recognition of multi-category targets. However, the deep learning method is currently used for multi-category target recognition. Due to the problems of mutual occlusion and scale changes of targets in complex situations, the neural network model is prone to misdetection or missed detection of targets. Summary of the invention
[0004] Purpose of the invention: In order to overcome the shortcomings of the prior art, a multi-category target recognition method based on multi-scale prediction CNN and Loongson chip is proposed to address the problem of low accuracy and speed of multi-category target recognition in current target recognition technology. A multi-scale background prediction convolutional neural network model and algorithm are designed to improve the speed and accuracy of multi-category target recognition.
[0005] Technical solution: To achieve the above purpose, the present invention provides a multi-category target recognition method based on multi-scale prediction CNN and Loongson chip, comprising the following steps:
[0006] S1: Input the target image into the built multi-scale background prediction convolutional neural network, output the binary classification results of the object and background, and the candidate box size, position and category of the multi-scale target;
[0007] S2: The candidate boxes classified as objects are subjected to Gaussian weighted non-maximum suppression algorithm to remove duplicate candidate boxes;
[0008] S3: Select a network optimizer, use the COCO dataset as a training dataset for the multi-scale background prediction convolutional neural network, fully train the multi-scale background prediction convolutional neural network, and obtain a trained multi-scale background prediction convolutional neural network;
[0009] S4: The visual camera collects video files, then converts the collected video files into color images frame by frame, and unifies the size of the color images;
[0010] S5: Transplant the trained multi-scale background prediction convolutional neural network model into the embedded system based on Loongson chip to complete the recognition of multi-category targets.
[0011] Furthermore, the multi-scale background prediction convolutional neural network in step S1 includes three parts, and the specific construction method is as follows:
[0012] The first part of the construction:
[0013] The first part includes an input layer and three convolutional layers, which are used to extract convolutional features from the data. The input of the first part comes from the COCO dataset. Each image in the COCO dataset is sent to the input layer. The first convolutional layer of the first part is after the input layer.
[0014] For the first convolution layer of the first part, the output size of the first convolution layer of the first part is calculated by the output size calculation formula of the convolution layer;
[0015] After the first convolutional layer of the first part, a linear rectification function is used as the activation function, and the data after the activation function is sent to the second convolutional layer of the first part;
[0016] The second convolutional layer of the first part also uses the linear rectification function as the activation function, and the data after the activation function is sent to the third convolutional layer of the first part;
[0017] The second part of the construction:
[0018] The second part includes a convolutional layer and a fully connected layer; the output of the third convolutional layer of the first part is used to input the first convolutional layer of the second part;
[0019] After the first convolutional layer of the second part, a linear rectification function is used as the activation function, and the data after the activation function is sent to the first fully connected layer of the second part. The first fully connected layer of the second part has 2 neurons, corresponding to the two target categories of object and background. The first fully connected layer of the second part uses a softmax function to obtain the confidence coefficients of the two target categories. The confidence coefficient refers to the probability of the target category; a threshold K1 is set. If the confidence coefficient of the object is greater than K1 or the confidence coefficient of the background is less than K1, the output of the convolutional layer of the second part is sent to each convolutional layer of the third part;
[0020] The third part of the construction:
[0021] The third part includes six convolutional layers and two fully connected layers. The output of the convolutional layer of the second part is sent to the first convolutional layer of the third part; the output of the convolutional layer of the second part uses the linear rectification function as the activation function, and the data after the activation function is sent to the second convolutional layer of the third part; the output of the convolutional layer of the second part also uses the linear rectification function as the activation function, and the data after the activation function is sent to the third convolutional layer of the third part; the output of the convolutional layer of the second part also uses the linear rectification function as the activation function, and the data after the activation function is sent to the fourth convolutional layer of the third part; the output of the convolutional layer of the second part also uses the linear rectification function as the activation function, and the data after the activation function is sent to the fifth convolutional layer of the third part; the output of the convolutional layer of the second part also uses the linear rectification function as the activation function, and the data after the activation function is sent to the sixth convolutional layer of the third part;
[0022] After each convolutional layer of the third part, a linear rectification function is used as an activation function, and the data after the activation function is sent to the first fully connected layer of the third part. The first fully connected layer of the third part has N1 neurons, corresponding to N1 target categories. The first fully connected layer of the third part uses a softmax function to obtain the confidence coefficients of the N1 target categories. The confidence coefficient refers to the probability of the target category; at the same time, the first convolutional layer of the third part after the activation function is sent to the second fully connected layer of the third part. The second fully connected layer of the third part has N2 neurons, corresponding to the candidate box positions of the N1 target categories. Each candidate box position is determined by the coordinates (x, y, l, h), where x represents the horizontal distance from the upper left corner vertex of the candidate box to the upper left corner vertex of each image in each image of the multi-scale background prediction convolutional neural network input layer, y represents the vertical distance from the upper left corner vertex of the candidate box to the upper left corner vertex of each image in each image of the multi-scale background prediction convolutional neural network input layer, l represents the length of the candidate box, and h represents the height of the candidate box.
[0023] Furthermore, the step S2 specifically includes the following steps:
[0024] A1: Sort the 1000 candidate boxes in each target in descending order according to the corresponding confidence coefficient values.
[0025] A2: Select the candidate box corresponding to the maximum confidence coefficient, and calculate the IoU value between the remaining candidate boxes and the candidate box corresponding to the maximum confidence coefficient. The IoU value calculation formula is as follows:
[0026] IoU = S2 / S3
[0027] Among them, S2 represents the intersection area of the two candidate boxes, and S3 represents the union area of the two candidate boxes;
[0028] A3: Set a threshold T, and calculate the new confidence coefficient according to the following formula:
[0029]
[0030] Among them, M is the candidate box corresponding to the maximum confidence coefficient, s i is the confidence coefficient of the i-th candidate box, b i is the i-th candidate box, IOU(M, b i ) is the IOU value of M and b i , σ is a constant, e = 2.71828, and G is the new confidence coefficient obtained by calculating the candidate box corresponding to the maximum confidence coefficient through the above formula;
[0031] If G < T, then delete the candidate box corresponding to the maximum confidence coefficient;
[0032] A4: Continue to select a candidate box corresponding to the maximum confidence coefficient from the remaining candidate boxes, and repeat steps A1 to A3 until only one candidate box remains. The coordinate position (x, y, l, h) of the candidate box is the actual position of the target, and the target category corresponding to the maximum confidence coefficient is the actual category of the target.
[0033] Furthermore, the specific steps of the said step S3 are as follows:
[0034] B1: Arbitrarily select 10,000 real images in the COCO dataset, and use bilinear interpolation to obtain images with a size of 500 * 500 for the 10,000 images;
[0035] B2: Divide the 1000 images into a training set and a test set according to a ratio of 4:1. During the training process, take every 100 images out of the 1000 images as a training unit;
[0036] B3: The loss function uses the cross entropy loss function, and the network optimizer uses the Adam optimizer to fully train the multi-scale background prediction convolutional neural network to obtain a trained multi-scale background prediction convolutional neural network model.
[0037] Furthermore, the step S4 specifically includes the following steps:
[0038] C1: Visual camera collects video files;
[0039] C2: Capture an image every 3 frames of the acquired video file;
[0040] C3: If the captured image is a grayscale image, it is converted into a color image using the grayscale-color conversion method;
[0041] C4: Perform bilinear interpolation on the color image to convert it into a size of 500*500, and obtain a color image of uniform size.
[0042] Furthermore, the step S5 specifically includes the following steps:
[0043] D1: Input the obtained color image of uniform size into the multi-scale background prediction convolutional neural network model to identify the target;
[0044] D2: The multi-scale background prediction convolutional neural network model generates 1,000 candidate boxes for each target, each of which has a corresponding coordinate position (x, y, l, h) and the confidence coefficient of the target category;
[0045] D3: Call the Improcess and Caffemodel library files of OpenCV to transplant the trained multi-scale background prediction convolutional neural network model to the embedded system based on the Loongson chip. Use the dynamic link library to package the library files used for program operation, execute the written script files, and complete the recognition function of multi-category targets.
[0046] Beneficial effects: Compared with the prior art, the present invention fully considers the characteristics of fast and accurate target recognition, designs a multi-scale background prediction convolutional neural network, and uses a multi-scale network model to identify multi-category targets, thereby improving the recognition accuracy and speed of multi-category targets, and solves the problems of low recognition accuracy and slow recognition speed of multi-category targets in current target recognition technology. It also has high portability and broad application prospects. BRIEF DESCRIPTION OF THE DRAWINGS
[0047] Figure 1 It is a schematic diagram of the workflow of a multi-category target recognition method based on a multi-scale prediction CNN and a Loongson chip provided by an embodiment of the present invention;
[0048] Figure 2 It is a multi-scale background prediction convolutional neural network model diagram provided by an embodiment of the present invention;
[0049] Figure 3 This is a physical picture of the Loongson 2K1000 embedded system based on the Loongson chip provided by an embodiment of the present invention;
[0050] Figure 4 This is the multi-category target recognition result based on multi-scale prediction CNN and Loongson chip provided by the embodiment of the present invention Figure 1 ;
[0051] Figure 5 This is the multi-category target recognition result based on multi-scale prediction CNN and Loongson chip provided by the embodiment of the present invention Figure 2 ;
[0052] Figure 6 This is the multi-category target recognition result based on multi-scale prediction CNN and Loongson chip provided by the embodiment of the present invention Figure 3 ;
[0053] Figure 7 This is the multi-category target recognition result based on multi-scale prediction CNN and Loongson chip provided by the embodiment of the present invention Figure 4 . DETAILED DESCRIPTION
[0054] The present invention is further explained below in conjunction with the accompanying drawings and specific embodiments. It should be understood that these embodiments are only used to illustrate the present invention and are not used to limit the scope of the present invention. After reading the present invention, various equivalent forms of modifications to the present invention by those skilled in the art all fall within the scope defined by the claims attached to this application.
[0055] The present invention provides a multi-category target recognition method based on multi-scale prediction CNN and Loongson chip, such as Figure 1 As shown, it includes the following steps:
[0056] Step 1: Build a multi-scale background prediction convolutional neural network, input the target image into the multi-scale background prediction convolutional neural network, and output the binary classification results of the object and background as well as the candidate box size, position and category of the multi-scale target;
[0057] Step 2: The candidate boxes classified as objects are subjected to Gaussian weighted non-maximum suppression algorithm to remove duplicate candidate boxes.
[0058] Step 3, select a network optimizer, use the COCO dataset as a training dataset for the multi-scale background prediction convolutional neural network, and fully train the multi-scale background prediction convolutional neural network to obtain a trained multi-scale background prediction convolutional neural network;
[0059] Step 4, collecting video files through a visual camera, then converting the collected video files into color images frame by frame, and unifying the size of the color images;
[0060] Step 5: Transplant the trained multi-scale background prediction convolutional neural network model into the Loongson 2K1000 embedded system based on the Loongson chip to complete the recognition of multiple categories of targets.
[0061] like Figure 2 As shown, this embodiment provides a multi-scale background prediction convolutional neural network structure diagram. In step 1, the multi-scale background prediction convolutional neural network includes three parts, and the construction process is as follows:
[0062] The first part includes an input layer and three convolutional layers, which are used to extract convolutional features from the data. The input of the first part comes from the COCO dataset. The size of each image in the COCO dataset is 500*500, and each image is sent to the input layer; the input layer is followed by the first convolutional layer of the first part. The first convolutional layer of the first part uses 16 3*3 convolution kernels, a step size of 1, and a padding number of 1. The output size calculation formula of the convolutional layer is as follows:
[0063]
[0064] Among them, Z is the length of the convolution output data, W is the length of the convolution input data, P is the number of padding, F is the length of the convolution kernel, and S represents the step size;
[0065] For the first convolutional layer of the first part, the output size of the convolutional layer is calculated by formula (1) to obtain the output size of the first convolutional layer of the first part is 500*500*16;
[0066] After the first convolution layer of the first part, a linear rectification function is used as the activation function, and the data after the activation function is sent to the second convolution layer of the first part. The second convolution layer of the first part uses 32 3*3 convolution kernels, the step size is 1, and the padding number is 1. According to the output size of the convolution layer, the output size of the second convolution layer of the first part is calculated by formula (1). It is 500*500*32;
[0067] The second convolutional layer of the first part also uses the linear rectification function as the activation function, and the data after the activation function is sent to the third convolutional layer of the first part. The third convolutional layer of the first part uses 64 3*3 convolution kernels, the step size is 2, and the padding number is 1. According to the output size of the convolutional layer, the output size of the third convolutional layer of the first part is calculated by formula (1). It is 250*250*64;
[0068] The second part includes a convolutional layer and a fully connected layer. The output of the third convolutional layer of the first part is used to input the first convolutional layer of the second part. The first convolutional layer of the second part uses 128 3*3 convolution kernels, a step size of 1, and a padding number of 1. According to the output size of the convolutional layer, the output size of the first convolutional layer of the second part is calculated by formula (1) to be 250*250*128.
[0069] After the first convolutional layer of the second part, a linear rectification function is used as the activation function, and the data after the activation function is sent to the first fully connected layer of the second part. The first fully connected layer of the second part has 2 neurons, corresponding to the two target categories of object and background. The first fully connected layer of the second part uses a softmax function to obtain the confidence coefficients of the two target categories. The confidence coefficient refers to the probability of the target category; a threshold K1 is set. If the confidence coefficient of the object is greater than K1 or the confidence coefficient of the background is less than K1, the output of the convolutional layer of the second part is sent to each convolutional layer of the third part;
[0070] The third part includes six convolutional layers and two fully connected layers. The output of the convolutional layer of the second part is sent to the first convolutional layer of the third part. The first convolutional layer of the third part uses 64 3*3 convolution kernels, a step size of 1, and a padding number of 1. The calculation formula based on the output size of the convolutional layer is as follows:
[0071] Z1=(W1-1)×S1-2×P1+F1 (2)
[0072] Among them, Z1 is the length of the convolution output data, W1 is the length of the convolution input data, P1 is the padding number, F1 is the length of the convolution kernel, and S1 represents the step size. According to the output size of the convolution layer, the output size of the first convolution layer in the third part is 250*250*64. The output of the convolution layer in the second part also uses the linear rectification function as the activation function, and the data after the activation function is sent to the second convolution layer in the third part. The second convolution layer in the third part uses 64 4*4 convolution kernels, the step size is 2, and the padding number is 1. According to the convolution layer, The output size is calculated by formula (2). The output size of the second convolutional layer of the third part is 500*500*64. The output of the convolutional layer of the second part also uses the linear rectification function as the activation function. The data after the activation function is sent to the third convolutional layer of the third part. The third convolutional layer of the third part uses 64 5*5 convolution kernels, the step size is 3, and the padding number is 1. According to the output size of the convolutional layer, the output size of the third convolutional layer of the third part is calculated by formula (2). It is 750*750*64. The output of the convolutional layer of the second part also uses the linear rectification function. As the activation function, the data after the activation function is sent to the fourth convolution layer of the third part. The fourth convolution layer of the third part uses 64 6*6 convolution kernels, the step size is 4, and the padding number is 1. According to the output size of the convolution layer, the output size of the fourth convolution layer of the third part is calculated by formula (2). The output size of the fourth convolution layer of the third part is 3000*3000*64; the output of the convolution layer of the second part also uses the linear rectification function as the activation function, and the data after the activation function is sent to the fifth convolution layer of the third part. The fifth convolution layer of the third part uses 64 7*7 convolution kernels, the step size is 5, The padding amount is 1. According to the output size calculation formula (2) of the convolution layer, the output size of the fifth convolution layer in the third part is 15000*15000*64. The output of the convolution layer in the second part also uses the linear rectification function as the activation function, and the data after the activation function is sent to the sixth convolution layer in the third part. The sixth convolution layer in the third part uses 64 8*8 convolution kernels, the step size is 6, and the padding amount is 1. According to the output size of the convolution layer, the output size of the sixth convolution layer in the third part is calculated by formula (2). It is 90000*90000*64.
[0073] After each convolutional layer of the third part, a linear rectifier function is used as an activation function, and the data after the activation function is sent to the first fully connected layer of the third part. The first fully connected layer of the third part has 1000 neurons, corresponding to 1000 target categories. The first fully connected layer of the third part uses a softmax function to obtain the confidence coefficients of the 1000 target categories. The confidence coefficient refers to the probability of the target category. At the same time, the first convolutional layer of the third part after the activation function is sent to the second fully connected layer of the third part. The second fully connected layer of the third part has 4000 neurons, corresponding to the candidate box positions of the 1000 target categories. The position of each candidate box is determined by the coordinates (x, y, l, h), where x represents the horizontal distance from the upper left corner vertex of the candidate box to the upper left corner vertex of each image in each image of the multi-scale background prediction convolutional neural network input layer, y represents the vertical distance from the upper left corner vertex of the candidate box to the upper left corner vertex of each image in each image of the multi-scale background prediction convolutional neural network input layer, l represents the length of the candidate box, and h represents the height of the candidate box.
[0074] In this embodiment, step 2 specifically includes the following steps:
[0075] Step 2-1: sort the 1000 candidate boxes in each target from large to small according to the corresponding confidence coefficients;
[0076] Step 2-2, select the candidate box corresponding to the maximum confidence coefficient, and calculate the IoU value between the remaining candidate boxes and the candidate box corresponding to the maximum confidence coefficient. The IoU value calculation formula is as follows:
[0077] IoU=S2 / S3 (3)
[0078] Among them, S2 represents the intersection of the areas of the two candidate boxes, and S3 represents the union of the areas of the two candidate boxes;
[0079] Step 2-3, set a threshold T and calculate the new confidence coefficient according to formula (4):
[0080]
[0081] Where M is the candidate box corresponding to the maximum confidence coefficient, s i is the confidence coefficient of the i-th candidate box, b i For the i-th candidate box, IOU(M, b i ) is M and b i The IOU value of σ is a constant (σ is generally taken as 0.6), e = 2.71828, and G is the new confidence coefficient of the candidate box corresponding to the maximum confidence coefficient calculated by formula (4);
[0082] If G < T, then delete the candidate box corresponding to the maximum confidence coefficient;
[0083] Step 2-4: Continue to select a candidate box corresponding to the maximum confidence coefficient from the remaining candidate boxes, and repeat steps 2-1 to 2-3 until only one candidate box remains. The coordinate position (x, y, l, h) of the candidate box is the actual position of the target, and the target category corresponding to the maximum confidence coefficient is the actual category of the target.
[0084] In this embodiment, step 3 specifically includes the following steps:
[0085] Step 3-1: Arbitrarily select 10,000 real images in the COCO dataset, and use bilinear interpolation to obtain images with a size of 500 * 500 from the 10,000 images;
[0086] Step 3-2: Divide the 10,000 images into a training set and a test set according to a ratio of 4:1. During the training process, take every 100 images out of the 10,000 images as a training unit;
[0087] Step 3-3: Use the cross-entropy loss function as the loss function and the Adam optimizer as the network optimizer to fully train the multi-scale background prediction convolutional neural network to obtain a trained multi-scale background prediction convolutional neural network model.
[0088] In this embodiment, step 4 specifically includes the following steps:
[0089] Step 4-1: Collect a video file through a visual camera;
[0090] Step 4-2: Intercept one image every 3 frames from the collected video file;
[0091] Step 4-3: If the intercepted image is a grayscale image, use the gray-level to color transformation method to turn it into a color image;
[0092] Step 4-4: Use bilinear interpolation on the color image to make it into a size of 500 * 500 to obtain a color image with a unified size.
[0093] In this embodiment, the physical diagram of the Loongson 2K1000 embedded system based on the Loongson chip applied in step 5 is as Figure 3 shown.
[0094] In this embodiment, step 5 specifically includes the following steps:
[0095] Step 5-1: Input the color image with a unified size obtained in step 4-4 into the multi-scale background prediction convolutional neural network model to identify multi-category targets;
[0096] Step 5-2: The multi-scale background prediction convolutional neural network model generates 1,000 candidate boxes for each target. Each candidate box has a corresponding coordinate position (x, y, l, h) and a confidence coefficient of the target category.
[0097] Step 5-3, call the Improcess and Caffemodel library files of OpenCV, transplant the trained multi-scale background prediction convolutional neural network model to the Loongson 2K1000 embedded system based on the Loongson chip, use the dynamic link library to package the library files used for program operation, execute the written script files, and complete the recognition function of multi-category targets.
[0098] In this embodiment, the above method is applied to identify multiple categories of targets, and the specific identification results are:
[0099] Figure 4 This is the effect diagram of multi-scale background prediction convolutional neural network recognition. Figure 4 The upper left corner of is the origin, the horizontal direction is the horizontal axis, and the vertical direction is the vertical axis. A coordinate system is established. The multi-scale background prediction convolutional neural network completes the recognition of the objects in the image through the trained multi-scale background prediction convolutional neural network model. Figure 4 The identification result is a bridge; Figure 5 The recognition result is a vending machine; Figure 6 The identification result is Alaskan Malamute; Figure 7 The recognition result is Teddy.
[0100] In addition, this embodiment also provides a computer storage medium, wherein the computer storage medium may store a program, and when the program is executed, the program may include some or all of the steps in each embodiment of the multi-category target recognition method based on multi-scale prediction CNN and Loongson chip provided by the present invention. The storage medium may be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM).
[0101] Those skilled in the art can clearly understand that the technology in the embodiments of the present invention can be implemented by means of software plus a necessary general hardware platform. Based on this understanding, the technical solution in the embodiments of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which can be stored in a storage medium such as ROM / RAM, a disk, an optical disk, etc., and includes a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present invention or some parts of the embodiments.
Claims
1. A multi-category target recognition method based on multi-scale prediction CNN and Loongson chip, characterized by: The steps include: S1: Input the target image into the built multi-scale background prediction convolutional neural network, output the binary classification results of the object and background, and the candidate box size, position and category of the multi-scale target; S2: The candidate boxes classified as objects are subjected to Gaussian weighted non-maximum suppression algorithm to remove duplicate candidate boxes; S3: Select a network optimizer, use the COCO dataset as a training dataset for the multi-scale background prediction convolutional neural network, fully train the multi-scale background prediction convolutional neural network, and obtain a trained multi-scale background prediction convolutional neural network; S4: The visual camera collects video files, then converts the collected video files into color images frame by frame, and unifies the size of the color images; S5: Transplant the trained multi-scale background prediction convolutional neural network model into the embedded system based on Loongson chip to complete the recognition of multi-category targets; The multi-scale background prediction convolutional neural network in step S1 consists of three parts, and the specific construction method is as follows: The first part of the construction: The first part includes an input layer and three convolutional layers, which are used to extract convolutional features from the data. The input of the first part comes from the COCO dataset, and each image in the COCO dataset is sent to the input layer; The input layer is followed by the first convolutional layer of the first part; For the first convolution layer of the first part, the output size of the first convolution layer of the first part is calculated by the output size calculation formula of the convolution layer; After the first convolutional layer of the first part, a linear rectification function is used as the activation function, and the data after the activation function is sent to the second convolutional layer of the first part; The second convolutional layer of the first part also uses the linear rectification function as the activation function, and the data after the activation function is sent to the third convolutional layer of the first part; The second part of the construction: The second part includes a convolutional layer and a fully connected layer; the output of the third convolutional layer of the first part is used to input the first convolutional layer of the second part; The first convolutional layer in the second part uses a linear rectifier function as the activation function, and the data after the activation function is sent to the first fully connected layer in the second part. The first fully connected layer in the second part has two neurons, corresponding to the two target categories of object and background. The first fully connected layer in the second part uses a softmax function to obtain the confidence coefficients of the two target categories. The confidence coefficient refers to the probability of the target category. Set a threshold K1. If the confidence coefficient of the object is greater than K1 or the confidence coefficient of the background is less than K1, the output of the convolutional layer of the second part is sent to each convolutional layer of the third part. The third part of the construction: The third part includes six convolutional layers and two fully connected layers. The output of the convolutional layer of the second part is sent to the first convolutional layer of the third part; the output of the convolutional layer of the second part uses the linear rectification function as the activation function, and the data after the activation function is sent to the second convolutional layer of the third part; the output of the convolutional layer of the second part also uses the linear rectification function as the activation function, and the data after the activation function is sent to the third convolutional layer of the third part; the output of the convolutional layer of the second part also uses the linear rectification function as the activation function, and the data after the activation function is sent to the fourth convolutional layer of the third part; the output of the convolutional layer of the second part also uses the linear rectification function as the activation function, and the data after the activation function is sent to the fifth convolutional layer of the third part; the output of the convolutional layer of the second part also uses the linear rectification function as the activation function, and the data after the activation function is sent to the sixth convolutional layer of the third part; After each convolutional layer in the third part, a linear rectification function is used as an activation function, and the data after the activation function is sent to the first fully connected layer of the third part. The first fully connected layer of the third part has N1 neurons, corresponding to N1 target categories. The first fully connected layer of the third part uses a softmax function to obtain the confidence coefficients of the N1 target categories. The confidence coefficient refers to the probability of the target category. At the same time, the first convolutional layer of the third part after the activation function is sent to the second fully connected layer of the third part. The second fully connected layer of the third part has N2 neurons, corresponding to the candidate box positions of the N1 target categories. Each candidate box position is determined by the coordinates (x, y, l, h), where x represents the horizontal distance from the upper left corner vertex of the candidate box to the upper left corner vertex of each image in each image of the multi-scale background prediction convolutional neural network input layer, y represents the vertical distance from the upper left corner vertex of the candidate box to the upper left corner vertex of each image in each image of the multi-scale background prediction convolutional neural network input layer, l represents the length of the candidate box, and h represents the height of the candidate box. Step S5 specifically includes the following steps: D1: Input the obtained color image of uniform size into the multi-scale background prediction convolutional neural network model to identify the target; D2: The multi-scale background prediction convolutional neural network model generates 1,000 candidate boxes for each target, each of which has a corresponding coordinate position (x, y, l, h) and the confidence coefficient of the target category; D3: Call the Improcess and Caffemodel library files of OpenCV to transplant the trained multi-scale background prediction convolutional neural network model to the embedded system based on the Loongson chip. Use the dynamic link library to package the library files used for program operation, execute the written script files, and complete the recognition function of multi-category targets.
2. The multi-category target recognition method based on multi-scale prediction CNN and Loongson chip according to claim 1 is characterized in that: The step S2 specifically includes the following steps: A1: Sort the 1,000 candidate boxes in each target from large to small according to the corresponding confidence coefficients; A2: Select the candidate box corresponding to the maximum confidence coefficient, and calculate the IoU value between the remaining candidate boxes and the candidate box corresponding to the maximum confidence coefficient. The formula for calculating the IoU value is as follows: IoU = S2 / S3 where S2 represents the intersection area of the two candidate boxes, and S3 represents the union area of the two candidate boxes; A3: Set a threshold T, and calculate the new confidence coefficient according to the following formula: Among them, M is the candidate box corresponding to the maximum confidence coefficient, s i is the confidence coefficient of the i-th candidate box, b i For the i-th candidate box, IOU(M, b i ) is M and b i The IOU value, σ is a constant, e = 2.71828, G is the new confidence coefficient of the candidate box corresponding to the maximum confidence coefficient calculated by the above formula; If G < T, then delete the candidate box corresponding to the maximum confidence coefficient; A4: Continue to select a candidate box corresponding to the maximum confidence coefficient from the remaining candidate boxes, and repeat steps A1 to A3 until only one candidate box remains. The coordinate position (x, y, l, h) of the candidate box is the actual position of the target, and the target category corresponding to the maximum confidence coefficient is the actual category of the target.
3. The multi-category target recognition method based on multi-scale prediction CNN and Loongson chip according to claim 1 is characterized in that: The specific steps of the said step S3 include the following steps: B1: Arbitrarily select 10,000 real images in the COCO dataset, and use bilinear interpolation to obtain images with a size of 500*500; B2: Divide the 1,000 images into a training set and a test set according to a ratio of 4:
1. During the training process, take every 100 images out of the 1,000 images as a training unit; B3: Use the cross-entropy loss function as the loss function, and adopt the Adam optimizer as the network optimizer to fully train the multi-scale background prediction convolutional neural network to obtain a trained multi-scale background prediction convolutional neural network model.
4. The multi-category target recognition method based on multi-scale prediction CNN and Loongson chip according to claim 1 is characterized in that: The specific steps of the said step S4 include the following steps: C1: The visual camera captures a video file; C2: Intercept one image from the captured video file every 3 frames; C3: If the intercepted image is a grayscale image, use the gray-level to color transformation method to turn it into a color image; C4: Use bilinear interpolation on the color image to make it into a size of 500*500 to obtain a color image with a unified size.
