Vote recognition method and system and vote counting system

Abstract

The invention discloses a vote recognition method and system and a vote counting system, and belongs to the technical field of voting, and the method comprises the steps: obtaining scanning image data of a vote, and carrying out the preprocessing of the scanning image data; inputting the preprocessed scanning image data into a convolutional neural network model, and outputting a multi-scale convolutional feature map by a feature extraction layer; the feature pyramid network module receives the multi-scale convolution feature map and outputs a multi-scale fusion feature map; the adaptive feature fusion module receives the multi-scale fusion feature map and outputs an adaptive fusion feature map; the feature semantic layer receives the adaptive fusion feature map and outputs a semantic feature map; the classification layer receives the semantic feature map and outputs a feature vector to the output layer, and the output layer outputs a vote recognition result. According to the invention, the performance of the ballot system is improved, so that the ballot system can process ballot tasks under various complex conditions more accurately and stably.

Description

Technical Field

[0001] The present invention relates to the field of voting and election technology, and in particular to a ballot identification method and system and a ballot counting system. Background Art

[0002] The existing voting system is suitable for use in large occasions. Its characteristics are that there are many voters, high solemnity requirements, many required equipment, complex connections, high requirements for information confidentiality, troublesome construction and dismantling, many people are required, long preparation time, many operators during use, and long operation time. This makes the existing voting system unsuitable for small occasions. Small voting occasions have few voters and their identities are certain. How to complete the voting work quickly and efficiently in small voting occasions is a problem that technical personnel in this field urgently need to solve. If you want to complete the voting work quickly and efficiently, you can't avoid identifying the ballots and quickly and accurately classifying the ballots.

[0003] With the rapid development of neural network model technology, neural network models have been widely used in ballot recognition due to their powerful learning ability. Although neural network models have achieved remarkable results in the field of image recognition, there are many problems in ballot recognition based on existing neural network models. In terms of feature extraction, the existing neural network model is not flexible enough when processing multi-scale features of ballot images. The scales of different areas in ballot images (such as candidate areas, voting areas for other candidates, etc.) vary. The existing feature extraction part cannot effectively capture the relationship between these different scale features, resulting in the loss of important information in the recognition process. In terms of feature fusion, the existing technology does not fully consider the spatial position relationship between feature maps when fusing multi-scale features, and lacks key information of different positions of the ballot, which will weaken the influence of key information in the fusion process. In terms of feature semantics, traditional feature processing methods often simply extract and process global or local features, and do not fully consider the semantic association between different areas in the ballot, resulting in errors in classification. Summary of the invention

[0004] In view of the shortcomings of the prior art, the present invention provides a ballot recognition method, system and ballot counting system, which introduces a convolutional neural network model into ballot recognition and introduces an improved multi-scale fusion layer and feature semantic layer into the convolutional neural network model, thereby improving the performance of the ballot system and enabling it to more accurately and stably handle ballot tasks in various complex situations.

[0005] In order to achieve the above object, the present invention adopts the following technical solution: In a first aspect, the present invention provides a ballot identification method, comprising: Obtaining scanned image data of the ballot, and preprocessing the scanned image data; The preprocessed scanned image data is input into a convolutional neural network model, which includes an input layer, a feature extraction layer, a multi-scale fusion layer, a feature semantic layer, a classification layer and an output layer in sequence. The feature extraction layer receives the preprocessed scanned image data and outputs a multi-scale convolutional feature map; the multi-scale fusion layer includes a feature pyramid network module and an adaptive feature fusion module, the feature pyramid network module receives the multi-scale convolutional feature map and outputs a multi-scale fusion feature map; the adaptive feature fusion module receives the multi-scale fusion feature map and outputs an adaptive fusion feature map; the feature semantic layer receives the adaptive fusion feature map and outputs a semantic feature map; the classification layer receives the semantic feature map and outputs a feature vector to the output layer, and the output layer outputs the ballot recognition result.

[0006] As a further technical solution, the preprocessing includes ballot type identification and ballot scan image cropping. Specifically: a ballot identification code and a positioning code are provided on the ballot; first, the ballot type is determined based on the ballot identification code, and then the relative position of the scribbled area on the ballot is determined based on the positioning code, and the scribbled area on the ballot is cropped and used as the preprocessed scanned image data.

[0007] As a further technical solution, an attention mechanism is added to each layer of the feature pyramid network module. Specifically: the feature pyramid network module receives convolutional feature maps of different scales at each layer, performs global average pooling operations on the convolutional feature maps in the spatial dimension and the channel dimension, and obtains a spatial feature vector and a channel feature vector respectively. According to the spatial feature vector and the channel feature vector, a spatial attention map and a channel attention map are obtained respectively, and the spatial attention map and the channel attention map are multiplied to obtain the final attention feature map of each layer.

[0008] As a further technical solution, the fusion method of the feature pyramid network module is: = ;in, is the upsampling operation, is a multi-scale fusion feature map, is the path selection probability, It is the multi-scale convolution feature map output by the feature extraction layer. and is the final attention feature map of different layers, is element-wise multiplication.

[0009] As a further technical solution, the adaptive feature fusion module receives a multi-scale fusion feature map, performs relative position encoding on the pixels in the multi-scale fusion feature map to obtain a spatial position encoding, and then concatenates the spatial position encoding with the multi-scale convolution feature map to obtain a spatial fusion feature containing spatial position information.

[0010] As a further technical solution, the fusion weight is calculated according to the spatial fusion feature, and the formula is: = ;in, is the fusion weight, is the activation function, is the fully connected layer operation, It is a convolution operation; the fusion feature maps of different scales are fused according to the fusion weights, and an adaptive fusion feature map is output.

[0011] As a further technical solution, the feature semantic layer receives an adaptive fusion feature map, divides the adaptive fusion feature map into multiple local regions, calculates the regional feature vector for each local region, and calculates the semantic association matrix of the regional feature vector; updates each local region feature vector according to the semantic association matrix to obtain an updated regional feature vector, and reorganizes the updated regional feature vector to obtain a semantic feature map.

[0012] As a further technical solution, the loss function in the convolutional neural network model includes positioning loss , classification loss and semantic loss , where the positioning loss for: = ;in is the index collection of positive samples, are the actual bounding box coordinates, are the predicted bounding box coordinates, (x)= ; Classification loss for: = ,in, is the predicted class probability, is the true category label, k is the number of categories; semantic loss for: = , where S is the set of all semantically related region pairs, is the regional feature vector corresponding to the two local regions, is the semantic similarity target value, is the actual semantic similarity.

[0013] In a second aspect, the present invention provides a ballot identification system, comprising: The data acquisition and processing module is configured to: acquire the scanned image data of the ballot and pre-process the scanned image data; The ballot recognition module is configured to: input the preprocessed scanned image data into a convolutional neural network model, wherein the convolutional neural network model sequentially includes an input layer, a feature extraction layer, a multi-scale fusion layer, a feature semantic layer, a classification layer and an output layer, wherein the feature extraction layer receives the preprocessed scanned image data and outputs a multi-scale convolutional feature map; the multi-scale fusion layer includes a feature pyramid network module and an adaptive feature fusion module, wherein the feature pyramid network module receives the multi-scale convolutional feature map and outputs a multi-scale fusion feature map; the adaptive feature fusion module receives the multi-scale fusion feature map and outputs an adaptive fusion feature map; the feature semantic layer receives the adaptive fusion feature map and outputs a semantic feature map; the classification layer receives the semantic feature map and outputs a feature vector to the output layer, and the output layer outputs the ballot recognition result.

[0014] In a third aspect, the present invention provides a ballot counting system, comprising: a counting host; the counting host is connected to a sub-host via a switch, and the sub-host is connected to a scanner; A scanner configured to: obtain scanned image data of a ballot; The sub-host is configured to: pre-process the scanned image data; The vote counting host is configured to: input the preprocessed scanned image data into a convolutional neural network model, wherein the convolutional neural network model sequentially includes an input layer, a feature extraction layer, a multi-scale fusion layer, a feature semantic layer, a classification layer and an output layer, wherein the feature extraction layer receives the preprocessed scanned image data and outputs a multi-scale convolutional feature map; the multi-scale fusion layer includes a feature pyramid network module and an adaptive feature fusion module, wherein the feature pyramid network module receives the multi-scale convolutional feature map and outputs a multi-scale fusion feature map; the adaptive feature fusion module receives the multi-scale fusion feature map and outputs an adaptive fusion feature map; the feature semantic layer receives the adaptive fusion feature map and outputs a semantic feature map; the classification layer receives the semantic feature map and outputs a feature vector to the output layer, and the output layer outputs the ballot recognition result.

[0015] One or more technical solutions of the present invention have the following beneficial effects: (1) The present invention adds a multi-scale fusion layer (an improved feature pyramid network module and an improved adaptive feature fusion module) to the traditional convolutional neural network model, which can more effectively process the multi-scale features of the ballot image. It can better capture the features of different areas in the ballot, improve the integrity of feature extraction, and provide richer and more accurate feature information for subsequent accurate recognition. The improved convolutional neural network model can adaptively adjust the fusion weight according to the spatial position relationship between the feature maps, improve the effectiveness of the fusion features, and especially better process the key information at different positions in the ballot.

[0016] (2) The present invention adds an improved feature semantic layer to the traditional convolutional neural network model. By calculating the semantic association matrix between local regions, it can explore the deep semantic connections between different regions of the ballot and improve the ability to understand the content of the ballot. At the same time, it provides a feature map with more semantic value for the classification layer, so that the classification layer can better use these features to make classification decisions.

[0017] (3) The present invention optimizes and improves the loss function of the convolutional neural network model. The optimized loss function includes positioning loss, classification loss and semantic loss. During the training process, more emphasis is placed on the consistency of feature semantics, so that the model can better establish the semantic relationship between ballot features during the learning process. In conjunction with other improved modules, the model's overall ability to handle complex ballot situations is improved, the recognition error rate caused by insufficient semantic understanding is reduced, and the stability and accuracy of the ballot recognition system are improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0019] Figure 1 This is a flowchart of the ballot identification method of the present invention; Figure 2 Schematic diagram of the ballot of the present invention. DETAILED DESCRIPTION

[0020] It should be noted that the following detailed descriptions are illustrative and intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used in the present invention have the same meanings as those commonly understood by those skilled in the art to which the present invention belongs.

[0021] Embodiment 1 The present invention provides a ballot identification method, comprising the following steps: S1: Obtaining scanned image data of a ballot and preprocessing the scanned image data; S2: Input the preprocessed scanned image data into the convolutional neural network model, which includes an input layer, a feature extraction layer, a multi-scale fusion layer, a feature semantic layer, a classification layer and an output layer in sequence. The feature extraction layer receives the preprocessed scanned image data and outputs a multi-scale convolutional feature map; the multi-scale fusion layer includes a feature pyramid network module and an adaptive feature fusion module, the feature pyramid network module receives the multi-scale convolutional feature map and outputs a multi-scale fusion feature map; the adaptive feature fusion module receives the multi-scale fusion feature map and outputs an adaptive fusion feature map; the feature semantic layer receives the adaptive fusion feature map and outputs a semantic feature map; the classification layer receives the semantic feature map and outputs a feature vector to the output layer, and the output layer outputs the ballot recognition result.

[0022] In step S1, the acquired scanned image data is preprocessed, specifically including: ballot type identification and ballot scanned image cropping. The ballot is provided with a ballot identification code and at least three positioning codes. First, the ballot type is determined according to the ballot identification code. The ballot types include but are not limited to: professor selection, associate professor selection, director selection or deputy director selection. Secondly, the relative position of the scribbled area in the ballot is determined according to the positioning code, and the scribbled area in the ballot is cropped and used as the preprocessed scanned image data.

[0023] In step S2, the preprocessed scanned image data is input into the convolutional neural network model. In this embodiment, the traditional convolutional neural network model is improved. The improved convolutional neural network model includes an input layer, a feature extraction layer, a multi-scale fusion layer, a feature semantic layer, a classification layer and an output layer in sequence, wherein the multi-scale fusion layer includes a feature pyramid network module and an adaptive feature fusion module. Each layer of the feature pyramid network module is added with an attention mechanism, which not only pays attention to the channel information of the feature map, but also considers the spatial position information of the feature map. Specifically: the feature pyramid network module receives convolutional feature maps of different scales at each layer. Suppose the convolutional feature map of the ath layer in the feature pyramid network module is , whose size is ( is the height, is the width, is the number of channels), for the convolution feature map Perform global average pooling operations in the spatial dimension and channel dimension to obtain the spatial feature vectors (Size is 1 1 ) and channel feature vector (Size is ), specifically: spatial dimension: the convolution feature map is In the height and width direction, the size is The convolution feature map is After global average pooling in the spatial dimension, we get a size of 1 1 The vector is as follows: for each channel c (c=1, 2, 3... ), calculate the average value of all pixel values ​​in this channel, that is = ;in is the convolution feature map In Location , the pixel value at channel c, is the value of the vector at channel c after pooling; channel dimension: is the convolution feature map In the channel direction, the size is The convolution feature map is After global average pooling in the channel dimension, we get a size of The method is as follows: for each position (i, j) (i=1, 2...H; j=1, 2...W), calculate the average value of all channel pixel values, that is, = ;in is the value of the vector at position (i, j) after pooling.

[0024] According to the spatial eigenvector and channel feature vector , and then pass through two fully connected layers to obtain the spatial attention map and channel attention map , specifically: spatial attention map The calculation is as follows: = ( ); = + ( ));in, is the spatial average pooling operation, and are two fully connected layers with different parameters, is the activation function (such as the sigmoid function).

[0025] Channel Attention Map The calculation is as follows: = ( ); = + ( ));in, is the channel average pooling operation, and are two other fully connected layers with different parameters, The same activation function as above (such as the sigmoid function).

[0026] The spatial attention map and channel attention map Multiply to get the final attention feature map of each layer , = ,in is element-by-element multiplication. Through the above method, the model can focus more on the important channels in the key areas of the feature map and improve the representation ability of the features. The final attention feature map of each layer is obtained After that, the fusion is performed to obtain a multi-scale fusion feature map , the fusion method of the feature pyramid network module is: = ;in, is the upsampling operation, is a multi-scale fusion feature map, is the path selection probability, It is the multi-scale convolution feature map output by the feature extraction layer. and is the final attention feature map of different layers, This dynamic fusion path can better use the feature distribution of the ballot and improve the flexibility and accuracy of the model. It can more effectively process the multi-scale features of the ballot image, better capture the features of different areas in the ballot, improve the integrity of feature extraction, and provide richer and more accurate feature information for subsequent accurate recognition.

[0027] In step S2, the adaptive feature fusion module receives the multi-scale fusion feature map , according to the multi-scale fusion feature map The relative position encoding of the pixels in is performed. Specifically: The spatial position of the pixel (h, w) is encoded as , using a relative position encoding method, =[ , ],in, for Height, for The width of . Then encode the spatial position Fusion with multi-scale feature maps Splice to obtain spatial fusion features containing spatial location information , =Concat( , ). When performing fusion, according to the spatial fusion features Calculate the fusion weight, set and The fusion feature maps of two different scales contain the features of spatial location information, and a simple neural network (such as a convolutional layer and a fully connected layer) is used to and Processing to obtain fusion weight , the specific formula is: = ;in, is the fusion weight, is the activation function, is the fully connected layer operation, It is a convolution operation; the fusion feature maps of different scales are fused according to the fusion weights, and the adaptive fusion feature map is output. Specifically: = +(1- ) .

[0028] In step S2, the feature semantic layer receives the adaptive fusion feature map , let its shape be ( is the height, is the width, is the number of channels), the adaptive fusion feature map Divide into n local areas , i=1, 2, 3…n, the size of each region is ,in = , = For each local area , calculate its regional feature vector , calculated as: = ;in, Represents adaptive fusion feature map In coordinates The eigenvector at .

[0029] Then the semantic association matrix Z∈ between the regional feature vectors is calculated , where n is the n local areas divided above, and the semantic association matrix Z, whose elements Represents a local area and The degree of semantic association between them is calculated as follows: = ,in, ) represents a function to measure the similarity of two regional feature vectors, such as a cosine similarity function: )= Next, each local region feature vector is updated according to the semantic association matrix to obtain the updated regional feature vector , = , the updated regional feature vector Recombination to obtain semantic feature map .

[0030] In this embodiment, the loss function L in the convolutional neural network model includes the positioning loss , classification loss and semantic loss , where the positioning loss for: = ;in is the index collection of positive samples, are the actual bounding box coordinates, are the predicted bounding box coordinates, (x)= ; Classification loss for: = ,in, is the predicted class probability, is the true category label, k is the number of categories; semantic loss for: = , where S is the set of all semantically related region pairs, is the regional feature vector corresponding to the two local regions, is the semantic similarity target value, is the actual semantic similarity. Finally, L= + + By introducing semantic loss, the model is forced to pay more attention to the consistency of feature semantics during training. This enables the model to better establish semantic relationships between ballot features during learning, which improves the model's ability to handle complex ballot situations, reduces the recognition error rate caused by insufficient semantic understanding, and improves the stability and accuracy of the ballot recognition system.

[0031] In step S2, the classification layer receives the semantic feature map , and outputs the feature vector to the output layer, and the output layer outputs the ballot recognition result. Specifically: The classification layer includes a fully connected layer and a Softmax layer. Feature mapping and classification decisions are performed through multiple fully connected layers. For example, in ballot recognition, multiple output categories are set according to the type of ballot (such as ballot, non-ballot, etc.), the selection of candidates (such as approval, opposition, abstention, etc.), etc. The fully connected layer learns the mapping relationship between the ballot image features and these categories to achieve accurate classification of ballots. The Softmax layer is usually located at the last layer of the classification layer and is used to convert the numerical value output by the fully connected layer into a probability distribution. The Softmax function normalizes the score of each category output by the fully connected layer so that the sum of all category scores is 1, thereby obtaining the probability value of each category. The above-mentioned fully connected layer and Softmax layer can select existing technologies to achieve the above purpose. The output layer outputs the ballot recognition result, including the type of ballot (such as ballot, non-ballot), the selection of candidates (such as the specific number of votes for approval, opposition, and abstention).

[0032] Embodiment 2 The present invention provides a ballot identification system, comprising the following modules: The data acquisition and processing module is configured to: acquire the scanned image data of the ballot and pre-process the scanned image data; The ballot recognition module is configured to: input the preprocessed scanned image data into a convolutional neural network model, wherein the convolutional neural network model sequentially includes an input layer, a feature extraction layer, a multi-scale fusion layer, a feature semantic layer, a classification layer and an output layer, wherein the feature extraction layer receives the preprocessed scanned image data and outputs a multi-scale convolutional feature map; the multi-scale fusion layer includes a feature pyramid network module and an adaptive feature fusion module, wherein the feature pyramid network module receives the multi-scale convolutional feature map and outputs a multi-scale fusion feature map; the adaptive feature fusion module receives the multi-scale fusion feature map and outputs an adaptive fusion feature map; the feature semantic layer receives the adaptive fusion feature map and outputs a semantic feature map; the classification layer receives the semantic feature map and outputs a feature vector to the output layer, and the output layer outputs the ballot recognition result.

[0033] Embodiment 3 The present invention provides a ballot counting system, comprising: a counting host; the counting host is connected to a sub-host via a switch, and the sub-host is connected to a scanner; Wherein, the scanner is configured to: obtain scanned image data of the ballot; The sub-host is configured to: pre-process the scanned image data; The vote counting host is configured to: input the preprocessed scanned image data into a convolutional neural network model, wherein the convolutional neural network model sequentially includes an input layer, a feature extraction layer, a multi-scale fusion layer, a feature semantic layer, a classification layer and an output layer, wherein the feature extraction layer receives the preprocessed scanned image data and outputs a multi-scale convolutional feature map; the multi-scale fusion layer includes a feature pyramid network module and an adaptive feature fusion module, wherein the feature pyramid network module receives the multi-scale convolutional feature map and outputs a multi-scale fusion feature map; the adaptive feature fusion module receives the multi-scale fusion feature map and outputs an adaptive fusion feature map; the feature semantic layer receives the adaptive fusion feature map and outputs a semantic feature map; the classification layer receives the semantic feature map and outputs a feature vector to the output layer, and the output layer outputs the ballot recognition result.

[0034] In this embodiment, at least one sub-host, scanner, and switch are set. If only one is set, it can form a minimum ballot counting system, and one person can complete all the counting work. If N (N<6) are set, a small local area network can be formed to achieve fast voting and counting.

[0035] The system is compact and flexible. Professionals only need to do the preparatory work (preliminary ballot confirmation, ballot template production, and vote counting report template production), while ordinary computer operators can operate the vote counting work on site. It is particularly suitable for fixed places, fixed number of voters, one-time single or multiple voting on site to complete the vote counting of an election, and output the vote counting result report. For example: in the selection of professional titles, when selecting medium and senior professional titles, 20 to 30 experts on site select ten candidates from 50 to 60 or even more candidates, and they are selected as medium and senior professional titles. At this time, the director of the selection expert needs to organize the experts on site many times, carefully read the materials of the candidates, and vote many times to select ten qualified candidates to pass the selection of medium and senior professional titles. For another example, in the selection of ten good things for the people in a certain year in the community, community representatives proposed 20 to 30 good things for the people. Due to limited financial resources, only ten items can be solved. This requires community representatives to vote anonymously to select ten things that everyone cares about the most to solve.

[0036] Before using the system for an election meeting, it is necessary to determine the basic information of the election meeting in preparation for the ballot template, including the number of ballots, the election positions on each ballot, the number of candidates for each position, and the number of candidates to be elected and the number of alternative candidates. The system-specific positioning code and ballot identification code are added to the ballot template to ensure that the system can identify each type of ballot and the relative position of the writing area on the ballot. The system needs to define the area of ​​the ballot positioning code, ballot identification code, candidate information writing area, and alternative candidate information writing area to form a ballot template file.

[0037] The scanner scans the ballot to obtain the scanned image data, which is then transmitted to the sub-host for preprocessing. The sub-host determines the type of ballot according to the ballot identification code, and determines the relative position of the scribbled area in the ballot according to the positioning code. The scribbled area in the ballot is cropped and used as the preprocessed scanned image data. The sub-host uploads the preprocessed scanned image data to the counting host through the switch for ballot identification. The specific method of ballot identification is shown in Example 1. On the counting host, there is a manual confirmation reminder for invalid ballots: if it is manually confirmed that the ballot image is a real invalid ballot, click the "Invalid Ballot" button; if it is a fake invalid ballot, select the corresponding position and candidate, and manually modify the information that was incorrectly identified in the image. The system has a candidate statistical vote query on the counting host. The statistical votes for a candidate's approval, opposition, and abstention can be queried. Double-click the number to view the corresponding image, which is convenient for manual confirmation of whether the identification is accurate. The system has a counting result report output module on the counting host, which can number each position on the ballot. The program can pass the statistical results of each position in the form of a table to a word file: "Counting Result Report Template" through the reference of a word bookmark, to form a counting result report file for print output.

[0038] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A ballot identification method, characterized in that: include: Obtaining scanned image data of the ballot, and preprocessing the scanned image data; The preprocessed scanned image data is input into a convolutional neural network model, which includes an input layer, a feature extraction layer, a multi-scale fusion layer, a feature semantic layer, a classification layer and an output layer in sequence. The feature extraction layer receives the preprocessed scanned image data and outputs a multi-scale convolutional feature map; the multi-scale fusion layer includes a feature pyramid network module and an adaptive feature fusion module, the feature pyramid network module receives the multi-scale convolutional feature map and outputs a multi-scale fusion feature map; the adaptive feature fusion module receives the multi-scale fusion feature map and outputs an adaptive fusion feature map; the feature semantic layer receives the adaptive fusion feature map and outputs a semantic feature map; the classification layer receives the semantic feature map and outputs a feature vector to the output layer, and the output layer outputs the ballot recognition result.

2. A ballot identification method as claimed in claim 1, characterized in that: The preprocessing includes ballot type identification and ballot scan image cropping. Specifically: a ballot identification code and a positioning code are set on the ballot; first, the ballot type is determined based on the ballot identification code, and then the relative position of the scribbled area on the ballot is determined based on the positioning code, and the scribbled area on the ballot is cropped and used as the preprocessed scanned image data.

3. A ballot identification method as claimed in claim 1, characterized in that: Each layer of the feature pyramid network module adds an attention mechanism. Specifically, the feature pyramid network module receives convolutional feature maps of different scales at each layer, performs global average pooling operations on the convolutional feature maps in the spatial dimension and the channel dimension, and obtains a spatial feature vector and a channel feature vector respectively. A spatial attention map and a channel attention map are obtained according to the spatial feature vector and the channel feature vector respectively, and the spatial attention map and the channel attention map are multiplied to obtain the final attention feature map of each layer.

4. A ballot identification method as claimed in claim 3, characterized in that: The fusion method of the feature pyramid network module is: = ;in, is the upsampling operation, is a multi-scale fusion feature map, is the path selection probability, It is the multi-scale convolution feature map output by the feature extraction layer. and is the final attention feature map of different layers, is element-wise multiplication.

5. A ballot identification method as claimed in claim 1, characterized in that: The adaptive feature fusion module receives a multi-scale fusion feature map, performs relative position encoding on pixels in the multi-scale fusion feature map to obtain a spatial position encoding, and then concatenates the spatial position encoding with the multi-scale convolution feature map to obtain a spatial fusion feature containing spatial position information.

6. A ballot identification method as claimed in claim 5, characterized in that: The fusion weight is calculated according to the spatial fusion feature, and the formula is: = ;in, is the fusion weight, is the activation function, is the fully connected layer operation, It is a convolution operation; the fusion feature maps of different scales are fused according to the fusion weights, and an adaptive fusion feature map is output.

7. A ballot identification method as claimed in claim 1, characterized in that: The feature semantic layer receives an adaptive fusion feature map, divides the adaptive fusion feature map into multiple local areas, calculates a regional feature vector for each local area, and calculates a semantic association matrix of the regional feature vector; updates each local regional feature vector according to the semantic association matrix to obtain an updated regional feature vector, and reorganizes the updated regional feature vector to obtain a semantic feature map.

8. A ballot identification method as claimed in claim 1, characterized in that: The loss function in the convolutional neural network model includes positioning loss , classification loss and semantic loss , where the positioning loss for: = ;in is the index collection of positive samples, are the actual bounding box coordinates, are the predicted bounding box coordinates, (x)= ; Classification loss for: = ,in, is the predicted class probability, is the true category label, k is the number of categories; semantic loss for: = , where S is the set of all semantically related region pairs, is the regional feature vector corresponding to the two local regions, is the semantic similarity target value, is the actual semantic similarity.

9. A ballot identification system, characterized in that: include: The data acquisition and processing module is configured to: acquire the scanned image data of the ballot and pre-process the scanned image data; The ballot recognition module is configured to: input the preprocessed scanned image data into a convolutional neural network model, wherein the convolutional neural network model sequentially includes an input layer, a feature extraction layer, a multi-scale fusion layer, a feature semantic layer, a classification layer and an output layer, wherein the feature extraction layer receives the preprocessed scanned image data and outputs a multi-scale convolutional feature map; the multi-scale fusion layer includes a feature pyramid network module and an adaptive feature fusion module, wherein the feature pyramid network module receives the multi-scale convolutional feature map and outputs a multi-scale fusion feature map; the adaptive feature fusion module receives the multi-scale fusion feature map and outputs an adaptive fusion feature map; the feature semantic layer receives the adaptive fusion feature map and outputs a semantic feature map; the classification layer receives the semantic feature map and outputs a feature vector to the output layer, and the output layer outputs the ballot recognition result.

10. A ballot counting system, characterized in that: include: A vote counting host; the vote counting host is connected to a sub-host via a switch, and the sub-host is connected to a scanner; A scanner configured to: obtain scanned image data of a ballot; The sub-host is configured to: pre-process the scanned image data; The vote counting host is configured to: input the preprocessed scanned image data into a convolutional neural network model, wherein the convolutional neural network model sequentially includes an input layer, a feature extraction layer, a multi-scale fusion layer, a feature semantic layer, a classification layer and an output layer, wherein the feature extraction layer receives the preprocessed scanned image data and outputs a multi-scale convolutional feature map; the multi-scale fusion layer includes a feature pyramid network module and an adaptive feature fusion module, wherein the feature pyramid network module receives the multi-scale convolutional feature map and outputs a multi-scale fusion feature map; the adaptive feature fusion module receives the multi-scale fusion feature map and outputs an adaptive fusion feature map; the feature semantic layer receives the adaptive fusion feature map and outputs a semantic feature map; the classification layer receives the semantic feature map and outputs a feature vector to the output layer, and the output layer outputs the ballot recognition result.

Images