Classification method, system, device and medium based on automobile document image enhancement

Abstract

The application discloses a classification method, system, device and medium based on automobile document image enhancement, and belongs to the technical field of data document classification. Image data of automobile data documents, main classification field features and corresponding automobile data document field classification results are acquired; key image features are acquired from the image data by using an image segmentation enhancement technology, and image field fusion features are formed according to the key image features and the main classification field features; a multiple neural network automobile data document image classification model is constructed; the image field fusion features are input into the multiple neural network automobile data document image classification model for training; image field fusion features of automobile data documents to be classified are acquired and input into the trained multiple neural network automobile data document image classification model, so that automobile data document field classification results of the automobile data documents to be classified are obtained. The application has strong classification capability and can realize accurate classification of automobile field data documents.

Description

Technical Field

[0001] This invention relates to the field of data document classification technology, and more specifically, to a classification method, system, device, and medium based on image enhancement of automotive documents. Background Technology

[0002] The common classification method for patent documents is the IPC (International Patent Classification) classification. However, in specific fields, such as the automotive industry, technicians may not be able to directly know which technologies the patent documents mainly involve based on the IPC classification number (such as "engine technology", "body electronics", "new energy vehicles"). Therefore, in the automotive industry, further classification based on patent documents can more conveniently reflect the main automotive technologies involved in the patent documents, thereby improving the convenience for technicians to use patent documents.

[0003] Currently, for images in automotive data documents containing circuit diagrams, structural schematics, formula data diagrams, and flowcharts, the commonly used automated classification and retrieval methods mainly rely on traditional OCR technology and general single machine learning image data classification models. These methods perform poorly when dealing with blurry, dense lines, complex diagrams, nested annotations, and mixed text in automotive data documents.

[0004] The image data in automotive data documents is often of poor clarity, making it difficult to apply key information to relevant models. Typically, the image data is simply used directly from the document. A single model struggles to accurately understand the topological information of technical illustrations and parse the classification information within them, resulting in coarse classification granularity and impacting the efficiency of patent search and analysis. Summary of the Invention

[0005] The technical problem to be solved by the present invention is to address the above-mentioned shortcomings of the prior art. The purpose of the present invention is to provide a classification method based on automotive document image enhancement. By using image enhancement technology to obtain key image features in automotive data documents, and constructing a multi-neural network model to classify the image data in automotive data documents, it has a stronger classification ability than a single classifier and achieves accurate classification of automotive data documents.

[0006] The second objective of this invention is to provide a classification system based on automotive document image enhancement.

[0007] The third objective of this invention is to provide a computer device.

[0008] The fourth objective of this invention is to provide a computer storage medium.

[0009] To achieve the first objective mentioned above, this invention provides a classification method based on automotive document image enhancement, comprising: Obtain vehicle data documents and classify them according to a preset vehicle data document domain classification method; Obtain the image data, main classification field features, and corresponding vehicle data document domain classification results from the vehicle data document; Image segmentation enhancement technology is used to obtain key image features from the image data, and image field fusion features are formed based on the key image features and the main classification field features. A multi-neural network model for classifying automotive data documents and images is constructed using a neural network model with multiple improved activation functions. The image field fusion features are input into the multi-neural network vehicle data document image classification model for training; The image field fusion features of the car data document to be classified are obtained and input into the trained multi-neural network car data document image classification model to obtain the car data document domain classification result of the car data document to be classified.

[0010] As a further improvement, the main classification field feature refers to the feature value set by the existing main classification number of the document, where the feature value corresponding to the part is a, the feature value corresponding to the major category is b, the feature value corresponding to the minor category is c, the feature value corresponding to the major group is d, and the feature value corresponding to the minor group is e. The image field fusion feature is represented as follows:

[0011] in, ~ These are the grayscale values ​​of a single pixel in the key image features.

[0012] Furthermore, obtaining key image features from the image data using image segmentation and enhancement techniques includes: Non-digital image data is selected using the mean of local variance; Select the preferred non-digital image data from the non-digital image data, which has the highest sum of image grayscale values. The preferred non-digital image data is subjected to key image feature separation and non-key image feature separation.

[0013] Furthermore, the mean of the local variance is calculated as follows:

[0014] In the formula, Let be the local variance at the center pixel (x, y), and N be the total number of pixels within the window. Let be the grayscale value of the i-th pixel within the window. This is the average grayscale value of all pixels within the window.

[0015] Furthermore, the threshold variance R between the key image features and non-key image features is defined as follows:

[0016] In the formula, The proportion of key images in the overall preferred non-digital image data. G represents the proportion of non-critical images in the total preferred non-digital image data, and G is the total average gray level of the preferred non-digital image data. The average gray level of the key image. The average gray level of the non-critical image.

[0017] Furthermore, the activation function calculation formula for the j-th improved activation function neural network model in the multiple improved activation function neural network model is as follows:

[0018] In the formula, Let j be the activation function value of the improved activation function neural network model. The feature values ​​of the image field fusion features are linearly transformed by weights and biases. The number of training datasets for training the j-th improved activation function neural network model is given by [reference to dataset number]. This represents the total amount of the training dataset.

[0019] Furthermore, the trained multi-neural network vehicle data document image classification model includes multiple parallel sub-neural network models. The image field fusion features of the vehicle data document to be classified are input into each of the sub-neural network models to obtain multiple output values, and the multiple output values ​​are voted on. If there is only one output value with the highest number of votes, then the automotive data document technology field corresponding to the output value with the highest number of votes is used as the main field for classification, and the automotive data document technology field corresponding to the average weighted value of each output value is used as the sub-field for further classification. If there is more than one output value with the highest number of votes, the corresponding automotive data document technology field is selected as the candidate field based on the output value with the highest number of votes, and then voting is conducted on each candidate field; if there is only one candidate field with the highest number of votes, the candidate field with the highest number of votes is selected as the main field for classification, and each candidate field is used as the sub-field for further classification; otherwise, the automotive data document technology field corresponding to the average weighted value of the output values ​​with the highest number of votes is selected as the main field for classification, and each candidate field is used as the sub-field for further classification. If all of the output values ​​have a vote of 1, then the technical field of the automotive data document corresponding to the average weighted value of each output value is used as the main field for classification.

[0020] To achieve the second objective mentioned above, this invention provides a classification system based on automotive document image enhancement, comprising: The first acquisition module is used to acquire vehicle data documents and classify the vehicle data documents according to a preset vehicle data document domain classification method; The second acquisition module is used to acquire the image data, main classification field features, and corresponding vehicle data document domain classification results of the vehicle data document; The segmentation module is used to obtain key image features from the image data using image segmentation enhancement technology, and to form image field fusion features based on the key image features and the main classification field features. The model building module is used to construct a multi-neural network vehicle data document image classification model using a multi-improved activation function neural network model. The training module is used to input the fused features of the image fields into the multi-neural network vehicle data document image classification model for training; The classification module is used to obtain the image field fusion features of the car data document to be classified, and input them into the trained multi-neural network car data document image classification model to obtain the car data document domain classification result of the car data document to be classified.

[0021] To achieve the above-mentioned objective three, the present invention provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the above-mentioned classification method based on automotive document image enhancement.

[0022] To achieve the fourth objective mentioned above, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the aforementioned classification method based on automotive document image enhancement.

[0023] Beneficial effects Compared with the prior art, the advantages of this invention are as follows: This invention uses image enhancement technology to obtain key image features in automotive data documents and constructs a multi-neural network model to classify image data in automotive data documents. It has a stronger classification ability than a single classifier and achieves accurate classification of automotive data documents. Attached Figure Description

[0024] Figure 1 This is a flowchart of the present invention; Figure 2This is a structural diagram of a multi-neural network model for classifying automotive data documents and images. Figure 3 A schematic diagram illustrating the classification results for automotive data documents. Detailed Implementation

[0025] The present invention will be further described below with reference to specific embodiments shown in the accompanying drawings.

[0026] See Figures 1-3 A classification method based on automotive document image enhancement includes: Obtain automotive data documents (i.e., patent publications related to the automotive field), and classify the automotive data documents according to a pre-defined classification method for automotive data document fields. For example... Figure 3 As shown, the technical field is the preset automotive data document field classification. The existing patent documents' IPC classification in the automotive field is further classified according to the technical composition, so that it can more conveniently reflect the automotive technologies mainly involved in the patent documents and improve the convenience for technicians to use the patent documents.

[0027] This process involves acquiring the image data, main classification field features, and corresponding domain classification results from automotive data documents. The image data in these documents primarily refers to the accompanying illustrations in the instruction manual. The main classification field features are defined using the existing main classification number in the document. Specifically, feature value 'a' corresponds to a part, feature value 'b' to a major category, feature value 'c' to a minor category, feature value 'd' to a major group, and feature value 'e' to a minor group.

[0028] For example, if the document classification number is B60L14 / 20, then the corresponding feature space vector is:

[0029] That is, the feature value corresponding to part B is a, the feature value corresponding to the major category B60 is b, the feature value corresponding to the minor category B60L is c, the feature value corresponding to the major group B60L14 / 00 is d, and the feature value corresponding to the minor group BB60L14 / 20 is e. The feature values ​​are dynamically adjusted and set by technicians based on the effect of model training and classification, so as to improve the accuracy and robustness of the model for document classification.

[0030] Image segmentation enhancement techniques are used to obtain key image features from image data, and image field fusion features are formed based on key image features and main classification field features.

[0031] Image segmentation and enhancement techniques are used to obtain key image features from image data, including: Step 1. Select non-digital image data using the mean of local variance; Step 2. Select the non-digital image data with the highest sum of image grayscale values ​​from the non-digital image data; Step 3. Separate key image features and non-key image features from the selected non-digital image data.

[0032] Local variance is a statistic that measures the degree of drastic change in grayscale values ​​in local regions of an image. The mean of local variance is calculated as follows:

[0033] In the formula, Let be the local variance at the center pixel (x, y), and N be the total number of pixels within the window. Let be the grayscale value of the i-th pixel within the window. This is the average grayscale value of all pixels within the window.

[0034] For example, for a 4x4 image, if the selected window is 2x2, then N is 4. The mean of the local variance of the four windows is calculated and averaged. The average of the total mean of the local variance is used to select the best non-digital image data. The threshold is selected by the difference between the pure digital image and the image with graphics. The threshold range is divided into Y between 50 and 100. Data greater than Y is selected as non-digital image data.

[0035] The more features an image data has, the better. Therefore, the image with the highest total grayscale value is selected as the preferred non-digital image data.

[0036] Key image features are separated from non-key image features based on the threshold variance R between them. The threshold variance R between key image features and non-key image features is defined as follows:

[0037] In the formula, The proportion of key images in the overall preferred non-digital image data. G represents the proportion of non-critical images in the total preferred non-digital image data, and G is the total average gray level of the preferred non-digital image data. The average gray level of the key image. The average gray level of the non-critical images is used. The variance greater than the threshold is selected as the key image feature. The key and non-key images are simply defined by a window, but the specific key image feature values ​​are not obtained. It is equivalent to roughly defining the window and then thresholding the data to obtain the spatial feature vector of the key image features.

[0038] Image field fusion features are represented as follows:

[0039] in, ~ These are the gray values ​​of a single pixel in the key image features. This fusion method is a spatial feature stitching process performed to facilitate the subsequent processing of the corresponding feature vectors in the model.

[0040] A multi-neural network model for classifying automotive data documents into images was constructed using a multi-activation function neural network model. The classification results for automotive data documents were obtained by fusing the output values ​​of the multi-activation function neural network model.

[0041] The formula for calculating the activation function of the j-th improved activation function in a neural network model with multiple improved activation functions is as follows:

[0042] In the formula, Let j be the activation function value of the improved activation function neural network model. The feature values ​​of the image field are linearly transformed by weights and biases to fuse features. The number of training datasets for training the j-th improved activation function neural network model. This represents the total amount of the training dataset.

[0043] Ensemble learning, based on the concept of ensemble learning, combines multiple neural network models using different improved activation functions to enhance overall performance. Its core logic lies in leveraging the feature diversity brought by different activation functions to compensate for the shortcomings of a single model. Specifically, the final classification result is determined by a weighted average of multiple neural network models and a vote among the classifiers trained by each model.

[0044] Specifically, the multiple improved activation function neural network model employs an input layer, a multi-layer independent parallel sub-neural network input-output layer, and an ensemble and fusion decision layer, such as... Figure 2 The diagram shows a three-layer independent parallel sub-neural network model. The activation function for each layer is the improved activation function described above. In this application, the hidden layers of the neural network model consist of three fully connected layers with 256, 128, and 64 neurons respectively. The improved activation function is used. Dropout is applied after each layer with a coefficient of 0.25 to suppress overfitting and adapt to scenarios with varying total training data volumes. The output layer has three neurons, containing the professional classification results for automotive data documents, as shown below. Figure 3As shown. The training hyperparameters use the Adam optimizer, with an initial learning rate of 0.001 and a learning rate decay coefficient of 0.95; the loss function uses classification cross-entropy and mean squared error of MSE, balancing risk classification and scheduling value regression; the batch size is set to 32, and the maximum number of iterations (epochs) is 200. Except for the hyperparameter settings of the neural network model described above, the remaining steps in this application are the same as the processing flow and hyperparameters of traditional neural network models.

[0045] The image field fusion features are input into a multi-neural network for training an image classification model of automotive data documents.

[0046] To obtain the image field fusion features of the car data document to be classified, we first obtain the image data and main classification field features of the car data document to be classified. Then, we use image segmentation enhancement technology to obtain key image features from the image data. Based on the key image features and the main classification field features, we form the image field fusion features. We then input the image field fusion features of the car data document to be classified into a trained multi-neural network car data document image classification model to obtain the car data document domain classification result of the car data document to be classified.

[0047] Specifically, the trained multi-neural network vehicle data document image classification model includes multiple parallel sub-neural network models. The image field fusion features of the vehicle data document to be classified are input into each sub-neural network model to obtain multiple output values, and the multiple output values ​​are voted on. If there is only one output value with the highest number of votes, then the automotive data document technology field corresponding to the output value with the highest number of votes is used as the main field for classification, and the automotive data document technology field corresponding to the average weighted value of each output value is used as the sub-field for further classification. If there is more than one output value with the highest number of votes, the corresponding automotive data document technology field is selected as the candidate field based on the output value with the highest number of votes, and then voting is conducted on each candidate field; if there is only one candidate field with the highest number of votes, the candidate field with the highest number of votes is selected as the main field for classification, and each candidate field is used as the sub-field for further classification; otherwise, the automotive data document technology field corresponding to the average weighted value of the output values ​​with the highest number of votes is selected as the main field for classification, and each candidate field is used as the sub-field for further classification. If multiple output values ​​all have a vote of 1, then the technical field of the automotive data document corresponding to the average weighted value of each output value is used as the main field for classification.

[0048] by Figure 2For example, the three trained sub-neural network models, inputting image field fusion features, produce three output values ​​of 0.267, 0.267, and 0.577 respectively. This corresponds to 0.267 representing two votes and 0.577 representing one vote. Their average weighted value is 0.370. Therefore, the classification is based on the technical field corresponding to 0.267, such as... Figure 3 As shown, 0.267 corresponds to the field of new energy vehicle technology, and is further subdivided into more specific technical fields based on an average weighted value of 0.370. 0.370 corresponds to the hybrid vehicle and integrated technology fields within the new energy vehicle technology field. If all three values ​​are different, the average weighted value is used to directly classify the corresponding technical field. Of course, this invention involves more than three trained neural network models; this is only one embodiment.

[0049] A classification system based on automotive document image enhancement, comprising: The first acquisition module is used to acquire vehicle data documents and classify them according to a preset vehicle data document domain classification method. The second acquisition module is used to acquire image data, main classification field features, and corresponding automotive data document domain classification results from automotive data documents. The segmentation module is used to obtain key image features from image data using image segmentation enhancement technology, and to form image field fusion features based on key image features and main classification field features. The model building module is used to construct a multi-neural network vehicle data document image classification model using a multi-improved activation function neural network model. The training module is used to input the fused features of image fields into a multi-neural network for training the automotive data document image classification model; The classification module is used to obtain the image field fusion features of the car data document to be classified, and input them into the trained multi-neural network car data document image classification model to obtain the car data document domain classification result of the car data document to be classified.

[0050] A computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the above-described classification method based on automotive document image enhancement.

[0051] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the aforementioned classification method based on automotive document image enhancement.

[0052] The above are merely preferred embodiments of the present invention. It should be noted that those skilled in the art can make several modifications and improvements without departing from the structure of the present invention, and these will not affect the effectiveness of the implementation of the present invention or the practicality of the patent.

Claims

1. A classification method based on automotive document image enhancement, characterized in that, include: Obtain vehicle data documents and classify them according to a preset vehicle data document domain classification method; Obtain the image data, main classification field features, and corresponding vehicle data document domain classification results from the vehicle data document; Image segmentation enhancement technology is used to obtain key image features from the image data, and image field fusion features are formed based on the key image features and the main classification field features. A multi-neural network vehicle data document image classification model is constructed using a multi-improved activation function neural network model; the activation function calculation formula for the j-th improved activation function neural network model in the multi-improved activation function neural network model is as follows: In the formula, Let j be the activation function value of the improved activation function neural network model. The feature values ​​of the image field fusion features are linearly transformed by weights and biases. The number of training datasets for training the j-th improved activation function neural network model is given by [reference to dataset number]. Total amount of training dataset; The image field fusion features are input into the multi-neural network vehicle data document image classification model for training; The image field fusion features of the car data document to be classified are obtained and input into the trained multi-neural network car data document image classification model to obtain the car data document domain classification result of the car data document to be classified.

2. The classification method based on automotive document image enhancement according to claim 1, characterized in that, The main category field feature refers to the feature value set by the existing main category number of the document. The feature value corresponding to the part is a, the feature value corresponding to the major category is b, the feature value corresponding to the minor category is c, the feature value corresponding to the major group is d, and the feature value corresponding to the minor group is e. The image field fusion feature is represented as follows: in, ~ These are the grayscale values ​​of a single pixel in the key image features.

3. The classification method based on automotive document image enhancement according to claim 1, characterized in that, Obtaining key image features from the image data using image segmentation and enhancement techniques includes: Non-digital image data is selected using the mean of local variance; Select the preferred non-digital image data from the non-digital image data, which has the highest sum of image grayscale values. The preferred non-digital image data is subjected to key image feature separation and non-key image feature separation.

4. The classification method based on automotive document image enhancement according to claim 3, characterized in that, The mean of the local variance is calculated as follows: In the formula, Let be the local variance at the center pixel (x, y), and N be the total number of pixels within the window. Let be the grayscale value of the i-th pixel within the window. This is the average grayscale value of all pixels within the window.

5. The classification method based on automotive document image enhancement according to claim 3, characterized in that, The threshold variance R between key image features and non-key image features is defined as follows: In the formula, The proportion of key images in the overall preferred non-digital image data. G represents the proportion of non-critical images in the total preferred non-digital image data, and G is the total average gray level of the preferred non-digital image data. The average gray level of the key image. The average gray level of the non-critical image.

6. The classification method based on automotive document image enhancement according to claim 1, characterized in that, The trained multi-neural network vehicle data document image classification model includes multiple parallel sub-neural network models. The image field fusion features of the vehicle data document to be classified are input into each of the sub-neural network models to obtain multiple output values, and the multiple output values ​​are voted on. If there is only one output value with the highest number of votes, then the automotive data document technology field corresponding to the output value with the highest number of votes is used as the main field for classification, and the automotive data document technology field corresponding to the average weighted value of each output value is used as the sub-field for further classification. If there is more than one output value with the highest number of votes, the corresponding automotive data document technology field is taken as the candidate field based on the output value with the highest number of votes, and then a vote is taken on each candidate field. If there is only one candidate field with the most votes, then the candidate field with the most votes is classified as the main field, and each candidate field is further classified as a sub-field. Otherwise, the automotive data documents are classified into primary domains based on the average weighted value of the output values ​​with the highest number of votes, and further subdivided into secondary domains based on each candidate domain. If all of the output values ​​have a vote of 1, then the technical field of the automotive data document corresponding to the average weighted value of each output value is used as the main field for classification.

7. A classification system based on automotive document image enhancement, characterized in that, include: The first acquisition module is used to acquire vehicle data documents and classify the vehicle data documents according to a preset vehicle data document domain classification method; The second acquisition module is used to acquire the image data, main classification field features, and corresponding vehicle data document domain classification results of the vehicle data document; The segmentation module is used to obtain key image features from the image data using image segmentation enhancement technology, and to form image field fusion features based on the key image features and the main classification field features. The model building module is used to construct a multi-neural network vehicle data document image classification model using a multi-improved activation function neural network model; the activation function calculation formula of the j-th improved activation function neural network model in the multi-improved activation function neural network model is: In the formula, Let j be the activation function value of the improved activation function neural network model. The feature values ​​of the image field fusion features are linearly transformed by weights and biases. The number of training datasets for training the j-th improved activation function neural network model is given by [reference to dataset number]. Total amount of training dataset; The training module is used to input the fused features of the image fields into the multi-neural network vehicle data document image classification model for training; The classification module is used to obtain the image field fusion features of the car data document to be classified, and input them into the trained multi-neural network car data document image classification model to obtain the car data document domain classification result of the car data document to be classified.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements a classification method based on automotive document image enhancement as described in any one of claims 1-6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the classification method based on automotive document image enhancement as described in any one of claims 1-6.

Images