Vehicle parts document processing method, device, equipment and storage medium

Abstract

The present invention discloses a vehicle parts document processing method, device, equipment, and storage medium. The method comprises: converting a vehicle parts document to be processed into a document image; recognizing text in the document image to obtain document information for each portion of text in the document image; determining the entity type of each portion of text based on the document information and the document image, and processing the document information according to a processing method corresponding to the entity type to obtain a structured document corresponding to the vehicle parts document. The present invention converts a vehicle parts document into a document image, determines the entity type of each portion of text based on the document information obtained by recognizing text in the document image and the document image, and processes the document information into a structured document according to a processing method corresponding to the entity type. The method can extract part information based on the structured document, thereby reducing the workload of manual reading and screening and improving the efficiency of part information extraction.

Description

Technical Field

[0001] The present invention relates to the technical field of document processing, and in particular to a vehicle parts document processing method, device, equipment and storage medium. Background Art

[0002] With the development of technology, vehicles are becoming more and more intelligent, with more and more parts, and the corresponding standard documents are becoming more and more complicated. When it is necessary to obtain information about certain parts from standard documents, it is generally done by manually reading the standard documents and filtering out the required parts information from the standard documents. This work efficiency is low and cannot meet production needs.

[0003] The above content is only used to assist in understanding the technical solution of the present invention and does not constitute an admission that the above content is prior art. Summary of the Invention

[0004] The main purpose of the present invention is to provide a vehicle parts document processing method, device, equipment and storage medium, aiming to solve the technical problem of low efficiency in obtaining part information from part standard documents in the prior art.

[0005] To achieve the above object, the present invention provides a vehicle parts document processing method, the method comprising the following steps:

[0006] Converting a vehicle part document to be processed into a document image;

[0007] Recognizing the text in the document image to obtain document information of each part of the text in the document image;

[0008] The entity type of each part of the text is determined according to the document information and the document image, and the document information is processed according to a processing method corresponding to the entity type to obtain a structured document corresponding to the vehicle part document.

[0009] Optionally, converting the vehicle part document to be processed into a document image includes:

[0010] Get image conversion parameters;

[0011] converting the vehicle part document into an initial document image according to the image conversion parameters;

[0012] The initial document image is grayscaled, and the grayscaled image is scaled to obtain a document image.

[0013] Optionally, the recognizing text in the document image to obtain document information of each part of the text in the document image includes:

[0014] Inputting the document image into a preset text detection network to obtain text detection frames of each part of the text and position information of the text detection frames;

[0015] Expanding the corresponding text detection frame according to the position information to obtain expanded text detection frames for each part of the text;

[0016] cropping the document image according to the expanded text detection frame to obtain text images of each part of the text;

[0017] The text image is input into a preset text recognition network to obtain document information of each part of the text.

[0018] Optionally, the grayscale processing of the initial document image includes:

[0019] grayscale the initial document image using a preset grayscale formula;

[0020] Wherein, the preset grayscale formula is:

[0021] pixel(x,y)=0.299*r(x,y)+0.587*g(x,y)+0.114*b(x,y)

[0022] Where pixel(x,y) is the color value after grayscale processing; r(x,y) is the color value of the red channel before grayscale processing; g(x,y) is the color value of the green channel before grayscale processing; b(x,y) is the color value of the blue channel before grayscale processing.

[0023] Optionally, inputting the text image into a preset text recognition network to obtain document information of each part of the text includes:

[0024] Selecting a horizontal detection point on the text image according to the position information;

[0025] determining an angle between a text detection frame in the text image and a horizontal direction according to the horizontal detection point;

[0026] The text detection frame in the text image is adjusted to a horizontal direction according to the angle, and the adjusted text image is input into a preset text recognition network to obtain document information of each part of the text.

[0027] Optionally, the document information includes page number information, text detection frame information and text information;

[0028] The determining of the entity type of each portion of text based on the document information and the document image, processing the document information according to a processing method corresponding to the entity type, and obtaining a structured document corresponding to the vehicle part document includes:

[0029] Constructing a page number feature vector according to the page number information and the text detection frame information;

[0030] Performing high-level feature extraction on the text image corresponding to the document image to obtain a text feature vector;

[0031] Converting the text information into corresponding text feature vectors through a word embedding model;

[0032] constructing a three-dimensional feature vector based on the page number feature vector, the text feature vector, and the character feature vector;

[0033] Inputting the three-dimensional feature vector into a preset classification model to obtain entity types of each part of the text, wherein the entity types include document title, national standard code, paragraph title, table, image, formula and comment;

[0034] The document information and the document image are processed according to a processing method corresponding to the entity type to obtain a structured document corresponding to the vehicle part document.

[0035] Optionally, the processing the document information according to a processing method corresponding to the entity type includes:

[0036] When the entity type is a formula, cutting out a formula image on the document image according to a text detection frame corresponding to the formula; and

[0037] When the entity type is a document title, the text line is determined according to the position information of the text detection box, and the text information is merged according to the text line.

[0038] In addition, to achieve the above-mentioned purpose, the present invention further provides a vehicle parts document processing device, the device comprising:

[0039] A conversion module, used for converting a vehicle part document to be processed into a document image;

[0040] A recognition module, configured to recognize text in the document image and obtain document information of each part of the text in the document image;

[0041] A processing module is used to determine the entity type of each part of the text according to the document information and the document image, process the document information according to a processing method corresponding to the entity type, and obtain a structured document corresponding to the vehicle part document.

[0042] In addition, to achieve the above-mentioned purpose, the present invention also proposes a vehicle parts document processing device, which includes: a memory, a processor, and a vehicle parts document processing program stored on the memory and runnable on the processor, and the vehicle parts document processing program is configured to implement the steps of the vehicle parts document processing method described above.

[0043] In addition, to achieve the above-mentioned purpose, the present invention also proposes a storage medium, on which a vehicle parts document processing program is stored. When the vehicle parts document processing program is executed by a processor, the steps of the vehicle parts document processing method described above are implemented.

[0044] The present invention converts a vehicle part document to be processed into a document image; recognizes text in the document image to obtain document information for each portion of text in the document image; determines the entity type of each portion of text based on the document information and the document image, processes the document information according to a processing method corresponding to the entity type, and obtains a structured document corresponding to the vehicle part document. The present invention converts a vehicle part document into a document image, determines the entity type of each portion of text based on the document information obtained by recognizing text in the document image and the document image, processes the document information into a structured document according to a processing method corresponding to the entity type, and can extract part information based on the structured document, reducing the workload of manual reading and screening and improving the efficiency of part information extraction. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 It is a structural diagram of a vehicle parts document processing device in a hardware operating environment involved in an embodiment of the present invention;

[0046] Figure 2 1. A schematic flow chart of a first embodiment of a vehicle parts document processing method according to the present invention;

[0047] Figure 3 1. A schematic flow chart of a second embodiment of a vehicle parts document processing method according to the present invention;

[0048] Figure 4 1. A schematic flow chart of a third embodiment of a vehicle parts document processing method according to the present invention;

[0049] Figure 5 This is a structural block diagram of the first embodiment of the vehicle parts document processing device of the present invention.

[0050] The purpose, features and advantages of the present invention will be further described with reference to the accompanying drawings and in conjunction with the embodiments. DETAILED DESCRIPTION

[0051] It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0052] Reference Figure 1 , Figure 1 This is a schematic diagram of the structure of a vehicle parts document processing device in the hardware operating environment involved in an embodiment of the present invention.

[0053] like Figure 1 As shown, the vehicle parts document processing device may include: a processor 1001, such as a central processing unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. The communication bus 1002 is used to enable communication between these components. The user interface 1003 may include a display and an input unit such as a keyboard. Optionally, the user interface 1003 may also include a standard wired interface or a wireless interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a wireless fidelity (WI-FI) interface). The memory 1005 may be a high-speed random access memory (RAM) or a stable non-volatile memory (NVM), such as a disk storage device. The memory 1005 may also be a storage device independent of the processor 1001.

[0054] Those skilled in the art will understand that Figure 1 The structure shown in the figure does not constitute a limitation to the vehicle parts document processing device, and may include more or less components than shown in the figure, or combine certain components, or arrange the components differently.

[0055] like Figure 1 As shown, the memory 1005 as a storage medium may include an operating system, a network communication module, a user interface module, and a vehicle parts document processing program.

[0056] exist Figure 1 In the vehicle parts document processing device shown, the network interface 1004 is mainly used for data communication with the network server; the user interface 1003 is mainly used for data interaction with the user; the processor 1001 and the memory 1005 in the vehicle parts document processing device of the present invention can be set in the vehicle parts document processing device, and the vehicle parts document processing device calls the vehicle parts document processing program stored in the memory 1005 through the processor 1001, and executes the vehicle parts document processing method provided by the embodiment of the present invention.

[0057] The embodiment of the present invention provides a vehicle parts document processing method, referring to Figure 2 , Figure 2 2 is a flow chart of a first embodiment of a vehicle parts document processing method according to the present invention.

[0058] In this embodiment, the vehicle parts document processing method includes the following steps:

[0059] Step S10: Convert the vehicle part document to be processed into a document image.

[0060] It should be noted that the execution subject of this embodiment may be a computing service device with data processing, network communication, and program execution functions, such as a tablet computer, personal computer, mobile phone, etc., or an electronic device capable of performing the above functions, such as a vehicle parts document processing device. This embodiment and the following embodiments will be described below using a vehicle parts processing device as an example.

[0061] It can be understood that the vehicle parts document can be a national standard vehicle parts standard document, and the vehicle parts document is generally a document in PDF format; the document image can be an image corresponding to the vehicle parts document; converting the vehicle parts document to be processed into a document image can be converting the vehicle parts document in PDF format into a corresponding document image through a preset tool.

[0062] Step S20: Recognize the text in the document image to obtain document information of each part of the text in the document image.

[0063] It can be understood that recognizing the text in the document image can be performed by first detecting the text in the document image and then performing text recognition on the text detected in the document image; the document information can be the text information of each part in the document, and the document information includes the position information and text information of each part of the text in the document image.

[0064] Step S30: determining the entity type of each part of the text according to the document information and the document image, processing the document information according to a processing method corresponding to the entity type, and obtaining a structured document corresponding to the vehicle part document.

[0065] It can be understood that the entity type can be the type of each part of the text in the document image, and the entity types include document titles, national standard codes, paragraph titles, tables, images, formulas and comments, etc. The entity type can be determined based on the relevant formats of the national standards for vehicle parts, document reconstruction algorithms and actual usage requirements; determining the entity type of each part of the text based on the document information and the document image can be to construct a feature vector based on the document information and the document image, input the feature vector into a preset model, and obtain the entity type corresponding to each part of the text; a structured document can be a document obtained by extracting the text information in the document information according to the entity type of each part of the text, and recombining the extracted text information according to the entity type; for example: the entity type of the text is a formula. Since the formula contains a large number of special characters, it is suitable to be presented in the form of an image. At this time, the corresponding processing method is to crop the text corresponding to the formula part, obtain the formula image corresponding to the formula, and place the formula image at the corresponding position in the structured document according to the position information of the formula.

[0066] In the specific implementation, the vehicle parts standard document in PDF format is converted into the corresponding document image through a preset image conversion tool. The document image is first subjected to text detection, and then the detected text is subjected to text recognition to obtain the document information corresponding to each part of the text in the document image. A feature vector is constructed based on the document information and the document image. The feature vector is input into a preset model to obtain the entity type corresponding to each part of the text. The corresponding document information is processed according to the processing method corresponding to the entity type to obtain a structured document corresponding to the vehicle parts standard document.

[0067] Furthermore, in order to convert the vehicle parts standard document into a structured document, the step S10 includes: obtaining image conversion parameters; converting the vehicle parts document into an initial document image according to the image conversion parameters; grayscale processing the initial document image, and performing image scaling processing on the grayscale processed image to obtain a document image.

[0068] It can be understood that the image conversion parameters can be parameters for converting a vehicle parts document into a document image, and the image conversion parameters include the starting page number, the ending page number, the rotation angle, the resolution and the scaling factor, etc.; converting the vehicle parts document into the initial document image according to the image conversion parameters can be based on the image conversion parameters, using an image conversion tool to convert a standard part document in PDF format into an initial document image, and the image conversion tool can be a PyMuPDF tool.

[0069] In the specific implementation, the parameters such as the start page number, end page number, rotation angle, resolution and zoom factor input by the user are obtained. The parameters are used to control the PyMuPDF tool to convert the vehicle parts document in PDF format into a document image. The converted document image is grayscaled and scaled to obtain the final document image. The conversion process of the vehicle parts document in PDF format can be referred to the following formula:

[0070] (image,page_num)=process(pdf_convert(pdf,page_range,config))

[0071] Where process is the image processing process, (image, page_num) is the output tuple of image and page number, pdf_convert represents the PDF conversion process, and (pdf, page_range, config) is the PDF file, page number range, and related conversion configuration information.

[0072] Furthermore, in order to improve the accuracy of text detection and text recognition, the grayscale processing of the initial document image includes: grayscale processing of the initial document image using a preset grayscale formula; wherein the preset grayscale formula is:

[0073] pixel(x,y)=0.299*r(x,y)+0.587*g(x,y)+0.114*b(x,y)

[0074] Where pixel(x,y) is the color value after grayscale processing; r(x,y) is the color value of the red channel before grayscale processing; g(x,y) is the color value of the green channel before grayscale processing; b(x,y) is the color value of the blue channel before grayscale processing.

[0075] It is understandable that scaling the grayscaled image to obtain the final document image may be performed by obtaining the height and width of the grayscaled document image, and when the height is less than the width, scaling the grayscaled document image using a first preset scaling formula to obtain the final document image; wherein the first preset scaling formula is:

[0076]

[0077] Wherein, short_size is the set short side length, height is the height of the grayscale document image, width is the width of the grayscale document image, new_height is the height of the final document image, and new_width is the width of the final document image; when the height is greater than the width, the grayscale document image is scaled using a second preset scaling formula to obtain the final document image; wherein the second preset scaling formula is:

[0078]

[0079] This embodiment converts a vehicle parts document to be processed into a document image; recognizes text in the document image to obtain document information for each portion of text in the document image; determines the entity type of each portion of text based on the document information and the document image, processes the document information according to a processing method corresponding to the entity type, and obtains a structured document corresponding to the vehicle parts document. This embodiment converts a vehicle parts document into a document image, determines the entity type of each portion of text based on the document information obtained by recognizing text in the document image and the document image, processes the document information into a structured document according to a processing method corresponding to the entity type, and can extract parts information based on the structured document, reducing the workload of manual reading and screening and improving the efficiency of parts information extraction.

[0080] refer to Figure 3 , Figure 3 2 is a flow chart of a second embodiment of a vehicle parts document processing method according to the present invention.

[0081] Based on the above first embodiment, in this embodiment, step S20 includes:

[0082] Step S201: inputting the document image into a preset text detection network to obtain text detection frames of each part of the text and position information of the text detection frames.

[0083] It can be understood that the preset text detection network can be a DB Net network; by inputting the document image into the DB Net network, the text detection boxes of each part of the text in the document image output by the network and the position information of the text detection boxes in the document image can be obtained; the position information can be the pixel coordinate information of the text detection box in the document image.

[0084] Step S202: expanding the corresponding text detection frame according to the position information to obtain an expanded text detection frame for each part of the text.

[0085] It can be understood that, according to the position information, the corresponding text detection frame is expanded to obtain the expanded text detection frame of each part of the text. The vertex pixel coordinates of the text detection frame are determined according to the position information of the text detection frame, and the text detection frame corresponding to each part of the text is expanded according to the vertex pixel coordinates using a preset expansion formula to obtain the expanded text detection frame of each part of the text; wherein the vertex pixel coordinates include the upper left vertex pixel coordinates, the lower left vertex pixel coordinates, the upper right vertex pixel coordinates, and the lower right vertex pixel coordinates; the preset expansion formula is:

[0086] P dst ((x lt ,y lt ),(x rt ,y rt ),(x lb ,y lb ),(x rb ,y rb ))=P src ((x lt -t,y lt -t),(x rt +t,y rt -t),(x lb -t,y lb +t),(x rb +t,y rb +t))

[0087] In the formula, (x lt ,y lt ) represents the pixel coordinates of the upper left vertex, (x rt ,y rt ) represents the pixel coordinates of the upper right vertex, (x lb ,y lb ) represents the pixel coordinates of the lower left vertex, (x rb ,y rb ) represents the pixel coordinates of the lower right vertex, t represents the expansion distance, P src represents the point before expansion, P dst represents the point after expansion.

[0088] Step S203: cropping the document image according to the expanded text detection frame to obtain text images of each part of the text;

[0089] Step S204: inputting the text image into a preset text recognition network to obtain document information of each part of the text.

[0090] It can be understood that the preset text recognition network can be a CRNN text recognition network; by inputting the text image into the CRNN text recognition network, the document information of each part of the text output by the CRNN text recognition network can be obtained.

[0091] In a specific implementation, the vehicle parts document processing device inputs a document image into a DB Net network, obtains the text detection boxes corresponding to each part of the text output by the network and the pixel coordinate position information of each text detection box in the document image, reads the upper left vertex pixel coordinates, lower left vertex pixel coordinates, upper right vertex pixel coordinates and lower right vertex pixel coordinates of the text detection box from the pixel coordinate position information, expands the text detection box according to the pixel coordinates of the four vertices and a preset expansion formula to obtain an expanded text detection box, cuts out each part of the text on the document image according to the expanded text detection box, obtains a text image corresponding to each part of the text, inputs the text image into the CRNN text detection network, and obtains the document information of each part of the text output by the CRNN text detection network.

[0092] Furthermore, in order to improve the accuracy of text recognition, the step S202 includes: selecting a horizontal detection point on the text image according to the position information; determining the angle between the text detection box in the text image and the horizontal direction according to the horizontal detection point; adjusting the text detection box in the text image to the horizontal direction according to the angle, and inputting the adjusted text image into a preset text recognition network to obtain document information of each part of the text.

[0093] It can be understood that the horizontal detection point can be a detection point used to detect whether the text detection box is in the horizontal direction. The horizontal detection point can be selected on the text detection box, for example, the upper left vertex and the upper right vertex of the text detection box are selected as horizontal detection points, or the lower left vertex and the lower right vertex are selected as horizontal detection points. Other points can also be selected as horizontal detection points. This embodiment does not limit this; determining the angle between the text detection box in the text image and the horizontal direction according to the horizontal detection point can be determining the coordinate position of the text detection box in a preset rectangular coordinate system according to the horizontal detection point, and determining the angle between the text detection box and the horizontal direction according to the coordinate position; the preset rectangular coordinate system can be a rectangular coordinate system constructed with the coordinates of the upper left vertex of the text image as the origin, the horizontal direction as the X-axis, and the vertical direction as the Y-axis.

[0094] In a specific implementation, for example, the selected horizontal detection points are the upper left vertex A and the upper right vertex B of the text detection frame, and the coordinates of the two horizontal detection points in the preset rectangular coordinate system are A: (x lt ,y lt ) and B:(x rt ,y rt ), calculate the first distance: x d =x rt -x lt ; Second distance: y d =y rt -y lt, if x d =0, then judge y d If the value of y d <0, then angel = 90 degrees, then rotate the text detection frame 90 degrees clockwise; assuming y d >0, then angel=-90 degrees, at this time the text detection frame is rotated 90 degrees counterclockwise; assuming y d =0, then angel = 0 degrees; if x d ≠0, the angle can be calculated by the inverse trigonometric function: angle = arctan(y d / x d ), rotate the text detection frame according to the calculated angle to adjust the text detection frame to the horizontal direction, input the adjusted text image into the CRNN text recognition network, and obtain the document information of each part of the text.

[0095] In this embodiment, the document image is input into a preset text detection network to obtain text detection frames and position information of each text portion; the corresponding text detection frames are expanded based on the position information to obtain expanded text detection frames for each text portion; the document image is cropped based on the expanded text detection frames to obtain text images of each text portion; and the text image is input into a preset text recognition network to obtain document information for each text portion. In this embodiment, by expanding the text detection frames to obtain expanded text detection frames and cropping the text image based on the expanded text detection frames, a larger text area can be obtained, thereby improving text recognition accuracy.

[0096] refer to Figure 4 , Figure 4 2 is a flow chart of a third embodiment of a vehicle parts document processing method according to the present invention.

[0097] Based on the above embodiments, in this embodiment, the document information includes page number information, text detection frame information, and text information. Step S30 includes:

[0098] Step S301: constructing a page number feature vector according to the page number information and the text detection frame information.

[0099] It can be understood that the page number information can be the page number of the vehicle parts document, and the text detection frame information can be the information of the text detection frame corresponding to each page number. The text detection frame information includes information such as the sequence number and position of the text detection frame. The page number feature vector of the document needs to be constructed into an N-dimensional vector based on the number N of text detection frames on the page. The vector is expressed as:

[0100] P=[page_num,0,0,......,0]

[0101] Where page_num is the page number. To make the vector dimensions the same, 0 is the filler value. The number of filler values ​​is determined by the number N of text detection boxes, and the number of filler values ​​= N-1.

[0102] Step S302: performing high-level feature extraction on the text image corresponding to the document image to obtain a text feature vector;

[0103] Step S303: Convert the text information into a corresponding text feature vector through a word embedding model.

[0104] It is understandable that high-level feature extraction can be performed on text images through mobile net to obtain text feature vectors; and word embedding can be used to construct text feature vectors corresponding to text information.

[0105] Step S304: constructing a three-dimensional feature vector according to the page number feature vector, the text feature vector and the character feature vector.

[0106] It can be understood that the three-dimensional feature vector can be: V = P N +CNN(I) N +wordembeding(T) N , where V represents the constructed three-dimensional feature vector, I represents the text image, and T represents the text information.

[0107] Step S305: Input the three-dimensional feature vector into a preset classification model to obtain the entity type of each part of the text, the entity type including document title, national standard code, paragraph title, table, image, formula and comment.

[0108] It is understandable that the preset classification model can be Bi-LSTM+CRF, and the three-dimensional feature vector is input into Bi-LSTM+CRF to perform entity classification on each part of the text to obtain the entity type of each part of the text.

[0109] In the specific implementation, page number information, text detection box information, text image and text information are utilized to construct a three-dimensional feature vector, which is input into Bi-LSTM+CRF for entity classification, then encoded by the encoder constructed by Bi-LSTM, and then decoded using CRF for dimensionality reduction and simple processing. The output format obtained is "index, transcript, entity", where index represents the serial number of the text detection box, transcript represents the text in the text detection box, and entity represents the entity type corresponding to the classification result.

[0110] Step S306: Process the document information and the document image according to a processing method corresponding to the entity type to obtain a structured document corresponding to the vehicle part document.

[0111] Furthermore, in order to improve the efficiency of extracting part information, the document information is processed according to the processing method corresponding to the entity type, including: when the entity type is a formula, cutting out a formula image on the document image according to the text detection box corresponding to the formula; and when the entity type is a document title, determining the text line according to the position information of the text detection box, and merging text information according to the text line.

[0112] In a specific implementation, the serial number of the text detection box and the text information corresponding to the text detection box and the entity type of the text information can be determined according to "index, transcript, entity". First, the entity type that will only appear once is determined, such as the document title, national standard code, etc., and the text information of this type of entity type is extracted; when the entity type is a paragraph title and comment, different text lines are divided according to the position information of the text detection box, and different text information is merged according to the text line; when the entity type is an image or a formula, the image containing the image and formula in the text image can be directly cropped according to the position information of the text detection box. For example, for formulas located in the same row, the upper left vertex and the lower right vertex can be determined, and the formula can be cropped out on the text image according to these two points. For general images, they can be cropped out directly on the document image; for paragraph titles, tables and comments, they can be sorted according to the position of the entities from front to back on the page number and from top to bottom on the current page. After processing different entity types, they are sorted according to the page number and position according to the extracted fields to obtain the structured document corresponding to the vehicle parts document.

[0113] This embodiment constructs a page number feature vector based on the page number information and the text detection frame information; performs high-level feature extraction on the text image corresponding to the document image to obtain a text feature vector; converts the text information into a corresponding text feature vector using a word embedding model; constructs a three-dimensional feature vector based on the page number feature vector, the text feature vector, and the text feature vector; inputs the three-dimensional feature vector into a preset classification model to obtain the entity type of each part of the text, the entity types including document title, national standard code, paragraph title, table, image, formula, and comment; processes the document information and the document image according to the processing method corresponding to the entity type to obtain a structured document corresponding to the vehicle parts document. This embodiment constructs a three-dimensional feature vector using the page number feature vector, the text feature vector, and the text feature vector; inputs the three-dimensional feature vector into a preset classification model to determine the entity type corresponding to each part of the text; processes the document information according to the method corresponding to the entity type to obtain a structured document; and extracts part information based on the structured document, thereby improving information acquisition efficiency.

[0114] Reference Figure 5 , Figure 5 This is a structural block diagram of the first embodiment of the vehicle parts document processing device of the present invention.

[0115] like Figure 5 As shown, the vehicle parts document processing device proposed in the embodiment of the present invention includes:

[0116] A conversion module 10, configured to convert a vehicle part document to be processed into a document image;

[0117] The recognition module 20 is used to recognize the text in the document image and obtain document information of each part of the text in the document image;

[0118] The processing module 30 is used to determine the entity type of each part of the text according to the document information and the document image, and process the document information according to the processing method corresponding to the entity type to obtain a structured document corresponding to the vehicle part document.

[0119] This embodiment converts a vehicle parts document to be processed into a document image; recognizes text in the document image to obtain document information for each portion of text in the document image; determines the entity type of each portion of text based on the document information and the document image, processes the document information according to a processing method corresponding to the entity type, and obtains a structured document corresponding to the vehicle parts document. This embodiment converts a vehicle parts document into a document image, determines the entity type of each portion of text based on the document information obtained by recognizing text in the document image and the document image, processes the document information into a structured document according to a processing method corresponding to the entity type, and can extract parts information based on the structured document, reducing the workload of manual reading and screening and improving the efficiency of parts information extraction.

[0120] Based on the first embodiment of the vehicle parts document processing device of the present invention, a second embodiment of the vehicle parts document processing device of the present invention is proposed.

[0121] In this embodiment, the conversion module 10 is further used to obtain image conversion parameters; convert the vehicle parts document into an initial document image according to the image conversion parameters; grayscale the initial document image, and perform image scaling on the grayscaled image to obtain a document image.

[0122] The recognition module 20 is also used to input the document image into a preset text detection network to obtain the text detection frame of each part of the text and the position information of the text detection frame; expand the corresponding text detection frame according to the position information to obtain the expanded text detection frame of each part of the text; crop the document image according to the expanded text detection frame to obtain the text image of each part of the text; input the text image into the preset text recognition network to obtain the document information of each part of the text.

[0123] The conversion module 10 is further configured to perform grayscale processing on the initial document image using a preset grayscale formula; wherein the preset grayscale formula is:

[0124] pixel(x,y)=0.299*r(x,y)+0.587*g(x,y)+0.114*b(x,y)

[0125] Where pixel(x,y) is the color value after grayscale processing; r(x,y) is the color value of the red channel before grayscale processing; g(x,y) is the color value of the green channel before grayscale processing; b(x,y) is the color value of the blue channel before grayscale processing.

[0126] The recognition module 20 is also used to select a horizontal detection point on the text image according to the position information; determine the angle between the text detection box in the text image and the horizontal direction according to the horizontal detection point; adjust the text detection box in the text image to the horizontal direction according to the angle, and input the adjusted text image into a preset text recognition network to obtain document information of each part of the text.

[0127] The processing module 30 is further used to construct a page number feature vector based on the page number information and the text detection frame information; perform high-level feature extraction on the text image corresponding to the document image to obtain a text feature vector; convert the text information into a corresponding text feature vector through a word embedding model; construct a three-dimensional feature vector based on the page number feature vector, the text feature vector and the text feature vector; input the three-dimensional feature vector into a preset classification model to obtain the entity type of each part of the text, the entity type including document title, national standard code, paragraph title, table, image, formula and annotation; process the document information and the document image according to the processing method corresponding to the entity type to obtain a structured document corresponding to the vehicle parts document; the document information includes page number information, text detection frame information and text information.

[0128] The processing module 30 is also used to, when the entity type is a formula, crop a formula image on the document image based on a text detection box corresponding to the formula; and when the entity type is a document title, determine a text line based on position information of the text detection box, and merge text information based on the text line.

[0129] Other embodiments or specific implementations of the vehicle parts document processing device of the present invention can refer to the above-mentioned method embodiments and will not be repeated here.

[0130] It should be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or system comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or system. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or system comprising the element.

[0131] The serial numbers of the above embodiments of the present invention are for description only and do not represent the advantages or disadvantages of the embodiments.

[0132] Through the description of the above embodiments, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better embodiment. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as read-only memory / random access memory, magnetic disk, optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present invention.

[0133] The above are only preferred embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention description and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A vehicle parts document processing method, characterized in that: The method comprises: Converting a vehicle part document to be processed into a document image; Recognizing the text in the document image to obtain document information of each part of the text in the document image; Determining the entity type of each portion of text based on the document information and the document image, processing the document information according to a processing method corresponding to the entity type, and obtaining a structured document corresponding to the vehicle part document; The document information includes page number information, text detection frame information and text information; The determining of the entity type of each portion of text based on the document information and the document image, processing the document information according to a processing method corresponding to the entity type, and obtaining a structured document corresponding to the vehicle part document includes: Constructing a page number feature vector according to the page number information and the text detection frame information; Performing high-level feature extraction on the text image corresponding to the document image to obtain a text feature vector; Converting the text information into corresponding text feature vectors through a word embedding model; constructing a three-dimensional feature vector based on the page number feature vector, the text feature vector, and the character feature vector; Inputting the three-dimensional feature vector into a preset classification model to obtain entity types of each part of the text, wherein the entity types include document title, national standard code, paragraph title, table, image, formula and comment; The document information and the document image are processed according to a processing method corresponding to the entity type to obtain a structured document corresponding to the vehicle part document.

2. The method according to claim 1, wherein The step of converting the to-be-processed vehicle part document into a document image comprises: Get image conversion parameters; converting the vehicle part document into an initial document image according to the image conversion parameters; The initial document image is grayscaled, and the grayscaled image is scaled to obtain a document image.

3. The method according to claim 1, wherein The step of recognizing the text in the document image to obtain document information of each part of the text in the document image includes: Inputting the document image into a preset text detection network to obtain text detection frames of each part of the text and position information of the text detection frames; Expanding the corresponding text detection frame according to the position information to obtain expanded text detection frames for each part of the text; cropping the document image according to the expanded text detection frame to obtain text images of each part of the text; The text image is input into a preset text recognition network to obtain document information of each part of the text.

4. The method according to claim 2, wherein The grayscale processing of the initial document image includes: grayscale the initial document image using a preset grayscale formula; Wherein, the preset grayscale formula is: pixel(x,y)=0.299*r(x,y)+0.587*g(x,y)+0.114*b(x,y) Where pixel(x,y) is the color value after grayscale processing; r(x,y) is the color value of the red channel before grayscale processing; g(x,y) is the color value of the green channel before grayscale processing; b(x,y) is the color value of the blue channel before grayscale processing.

5. The method according to claim 3, wherein The step of inputting the text image into a preset text recognition network to obtain document information of each part of the text includes: Selecting a horizontal detection point on the text image according to the position information; determining an angle between a text detection frame in the text image and a horizontal direction according to the horizontal detection point; The text detection frame in the text image is adjusted to a horizontal direction according to the angle, and the adjusted text image is input into a preset text recognition network to obtain document information of each part of the text.

6. The method according to claim 1, wherein The processing of the document information according to the processing method corresponding to the entity type includes: When the entity type is a formula, cutting out a formula image on the document image according to a text detection frame corresponding to the formula; and When the entity type is a document title, the text line is determined according to the position information of the text detection box, and the text information is merged according to the text line.

7. A vehicle parts document processing device, characterized in that: The device comprises: A conversion module, used for converting a vehicle part document to be processed into a document image; A recognition module, configured to recognize text in the document image and obtain document information of each part of the text in the document image; a processing module, configured to determine the entity type of each portion of text based on the document information and the document image, and process the document information according to a processing method corresponding to the entity type to obtain a structured document corresponding to the vehicle part document; The document information includes page number information, text detection frame information and text information; The processing module is further configured to construct a page number feature vector based on the page number information and the text detection frame information; Performing high-level feature extraction on the text image corresponding to the document image to obtain a text feature vector; Converting the text information into corresponding text feature vectors through a word embedding model; constructing a three-dimensional feature vector based on the page number feature vector, the text feature vector, and the character feature vector; Inputting the three-dimensional feature vector into a preset classification model to obtain entity types of each part of the text, wherein the entity types include document title, national standard code, paragraph title, table, image, formula and comment; The document information and the document image are processed according to a processing method corresponding to the entity type to obtain a structured document corresponding to the vehicle part document.

8. A vehicle parts document processing device, characterized in that: The device includes: a memory, a processor, and a vehicle part document processing program stored in the memory and executable on the processor, wherein the vehicle part document processing program is configured to implement the steps of the vehicle part document processing method according to any one of claims 1 to 6.

9. A storage medium, characterized in that: The storage medium stores a vehicle parts document processing program, which, when executed by a processor, implements the steps of the vehicle parts document processing method according to any one of claims 1 to 6.

Images