A method and platform for converting and generating drawings by drawing analysis

Abstract

The present invention discloses a method and platform for converting drawings by drawing analysis, and relates to the field of image processing. The method comprises: placing the drawings to be converted on a preset platform for scanning to obtain the digital images of the drawings to be converted; performing separation through an image segmentation network; performing character recognition on the separated character image list and the separated character distribution coordinate list to obtain text information and dimensioning; performing graphic recognition on the separated graphic image list and the separated graphic distribution coordinate list to obtain graphic features; performing vector image configuration to generate editable digital drawings; receiving drawing conversion constraint information, performing editing optimization, generating target conversion drawings, and sending them to the user end; when the user feedback information includes a conversion permission instruction, executing printing control on the target conversion drawings. The method solves the technical problems of low efficiency and insufficient precision in the existing digital conversion of drawings, and achieves the technical effect of improving conversion efficiency and ensuring conversion precision.

Description

Technical Field

[0001] The present application relates to the field of image processing, and in particular to a method and platform for converting and generating drawings through drawing analysis. Background Art

[0002] In the field of drafting and engineering design, drawings are important tools for conveying design ideas and guiding construction and manufacturing. With the continuous development of information technology, the digital conversion of drawings has become a common demand in the industry. The traditional conversion process of drawings to vector graphics mainly relies on manual operations, including scanning of drawings, recognition of characters and graphics, input and editing of information, and other links. However, this manual processing method requires a lot of manual operations when facing the demand for batch drawing conversion, resulting in low conversion efficiency. During the manual processing process, due to factors such as fatigue and skill differences, the recognition accuracy often decreases, affecting the quality of the converted vector graphics.

[0003] Among the current related technologies, the digital conversion of drawings has technical problems such as low efficiency and insufficient precision. Summary of the invention

[0004] The present application provides a method and platform for drawing conversion generation by drawing analysis. The digital image of the drawing to be converted is obtained through an image scanning device, and the image is separated by an image segmentation network to obtain information such as character image, character distribution coordinates, graphic image and graphic distribution coordinates, with directional constraint labels. Based on this information, character recognition and graphic recognition are performed to obtain text information, dimension annotations and graphic features. Then, vector images are configured according to this information to generate editable digital drawings. Finally, drawing conversion constraint information is received, the editable digital drawings are edited and optimized, and target conversion drawings are generated and sent to the user end for feedback. When the user allows the conversion, printing control and other technical means are executed, thereby achieving the technical effect of improving conversion efficiency and ensuring conversion accuracy.

[0005] The present application provides a method for converting and generating drawings by parsing drawings, including:

[0006] The drawing to be converted is placed on a preset platform, and the preset platform is scanned by an image scanning device to obtain a digital image of the drawing to be converted; the digital image of the drawing to be converted is separated by an image segmentation network to obtain a separated character image list, a separated character distribution coordinate list, a separated graphic image list and a separated graphic distribution coordinate list, wherein the separated character image and the separated graphic image have a direction constraint label; based on the direction constraint label, character recognition is performed on the separated character image list and the separated character distribution coordinate list to obtain text information and dimension annotation; based on the direction constraint label, graphic recognition is performed on the separated graphic image list and the separated graphic distribution coordinate list to obtain graphic features; vector image configuration is performed according to the text information, the dimension annotation and the graphic features to generate an editable digital drawing; drawing conversion constraint information is received, editing optimization is performed on the editable digital drawing, a target conversion drawing is generated, and the drawing is sent to a user end, and user feedback information is received; when the user feedback information includes an instruction to allow conversion, printing control is performed on the target conversion drawing.

[0007] In a possible implementation, the to-be-converted drawing digital image is separated by an image segmentation network to obtain a separated character image list, a separated character distribution coordinate list, a separated graphic image list, and a separated graphic distribution coordinate list, wherein the separated character image and the separated graphic image have direction constraint labels, and the following processing is performed:

[0008] Through the user end, the direction of the digital image of the drawing to be converted is marked to generate the direction constraint label; the image segmentation network includes a character separation channel and a graphic separation channel; through the character separation channel, the direction constraint label and the digital image of the drawing to be converted are processed to generate the separated character image list and the separated character distribution coordinate list; through the graphic separation channel, the direction constraint label and the digital image of the drawing to be converted are processed to generate the separated graphic image list and the separated graphic distribution coordinate list.

[0009] In a possible implementation, the direction constraint label and the to-be-converted drawing digital image are processed through the character separation channel to generate the separated character image list and the separated character distribution coordinate list, and the following processing is performed:

[0010] Through the encoder, according to the preset direction, based on the graphic database and the character database, a training sample image set and a character distribution coordinate identification data set are randomly generated, wherein any training sample image has at least one graphic element and one character element; the character distribution coordinate identification data set is used as the supervision data, and the training sample image set is used as the input data of the convolutional neural network to train the character separation channel.

[0011] In a possible implementation, the character distribution coordinate identification data set is used as supervision data, the training sample image set is used as input data of a convolutional neural network, the character separation channel is trained, and the following processing is performed:

[0012] Construct the character separation training loss function: ,in, Represents any training loss, Represents the coordinates of the ith character separation mark, Represents the supervised distribution coordinates of the i-th character, Characterization and The area of ​​the non-intersecting region, Characterization and The union area, The loss ratio threshold is represented, Q represents the number of characters trained at any time, and e is a constant; the character distribution coordinate identification data set is used as the supervision data, and the training sample image set is used as the input data of the convolutional neural network. When the output value of the character separation training loss function is less than or equal to the loss threshold for a preset number of consecutive times, the character separation channel is obtained.

[0013] In a possible implementation, based on the direction constraint tag, character recognition is performed on the separated character image list and the separated character distribution coordinate list to obtain text information and dimension annotation, and the following processing is performed:

[0014] A character recognition network is obtained, wherein the character recognition network includes a feature extraction channel and a character classification channel; through the feature extraction channel, based on the direction constraint label, the separated character image list is traversed to perform character shape feature recognition, and a separated character shape feature list is obtained, wherein the separated character shape feature has a character shape feature direction label; through the character classification channel, based on a character database and the character shape feature direction label, the separated character shape feature list is traversed to perform matching, and a separated character matching semantic list is obtained; according to the separated character distribution coordinate list, the separated character matching semantic list is traversed to perform image position embedding, and the text information and the dimension annotation are generated.

[0015] In a possible implementation, the feature extraction channel is used to traverse the separated character image list to perform character shape feature recognition based on the direction constraint label, obtain a separated character shape feature list, and perform the following processing:

[0016] Collect a character image training data set, a character direction identification data set and a character shape feature descriptor true value data set to generate a feature extraction channel construction data set; divide the feature extraction channel construction data set into k equal parts of feature extraction channel construction data; perform k-times sampling with replacement on the k equal parts of feature extraction channel construction data to obtain a first feature extraction channel construction data set and construct a first feature extraction base channel; until the k equal parts of feature extraction channel construction data are sampled with replacement k times to obtain an M-th feature extraction channel construction data set and construct an M-th feature extraction base channel; construct a fully connected layer, fuse the first feature extraction base channel until the output mean of the M-th feature extraction base channel is obtained, and generate the feature extraction channel.

[0017] In a possible implementation, drawing conversion constraint information is received, editing optimization is performed on the editable digital drawing, a target conversion drawing is generated, and the following processing is performed:

[0018] The drawing conversion constraint information includes a graphic size requirement parameter and a graphic layout parameter; the editable digital drawing is edited according to the image size requirement parameter and the graphic layout parameter to obtain a plurality of initial drawing schemes; the plurality of initial drawing schemes are traversed to perform a graphic volume summing analysis to obtain a plurality of graphic volume sum total values; the minimum value of the plurality of graphic volume sum total values ​​is extracted and set as the target conversion drawing.

[0019] The present application also provides a platform for drawing parsing and drawing conversion generation, including:

[0020] A drawing scanning module to be converted, the drawing scanning module to be converted is used to place the drawing to be converted on a preset platform, scan the preset platform through an image scanning device, and obtain a digital image of the drawing to be converted; a digital image separation module, the digital image separation module is used to separate the digital image of the drawing to be converted through an image segmentation network, and obtain a separated character image list, a separated character distribution coordinate list, a separated graphic image list and a separated graphic distribution coordinate list, wherein the separated character image and the separated graphic image have a direction constraint label; a character recognition module, the character recognition module is used to perform character recognition on the separated character image list and the separated character distribution coordinate list based on the direction constraint label, and obtain text information and size label. Note: Graphic recognition module, the graphic recognition module is used to perform graphic recognition on the separated graphic image list and the separated graphic distribution coordinate list based on the direction constraint label to obtain graphic features; vector image configuration module, the vector image configuration module is used to perform vector image configuration according to the text information, the dimension annotation and the graphic features to generate an editable digital drawing; editing optimization module, the editing optimization module is used to receive drawing conversion constraint information, perform editing optimization on the editable digital drawing, generate a target conversion drawing, send it to the user end, and receive user feedback information; printing control module, the printing control module is used to perform printing control on the target conversion drawing when the user feedback information includes an instruction to allow conversion.

[0021] A method and platform for drawing conversion generation through drawing parsing proposed in this application, firstly, the drawing to be converted is placed on a preset platform, the preset platform is scanned by an image scanning device, and a digital image of the drawing to be converted is obtained, then the digital image of the drawing to be converted is separated by an image segmentation network, and a separated character image list, a separated character distribution coordinate list, a separated graphic image list and a separated graphic distribution coordinate list are obtained, wherein the separated character image and the separated graphic image have a direction constraint label, and then based on the direction constraint label, character recognition is performed on the separated character image list and the separated character distribution coordinate list to obtain text information and dimension annotation, based on the direction constraint label, graphic recognition is performed on the separated graphic image list and the separated graphic distribution coordinate list to obtain graphic features, and then vector image configuration is performed according to the text information, dimension annotation and graphic features to generate an editable digital drawing, and then the drawing conversion constraint information is received, and the editable digital drawing is edited and optimized to generate a target conversion drawing, which is sent to the user end, and user feedback information is received. When the user feedback information includes an instruction to allow conversion, printing control is performed on the target conversion drawing, thereby achieving the technical effect of improving conversion efficiency and ensuring conversion accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solution of the embodiment of the present invention, the accompanying drawings of the embodiment of the present invention are briefly introduced below. A flow chart is used in the present application to illustrate the operations performed by the platform according to the embodiment of the present application. It should be understood that the previous or following operations are not necessarily performed precisely in order. On the contrary, various steps can be processed in reverse order or simultaneously as needed. At the same time, other operations can also be added to these processes, or one or more operations can be removed from these processes.

[0023] Figure 1 A flowchart of a method for converting and generating drawings by parsing drawings provided in an embodiment of the present application.

[0024] Figure 2 A schematic diagram of the structure of a platform for drawing conversion and generation by drawing parsing provided in an embodiment of the present application.

[0025] Explanation of reference numerals: scanning module 10 for drawings to be converted, digital image separation module 20, character recognition module 30, graphic recognition module 40, vector image configuration module 50, editing and optimization module 60, printing control module 70. DETAILED DESCRIPTION

[0026] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below.

[0027] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings. The described embodiments should not be regarded as limiting the present application. All other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of this application.

[0028] In the following description, reference is made to "some embodiments", which describe a subset of all possible embodiments, but it is understood that "some embodiments" may be the same subset or different subsets of all possible embodiments, and may be combined with each other without conflict, and the terms "first\second" involved are merely to distinguish similar objects and do not represent a specific ordering of objects. The terms "including" and "having" and any variations are intended to cover non-exclusive inclusions, for example, a process, method, platform, product, or server that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or modules that are not clearly listed or inherent to these processes, methods, products, or devices. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those generally understood by technicians in the technical field of this application. The terms used herein are for the purpose of describing the embodiments of the present application only.

[0029] The present application embodiment provides a method for drawing conversion generation by drawing parsing, such as Figure 1 As shown, the method includes:

[0030] Step S100, placing the drawing to be converted on a preset platform, scanning the preset platform through an image scanning device, and obtaining a digital image of the drawing to be converted. Specifically, the paper drawing to be converted is placed on a preset scanning platform (a fixed area for placing the drawing to be scanned), and the drawing on the platform is scanned using an image scanning device (such as a scanner or camera) to ensure that the drawing content is captured completely and clearly. After the scanning is completed, a digital image file of the drawing to be converted is obtained, which can be in a format such as JPEG, PNG, TIFF, etc.

[0031] Step S200, the digital image of the drawing to be converted is separated by an image segmentation network to obtain a separated character image list, a separated character distribution coordinate list, a separated graphic image list and a separated graphic distribution coordinate list, wherein the separated character image and the separated graphic image have a direction constraint label. Specifically, the user identifies the direction of the digital image of the drawing to be converted through a user terminal (such as a computer, tablet or mobile phone), specifies the up, down, left and right directions of the drawing, and generates a direction constraint label. The digital image of the drawing to be converted is processed using an image segmentation network, which can automatically identify and separate different elements (such as characters and graphics) in the image, and output a separated character image list, a separated character distribution coordinate list, a separated graphic image list and a separated graphic distribution coordinate list.

[0032] In a possible implementation, the digital image of the drawing to be converted is separated by an image segmentation network to obtain a separated character image list, a separated character distribution coordinate list, a separated graphic image list and a separated graphic distribution coordinate list, wherein the separated character image and the separated graphic image have a direction constraint label, and step S200 further includes step S210, through the user terminal, the direction of the digital image of the drawing to be converted is marked, and the direction constraint label is generated. Specifically, the user views the digital image of the drawing to be converted through an application or web interface on the user terminal (such as a computer, tablet or mobile phone, etc.). The application provides a tool or interface element that allows the user to select a certain direction along the edge of the image (for example, by dragging a mouse or finger to draw a line on the screen). After the user selects the direction, the application automatically generates a direction constraint label based on the user's input. This label is a data tag used to indicate the direction of the characters and graphics in the image. Step S220, the image segmentation network includes a character separation channel and a graphic separation channel. Through the character separation channel, the direction constraint label and the digital image of the drawing to be converted are processed to generate the separated character image list and the separated character distribution coordinate list. Through the graphic separation channel, the direction constraint label and the digital image of the drawing to be converted are processed to generate the separated graphic image list and the separated graphic distribution coordinate list. Specifically, the image segmentation network is a deep learning model, which includes two independent channels: a character separation channel and a graphic separation channel. The character separation channel receives the direction constraint label and the digital image of the drawing to be converted as input and recognizes the characters in the image. The graphic separation channel also receives the direction constraint label and the digital image of the drawing to be converted as input, but recognizes the graphics in the image. The two channels respectively output a separated character image list (a list containing recognized character images) and a separated character distribution coordinate list (a list containing the position information of the recognized characters in the image), as well as a separated graphic image list (a list containing the recognized graphic images) and a separated graphic distribution coordinate list (a list containing the position information of the recognized graphics in the image). This implementation method performs direction identification on the user side, allowing users to participate in the drawing conversion process. Users can adjust the direction constraint labels according to their needs to obtain more desired conversion results. At the same time, by introducing direction constraint labels, the system can more accurately recognize characters and graphics in the image, especially when they are rotated or tilted, which helps to reduce recognition errors and improve the quality of the converted drawings. Characters and graphics can be recognized separately, which also improves recognition accuracy.

[0033] In a possible implementation, the direction constraint label and the digital image of the drawing to be converted are processed through the character separation channel to generate the separated character image list and the separated character distribution coordinate list, and step S220 further includes step S221, randomly generating a training sample image set and a character distribution coordinate identification data set based on a graphic database and a character database according to a preset direction through an encoder, wherein any training sample image has at least one graphic element and one character element. Specifically, an encoder is set, which can extract graphic elements and character elements from a graphic database (a database storing various graphic elements, such as straight lines, curves, circles, rectangles, etc.) and a character database (a database storing various characters, such as letters, numbers, symbols, etc.) according to preset directions and rules. The encoder randomly selects graphic elements and character elements from the graphic database and the character database according to preset directions (such as horizontal, vertical, etc.), and combines them together to form a training sample image, and each training sample image contains at least one graphic element and one character element to simulate the complex scenes in real drawings. For each generated training sample image, the encoder will also generate a character distribution coordinate identification data set, which records in detail the position of each character in the image (such as the upper left corner coordinates, the lower right corner coordinates or the center point coordinates, etc.) and the direction of the character (if the character is tilted, the tilt angle is recorded). Among them, the training sample image set contains a series of images used to train the character separation channel; the character distribution coordinate identification data set is a data set corresponding to the training sample image set, which is used to record the position and direction of each character in the image. Step S222, using the character distribution coordinate identification data set as supervision data and the training sample image set as input data of the convolutional neural network to train the character separation channel. Specifically, the character distribution coordinate identification data set is used as supervision data, that is, during the training process, the character separation channel needs to learn to recognize characters from the input image and accurately predict the position and direction of the characters. The training sample image set is used as the input data of the convolutional neural network, and the parameters of the convolutional neural network are continuously adjusted through multiple iterations (i.e., training rounds), so that the character separation channel can gradually learn to separate the character image from the input image and predict the distribution coordinates of the characters. This implementation method trains the character separation channel based on a large number of training sample images and character distribution coordinate identification data sets, making the character separation channel have strong generalization ability and able to handle character recognition tasks in various complex scenarios. It also uses a convolutional neural network model to improve recognition accuracy and efficiency. The construction process of the graphic separation channel is similar to that of the character separation channel, so it will not be repeated here.

[0034] In a possible implementation, the character distribution coordinate identification data set is used as supervision data, the training sample image set is used as input data of a convolutional neural network, and the character separation channel is trained. Step S222 further includes step S2221, constructing a character separation training loss function: ,in, Represents any training loss, Represents the coordinates of the ith character separation mark, Represents the supervised distribution coordinates of the i-th character, Characterization and The area of ​​the non-intersecting region, Characterization and The union area, Characterize the loss ratio threshold, Q characterizes the number of characters trained at any one time, and e is a constant. Specifically, the character separation training loss function is used to measure the difference between the predicted result of the character separation channel and the true result, and the loss is calculated by calculating the difference between the predicted character distribution coordinates and the supervised character distribution coordinates, wherein the non-intersection area refers to the area of ​​the non-overlapping part between the predicted character distribution coordinates and the supervised character distribution coordinates, and the union area is the total area covered by the two. By calculating the ratio of the non-intersection area area to the union area, an indicator for measuring the accuracy of the prediction is obtained. Step S2222, using the character distribution coordinate identification data set as the supervised data, and the training sample image set as the input data of the convolutional neural network, when the output value of the character separation training loss function is less than or equal to the loss threshold for a preset number of consecutive times, the character separation channel is obtained. Specifically, the character distribution coordinate identification data set is used as the supervised data, and the training sample image set is used as the input data of the convolutional neural network. The prediction result is calculated by forward propagation, and then the gradient is calculated and the network parameters are updated by back propagation. In each iteration, the character separation training loss function constructed in step S2221 is used to calculate the difference between the predicted result and the true result. If the output value of the loss function is less than or equal to the loss threshold for a preset number of consecutive times, the character separation channel is considered to have been trained and training can be stopped. This implementation method ensures that the character separation channel can accurately identify characters from the input image and predict their distribution coordinates by constructing a character separation training loss function and continuously optimizing it during the training process, thereby improving the accuracy of character recognition.

[0035] Step S300, based on the direction constraint label, character recognition is performed on the separated character image list and the separated character distribution coordinate list to obtain text information and dimension annotation. Specifically, character recognition is performed using a character recognition network, which can automatically recognize characters and convert them into readable text to obtain text information and dimension annotation.

[0036] In a possible implementation, based on the direction constraint label, character recognition is performed on the separated character image list and the separated character distribution coordinate list to obtain text information and dimension annotations. Step S300 further includes step S310 to obtain a character recognition network, wherein the character recognition network includes a feature extraction channel and a character classification channel. Specifically, a character recognition network is constructed, which is specifically used to recognize characters in drawings. This network consists of two parts: a feature extraction channel and a character classification channel. The feature extraction channel is responsible for extracting useful feature information from the input character image, including the character's outline, texture, edge, etc.; the character classification channel classifies the character image into predefined character categories based on the extracted feature information. Step S320, through the feature extraction channel, based on the direction constraint label, traverse the separated character image list to perform character shape feature recognition, and obtain a separated character shape feature list, wherein the separated character shape feature has a character shape feature direction label. Specifically, for each character image in the separated character image list, the following operations are performed: when extracting character shape features, the direction constraint label of the character is considered, because the character may have different directions (such as rotation, tilt, etc.) in the drawing, and these direction information is crucial for accurately recognizing the character, and the shape features of the character, including the outline, edge, texture, etc. of the character, are extracted from the character image using the feature extraction channel, and the extracted character shape features and their corresponding direction labels are stored in the separated character shape feature list, that is, the separated character shape feature list stores the character shape features extracted from the separated character image list and their direction labels. Step S330, through the character classification channel, based on the character database and the character shape feature direction label, the separated character shape feature list is traversed for matching to obtain a separated character matching semantic list. Specifically, for each character shape feature in the separated character shape feature list, the following operations are performed: during the matching process, the direction label of the character shape feature is considered to ensure the accuracy of the matching, and the character shape feature is matched with the character in the character database, and the character database contains various possible characters and their corresponding semantic information, and the matched characters and their semantic information are stored in the separated character matching semantic list. Step S340, according to the separated character distribution coordinate list, traverse the separated character matching semantic list to embed the image position, and generate the text information and the dimension annotation. Specifically, for each character and its semantic information in the separated character matching semantic list, the following operation is performed: according to the coordinate information in the separated character distribution coordinate list, the character and its semantic information are embedded into the corresponding position of the original drawing image, and through the embedding operation, text information and dimension annotation containing all recognized characters and their semantic information are generated.This implementation method utilizes the feature extraction channel and character classification channel in the character recognition network to identify the characters and their semantic information in the drawing. Through the image position embedding operation, the identified characters and their semantic information are embedded into the corresponding positions of the original drawing image, thereby generating text information and dimension annotations containing all necessary information. This not only improves the accuracy of character recognition, but also makes the subsequent vector image configuration and drawing conversion process more efficient and accurate.

[0037] In a possible implementation, the separated character image list is traversed through the feature extraction channel based on the direction constraint label to perform character shape feature recognition, and a separated character shape feature list is obtained. Step S320 further includes step S321, collecting a character image training data set, a character direction identification data set, and a character shape feature descriptor true value data set to generate a feature extraction channel construction data set. Specifically, a large number of character images are collected from various drawings. These images contain various possible character types, fonts, sizes, and rotation directions. These images are used to train the feature extraction channel so that it can accurately identify the shape features of the characters. For each character image, its direction information is recorded by manual annotation or using an automatic method. An accurate character shape feature descriptor true value is provided for each character image by manual annotation or using a character recognition algorithm. The true value data set is used to adjust the parameters of the feature extraction channel during the training process to improve its accuracy. The above three data sets are combined together to form a feature extraction channel construction data set. This data set is used to train the feature extraction channel so that it can accurately extract shape features from character images and consider the direction information of the characters. Step S322, the feature extraction channel construction data set is divided into k equal parts of feature extraction channel construction data. Specifically, the feature extraction channel construction data set is divided into k equal parts by random sampling, and each data contains various possible character types, fonts, sizes and rotation directions. Step S323, k equal parts of the feature extraction channel construction data are sampled with replacement k times to obtain the first feature extraction channel construction data set, and the first feature extraction base channel is constructed, until the k equal parts of the feature extraction channel construction data are sampled with replacement k times to obtain the Mth feature extraction channel construction data set, and the Mth feature extraction base channel is constructed. Specifically, for the k parts of data after the feature extraction channel construction data set is divided, M cycles of sampling are performed. In each cycle, a part of samples are randomly extracted from each part of the data (with replacement sampling), so that each part of the data has the possibility of being selected in each cycle. For each data set obtained by the cycle sampling (including samples extracted from the k parts of the data), a feature extraction base channel is constructed, that is, after M cycles, M feature extraction base channels are obtained. Step S324, construct a fully connected layer, fuse the first feature extraction base channel until the output mean of the Mth feature extraction base channel, and generate the feature extraction channel. Specifically, in the fully connected layer, the outputs of the M feature extraction base channels are fused by calculating the mean, and the fused output is used as the final output of the feature extraction channel.This implementation method constructs multiple feature extraction base channels and fuses their outputs, taking into account various possible character types, fonts, sizes, and rotation directions. The use of sampling with replacement and data set partitioning ensures that each base channel can learn useful information from the original data set without information loss due to data partitioning, thereby improving the accuracy of character feature recognition.

[0038] Step S400, based on the direction constraint label, perform graphic recognition on the separated graphic image list and the separated graphic distribution coordinate list to obtain graphic features. Specifically, use a network and method similar to character recognition to perform graphic recognition and feature extraction on the separated graphic image list and the separated graphic distribution coordinate list.

[0039] Step S500, configure the vector image according to the text information, the dimensioning and the graphic features to generate an editable digital drawing. Specifically, according to the extracted text information, dimensioning and graphic features, use a vector image configuration tool (such as AutoCAD, Illustrator, etc.) to generate an editable digital drawing according to each graphic type and vector parameters (such as coordinates, dimensions, etc.). Among them, vector image configuration refers to converting the extracted graphics and text information into a vector format for editing and modification, and the editable digital drawing is a drawing stored in a vector format that can be edited and modified on a computer.

[0040] Step S600, receiving drawing conversion constraint information, performing editing optimization on the editable digital drawing, generating a target conversion drawing, sending it to the user end, and receiving user feedback information. Specifically, receiving drawing conversion constraint information input by the user, including graphic size requirement parameters and graphic layout parameters. Editing the editable digital drawing according to these parameters, generating a target conversion drawing, sending the target conversion drawing to the user end, and receiving user feedback information.

[0041] In a possible implementation, drawing conversion constraint information is received, editing optimization is performed on the editable digital drawing, and a target conversion drawing is generated. Step S600 further includes step S610, wherein the drawing conversion constraint information includes a graphic size requirement parameter and a graphic layout parameter, and the editable digital drawing is edited according to the graphic size requirement parameter and the graphic layout parameter to obtain a plurality of initial drawing schemes. Specifically, the drawing conversion constraint information is a specific requirement put forward by a user or a system for the drawing conversion process, including but not limited to a graphic size requirement parameter and a graphic layout parameter. Among them, the graphic size requirement parameter is that some graphics in the drawing or the entire drawing that the user wants to meet a specific size requirement, such as length, width, height, etc. The graphic layout parameter is that the graphics in the drawing that the user wants to arrange in a specific manner, such as alignment, spacing, hierarchy, etc. According to the received graphic size requirement parameter and graphic layout parameter, the editable digital drawing is edited, including adjusting the size, position, rotation angle, etc. of the graphics. After editing, a plurality of initial drawing schemes are generated, each of which meets the user's constraint conditions but differs in details. Step S620, traverse the several initial drawing schemes to perform graphic volume sum analysis to obtain several graphic volume sum total values. Specifically, add up the space or area occupied by all graphics in each initial drawing scheme to measure the complexity and potential manufacturing cost of the drawing. Step S630, extract the minimum value of the several graphic volume sum total values ​​and set it as the target conversion drawing. Specifically, after calculating the graphic volume sum total values ​​of all initial drawing schemes, extract the scheme corresponding to the minimum value as the target conversion drawing, and the drawing scheme corresponding to this minimum value is considered to be the scheme with the lowest cost and the least complexity. This implementation method quantitatively evaluates the complexity and cost of different drawing schemes by calculating the graphic volume sum total value, and finds a drawing conversion scheme with the lowest cost and the least complexity under the premise of meeting the user's constraints.

[0042] Step S700, when the user feedback information includes an instruction to allow conversion, the target conversion drawing is controlled by printing. Specifically, when the user feedback information includes an instruction to allow conversion, the target conversion drawing is controlled by printing, and it is output to the printer for printing. If the user feedback does not allow conversion or needs to be modified, it returns to further edit and optimize. The embodiment of the present application obtains a digital image of the drawing to be converted through an image scanning device, separates the image using an image segmentation network, obtains information such as character image, character distribution coordinates, graphic image and graphic distribution coordinates, and carries a direction constraint label, performs character recognition and graphic recognition based on this information, obtains text information, dimensioning and graphic features, and then configures the vector image according to this information to generate an editable digital drawing, and finally, receives the drawing conversion constraint information, edits and optimizes the editable digital drawing, generates the target conversion drawing, and sends it to the user end for feedback, and when the user allows conversion, executes technical means such as printing control, thereby achieving the technical effect of improving conversion efficiency and ensuring conversion accuracy.

[0043] In the above, refer to Figure 1 A method for converting and generating drawings by parsing drawings according to an embodiment of the present invention is described in detail. Figure 2 A platform for converting and generating drawings by parsing drawings according to an embodiment of the present invention is described.

[0044] A drawing conversion generation platform for drawing analysis according to an embodiment of the present invention is used to solve the technical problems of low efficiency and insufficient precision in the existing drawing digital conversion, so as to achieve the technical effect of improving conversion efficiency and ensuring conversion precision. A drawing conversion generation platform for drawing analysis includes: a drawing scanning module 10 to be converted, a digital image separation module 20, a character recognition module 30, a graphic recognition module 40, a vector image configuration module 50, an editing optimization module 60, and a printing control module 70.

[0045] The drawing scanning module 10 to be converted is used to place the drawing to be converted on a preset platform, scan the preset platform through an image scanning device, and obtain a digital image of the drawing to be converted; the digital image separation module 20 is used to separate the digital image of the drawing to be converted through an image segmentation network, and obtain a separated character image list, a separated character distribution coordinate list, a separated graphic image list, and a separated graphic distribution coordinate list, wherein the separated character image and the separated graphic image have a direction constraint label; the character recognition module 30 is used to perform character recognition on the separated character image list and the separated character distribution coordinate list based on the direction constraint label, and obtain text information and size. The graphic recognition module 40 is used to perform graphic recognition on the separated graphic image list and the separated graphic distribution coordinate list based on the direction constraint label to obtain graphic features; the vector image configuration module 50 is used to perform vector image configuration according to the text information, the dimension annotation and the graphic features to generate an editable digital drawing; the editing optimization module 60 is used to receive the drawing conversion constraint information, perform editing optimization on the editable digital drawing, generate a target conversion drawing, send it to the user end, and receive user feedback information; the printing control module 70 is used to perform printing control on the target conversion drawing when the user feedback information includes an instruction to allow conversion.

[0046] The specific configuration of the digital image separation module 20 will be described in detail below. As described above, the digital image of the drawing to be converted is separated by the image segmentation network to obtain a separated character image list, a separated character distribution coordinate list, a separated graphic image list and a separated graphic distribution coordinate list, wherein the separated character image and the separated graphic image have a direction constraint label, and the digital image separation module 20 may further include: a direction identification unit for directionally identifying the digital image of the drawing to be converted through the user end to generate the direction constraint label; a separation unit for the image segmentation network including a character separation channel and a graphic separation channel, through which the direction constraint label and the digital image of the drawing to be converted are processed to generate the separated character image list and the separated character distribution coordinate list, and through which the direction constraint label and the digital image of the drawing to be converted are processed to generate the separated graphic image list and the separated graphic distribution coordinate list.

[0047] Among them, through the character separation channel, the direction constraint label and the digital image of the drawing to be converted are processed to generate the separated character image list and the separated character distribution coordinate list, and the separation unit may further include: a training set generation subunit is used to randomly generate a training sample image set and a character distribution coordinate identification data set through an encoder according to a preset direction based on a graphic database and a character database, wherein any training sample image has at least one graphic element and one character element; a character separation channel training subunit is used to train the character separation channel using the character distribution coordinate identification data set as supervision data and the training sample image set as input data of a convolutional neural network.

[0048] Wherein, the character distribution coordinate identification data set is used as the supervision data, the training sample image set is used as the input data of the convolutional neural network, and the character separation channel is trained. The character separation channel training subunit may further include: a character separation training loss function construction microunit for constructing a character separation training loss function: ,in, Represents any training loss, Represents the coordinates of the ith character separation mark, Represents the supervised distribution coordinates of the i-th character, Characterization and The area of ​​the non-intersecting region, Characterization and The area of ​​the union, Characterize the loss ratio threshold, Q characterizes the number of characters trained at any time, and e is a constant; the character separation channel acquisition micro-unit is used to use the character distribution coordinate identification data set as supervision data, and the training sample image set as input data of the convolutional neural network. When the output value of the character separation training loss function is less than or equal to the loss threshold for a preset number of consecutive times, the character separation channel is obtained.

[0049] The specific configuration of the character recognition module 30 will be described in detail below. As described above, based on the direction constraint label, character recognition is performed on the separated character image list and the separated character distribution coordinate list to obtain text information and dimension annotations. The character recognition module 30 may further include: a character recognition network acquisition unit for obtaining a character recognition network, wherein the character recognition network includes a feature extraction channel and a character classification channel; a character shape feature recognition unit for traversing the separated character image list through the feature extraction channel and based on the direction constraint label to perform character shape feature recognition and obtain a separated character shape feature list, wherein the separated character shape feature has a character shape feature direction label; a separated character matching semantic list acquisition unit for traversing the separated character shape feature list through the character classification channel and based on the character database and the character shape feature direction label to perform matching and obtain a separated character matching semantic list; an image position embedding unit for traversing the separated character matching semantic list according to the separated character distribution coordinate list to perform image position embedding and generate the text information and the dimension annotations.

[0050] Among them, through the feature extraction channel, based on the direction constraint label, the separated character image list is traversed to perform character shape feature recognition, and a separated character shape feature list is obtained. The character shape feature recognition unit may further include: a feature extraction channel construction data set acquisition subunit is used to acquire a character image training data set, a character direction identification data set and a character shape feature descriptor true value data set to generate a feature extraction channel construction data set; a data set division subunit is used to divide the feature extraction channel construction data set into k equal parts of feature extraction channel construction data; a feature extraction base channel construction subunit is used to perform k times of sampling with replacement on the k equal parts of feature extraction channel construction data to obtain a first feature extraction channel construction data set and construct a first feature extraction base channel, until the k equal parts of feature extraction channel construction data are sampled with replacement k times to obtain the Mth feature extraction channel construction data set and construct the Mth feature extraction base channel; a feature extraction base channel fusion subunit is used to construct a fully connected layer, fuse the first feature extraction base channel until the output mean of the Mth feature extraction base channel is obtained to generate the feature extraction channel.

[0051] The specific configuration of the editing optimization module 60 will be described in detail below. As described above, the drawing conversion constraint information is received, the editing optimization is performed on the editable digital drawing, and the target conversion drawing is generated. The editing optimization module 60 may further include: a drawing editing unit is used for the drawing conversion constraint information including the graphic size requirement parameter and the graphic layout parameter, and the editable digital drawing is edited according to the image size requirement parameter and the graphic layout parameter to obtain a plurality of initial drawing schemes; a graphic volume summing and analyzing unit is used for traversing the plurality of initial drawing schemes to perform graphic volume summing and analysis to obtain a plurality of graphic volume sum total values; a minimum value extraction unit is used for extracting the minimum value of the plurality of graphic volume sum total values, and setting it as the target conversion drawing.

[0052] A platform for drawing conversion and generation by drawing analysis provided by an embodiment of the present invention can execute a method for drawing conversion and generation by drawing analysis provided by any embodiment of the present invention, and has functional modules and beneficial effects corresponding to the execution method.

[0053] Although the present application makes various references to certain modules in the platform according to the embodiments of the present application, any number of different modules may be used and run on the user terminal and / or server, and the various units and modules included are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be implemented; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of the present invention.

[0054] The above specific implementation manner does not constitute a limitation to the protection scope of the present application. It should be understood by those skilled in the art that various modifications, combinations and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present application should be included in the protection scope of the present application. In some cases, the actions or steps recorded in the present application can be performed in an order different from that in the embodiment and can still achieve the desired results. In addition, the process depicted in the accompanying drawings does not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

Claims

1. A method for converting and generating drawings by drawing analysis, characterized in that: include: Placing the drawing to be converted on a preset platform, and scanning the preset platform with an image scanning device to obtain a digital image of the drawing to be converted; Separating the to-be-converted drawing digital image through an image segmentation network to obtain a separated character image list, a separated character distribution coordinate list, a separated graphic image list, and a separated graphic distribution coordinate list, wherein the separated character image and the separated graphic image have direction constraint labels; Based on the direction constraint label, character recognition is performed on the separated character image list and the separated character distribution coordinate list to obtain text information and dimension annotation; Based on the direction constraint label, performing graphic recognition on the separated graphic image list and the separated graphic distribution coordinate list to obtain graphic features; Performing vector image configuration according to the text information, the dimension annotation and the graphic features to generate an editable digital drawing; Receiving drawing conversion constraint information, performing editing optimization on the editable digital drawing, generating a target conversion drawing, sending the target conversion drawing to a user terminal, and receiving user feedback information; When the user feedback information includes a conversion permission instruction, executing printing control on the target conversion drawing; Based on the direction constraint label, character recognition is performed on the separated character image list and the separated character distribution coordinate list to obtain text information and dimension annotation, including: Obtaining a character recognition network, wherein the character recognition network includes a feature extraction channel and a character classification channel; Through the feature extraction channel, based on the direction constraint label, the separated character image list is traversed to perform character shape feature recognition to obtain a separated character shape feature list, wherein the separated character shape feature has a character shape feature direction label; Through the character classification channel, based on the character database and the character shape feature direction label, the separated character shape feature list is traversed for matching to obtain a separated character matching semantic list; According to the separated character distribution coordinate list, the separated character matching semantic list is traversed to perform image position embedding, and the text information and the dimension annotation are generated.

2. The method for converting and generating drawings by drawing analysis as claimed in claim 1, characterized in that: The to-be-converted drawing digital image is separated by an image segmentation network to obtain a separated character image list, a separated character distribution coordinate list, a separated graphic image list, and a separated graphic distribution coordinate list, wherein the separated character image and the separated graphic image have direction constraint labels, including: Through the user terminal, the direction of the digital image of the drawing to be converted is marked to generate the direction constraint label; The image segmentation network includes a character separation channel and a graphic separation channel; Processing the direction constraint label and the to-be-converted drawing digital image through the character separation channel to generate the separated character image list and the separated character distribution coordinate list; The direction constraint label and the to-be-converted drawing digital image are processed through the graphic separation channel to generate the separated graphic image list and the separated graphic distribution coordinate list.

3. The method for converting and generating drawings by drawing analysis as claimed in claim 2, characterized in that: The direction constraint label and the to-be-converted drawing digital image are processed through the character separation channel to generate the separated character image list and the separated character distribution coordinate list, including: By means of an encoder, a training sample image set and a character distribution coordinate identification data set are randomly generated according to a preset direction based on a graphic database and a character database, wherein any training sample image has at least one graphic element and one character element; The character distribution coordinate identification data set is used as supervision data, and the training sample image set is used as input data of a convolutional neural network to train the character separation channel.

4. The method for converting and generating drawings by drawing analysis as claimed in claim 3, characterized in that: The character distribution coordinate identification data set is used as supervision data, and the training sample image set is used as input data of a convolutional neural network to train the character separation channel, including: Construct the character separation training loss function: ， in, Represents any training loss, Represents the coordinates of the ith character separation mark, Represents the supervised distribution coordinates of the i-th character, Characterization and The area of ​​the non-intersecting region, Characterization and The union area, represents the loss ratio threshold, Q represents the number of characters in any training, and e is a constant; The character distribution coordinate identification data set is used as supervision data, and the training sample image set is used as input data of a convolutional neural network. When the output value of the character separation training loss function is less than or equal to a loss threshold for a preset number of consecutive times, the character separation channel is obtained.

5. The method for converting and generating drawings by drawing analysis as claimed in claim 1, characterized in that: Through the feature extraction channel, based on the direction constraint label, the separated character image list is traversed to perform character shape feature recognition to obtain a separated character shape feature list, including: Collect character image training data set, character direction identification data set and character shape feature descriptor true value data set, and generate feature extraction channel construction data set; Dividing the feature extraction channel construction data set into k equal portions of feature extraction channel construction data; Perform k-times sampling with replacement on the k equal portions of feature extraction channel construction data to obtain a first feature extraction channel construction data set, and construct a first feature extraction base channel; Until the k equal portions of feature extraction channel construction data are sampled k times with replacement, an Mth feature extraction channel construction data set is obtained, and an Mth feature extraction base channel is constructed; A fully connected layer is constructed, and the first feature extraction base channel is fused until the output mean of the Mth feature extraction base channel to generate the feature extraction channel.

6. The method for converting and generating drawings by drawing analysis as claimed in claim 1, characterized in that: Receiving drawing conversion constraint information, performing editing optimization on the editable digital drawing, and generating a target conversion drawing, including: The drawing conversion constraint information includes drawing size requirement parameters and drawing layout parameters; Editing the editable digital drawing according to the drawing size requirement parameters and the drawing layout parameters to obtain a plurality of initial drawing schemes; Traversing the plurality of initial drawing schemes to perform graphic volume summation analysis to obtain a plurality of graphic volume summation total values; The minimum value of the total sum of the volumes of the plurality of graphics is extracted and set as the target conversion drawing.

7. A platform for drawing analysis and drawing conversion generation, characterized in that: The platform is used to implement the method for converting and generating drawings by parsing drawings as described in any one of claims 1 to 6, and the platform includes: A drawing scanning module for conversion, which is used to place the drawing for conversion on a preset platform and scan the preset platform through an image scanning device to obtain a digital image of the drawing for conversion; A digital image separation module, the digital image separation module is used to separate the to-be-converted drawing digital image through an image segmentation network, and obtain a separated character image list, a separated character distribution coordinate list, a separated graphic image list, and a separated graphic distribution coordinate list, wherein the separated character image and the separated graphic image have direction constraint labels; A character recognition module, the character recognition module is used to perform character recognition on the separated character image list and the separated character distribution coordinate list based on the direction constraint label to obtain text information and dimension annotation; A graphic recognition module, the graphic recognition module is used to perform graphic recognition on the separated graphic image list and the separated graphic distribution coordinate list based on the direction constraint label to obtain graphic features; A vector image configuration module, the vector image configuration module is used to configure the vector image according to the text information, the dimension annotation and the graphic features to generate an editable digital drawing; An editing optimization module, the editing optimization module is used to receive the drawing conversion constraint information, perform editing optimization on the editable digital drawing, generate a target conversion drawing, send it to the user end, and receive user feedback information; A printing control module is used to execute printing control on the target conversion drawing when the user feedback information includes an instruction to allow conversion.

Images