Cabinet model generation method and system

Abstract

The present invention relates to the field of image processing technology. In order to solve the technical problem of how to enable non-professional designers to conveniently obtain a three-dimensional cabinet model that meets the design goal, a cabinet model generation method and system are provided. The method comprises the following steps: inputting a hand-drawn content map preprocessed from a hand-drawn drawing of a cabinet to be designed into a dimensioning detection model to detect the dimensioning; obtaining cabinet lines according to the detected dimensioning and the hand-drawn content map; identifying the dimension numbers in each dimensioning; obtaining the principal coordinate range of the dimension line in each dimensioning and each line segment in the cabinet line; calculating the overlap between the principal coordinate ranges of the dimension line and the line segment, and matching the dimensioning and the line segment according to the overlap to obtain the width information and height information of the cabinet to be designed; obtaining the depth information of the cabinet to be designed; and generating the three-dimensional model of the cabinet to be designed according to the width information, height information and depth information of the cabinet to be designed.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a cabinet model generation method and system. Background Art

[0002] At present, in the design of custom furniture such as wardrobes, shoe cabinets, and TV cabinets, there are two-dimensional drawing design and three-dimensional model design. The three-dimensional model can see the cabinet effect more intuitively, but the three-dimensional model is generally designed and operated by professionals using professional design software. The use of professional design software requires a lot of time and effort to learn in advance, and the threshold for use is relatively high for ordinary users or designers who are just starting out. Therefore, how to enable non-professional designers to easily obtain a three-dimensional cabinet model that is relatively in line with the design goals is a problem that needs to be solved urgently. Summary of the invention

[0003] In order to solve the above technical problems, the present invention provides a cabinet model generation method and system, which can easily and quickly automatically generate a corresponding cabinet three-dimensional model based on a cabinet hand-drawn drawing, and can meet the size design goals without strict requirements on the relative proportions of the cabinet lines in the hand-drawn drawing.

[0004] The technical solution adopted by the present invention is as follows:

[0005] A cabinet model generation method comprises the following steps: obtaining a hand-drawn drawing of a cabinet to be designed; pre-processing the hand-drawn drawing of the cabinet to be designed to obtain a hand-drawn content drawing; inputting the hand-drawn content drawing into a dimensioning detection model to detect the dimensioning; obtaining cabinet lines according to the detected dimensioning and the hand-drawn content drawing; inputting each dimensioning into a digital recognition model to recognize the dimension number in each dimensioning; obtaining the horizontal coordinate range and the vertical coordinate range of the dimension line in each dimensioning, and selecting the larger one of the horizontal coordinate range and the vertical coordinate range as the dimension line in the corresponding dimensioning; The method comprises the following steps: obtaining the principal coordinate range of the dimension line in each dimension annotation; obtaining the abscissa range and the ordinate range of each line segment in the cabinet line, and selecting the larger one of the abscissa range and the ordinate range as the principal coordinate range of the corresponding line segment; calculating the degree of overlap between the principal coordinate range of the dimension line in each dimension annotation and the principal coordinate range of each line segment, and matching the dimension annotation and the line segment according to the degree of overlap to obtain the width information and height information of the cabinet to be designed; obtaining the depth information of the cabinet to be designed; and generating a three-dimensional model of the cabinet to be designed according to the width information, height information and depth information of the cabinet to be designed.

[0006] Furthermore, the dimensioning detection model is a YOLO (a target detection algorithm) model, and the digital recognition model is an OCR (Optical Character Recognition) model.

[0007] Furthermore, the YOLO model includes multiple adaptive convolution modules. When performing feature extraction, each of the adaptive convolution modules performs the following operations on the input:

[0008] ,

[0009] in, F out is the output of the adaptive convolution module, F in is the input of the adaptive convolution module, represents the convolution operation, N is the total number of convolution kernels of the adaptive convolution module, T j For the j The convolution weight tensor of the convolution kernels, λ j for T j The corresponding adaptive coefficient.

[0010] Furthermore, the degree of overlap between the main coordinate range of the dimension line in each dimension annotation and the main coordinate range of each line segment is calculated, specifically including: judging whether the main coordinate range of the dimension line in a dimension annotation and the main coordinate range of a line segment both belong to the horizontal coordinate range, or both belong to the vertical coordinate range; if not, the degree of overlap between the main coordinate range of the dimension line in the dimension annotation and the main coordinate range of the line segment is 0; if yes, calculating the ratio of the intersection length to the union length of the main coordinate range of the dimension line in the dimension annotation and the main coordinate range of the line segment, and obtaining the degree of overlap between the main coordinate range of the dimension line in the dimension annotation and the main coordinate range of the line segment.

[0011] The dimension annotation and the line segment are matched according to the overlap, specifically including: determining whether the overlap between the main coordinate range of the dimension line in the dimension annotation and the main coordinate range of the line segment is greater than a preset value; if not, determining that the dimension annotation does not match the line segment; if greater than the preset value, determining whether there is at least another dimension annotation in which the main coordinate range of the dimension line and the main coordinate range of the line segment overlap more than the preset value; if not, determining that the dimension annotation matches the line segment, and using the dimension number in the dimension annotation as the dimension of the line segment; if so, selecting the dimension annotation with the maximum overlap to match the line segment, and using the dimension number in the dimension annotation with the maximum overlap as the dimension of the line segment.

[0012] Furthermore, preprocessing the hand-drawn picture of the cabinet to be designed includes performing denoising and binarization processing on the hand-drawn picture of the cabinet to be designed.

[0013] A cabinet model generation system comprises: a first acquisition module, the first acquisition module is used to acquire a hand-drawn drawing of a cabinet to be designed; a preprocessing module, the preprocessing module is used to preprocess the hand-drawn drawing of the cabinet to be designed to obtain a hand-drawn content map; a detection module, the detection module is used to input the hand-drawn content map into a dimensioning detection model to detect the dimensioning; a second acquisition module, the second acquisition module is used to obtain cabinet lines according to the detected dimensioning and the hand-drawn content map; a recognition module, the recognition module is used to input each dimensioning into a digital recognition model to recognize the dimension number in each dimensioning; a third acquisition module, the third acquisition module is used to acquire the horizontal coordinate range and the vertical coordinate range of the dimension line in each dimensioning, and select the larger one of the horizontal coordinate range and the vertical coordinate range. as the principal coordinate range of the dimension line in the corresponding dimension annotation; a fourth acquisition module, the fourth acquisition module is used to obtain the horizontal coordinate range and the vertical coordinate range of each line segment in the cabinet line, and select the larger one of the horizontal coordinate range and the vertical coordinate range as the principal coordinate range of the corresponding line segment; a calculation module, the calculation module is used to calculate the overlap between the principal coordinate range of each dimension line in the dimension annotation and the principal coordinate range of each line segment, and match the dimension annotation and the line segment according to the overlap to obtain the width information and height information of the cabinet to be designed; a fifth acquisition module, the fifth acquisition module is used to obtain the depth information of the cabinet to be designed; a generation module, the generation module is used to generate a three-dimensional model of the cabinet to be designed according to the width information, height information and depth information of the cabinet to be designed.

[0014] Furthermore, the dimensioning detection model is a YOLO model, and the digital recognition model is an OCR model.

[0015] Furthermore, the YOLO model includes multiple adaptive convolution modules. When performing feature extraction, each of the adaptive convolution modules performs the following operations on the input:

[0016] ,

[0017] in, F out is the output of the adaptive convolution module, F in is the input of the adaptive convolution module, represents the convolution operation, N is the total number of convolution kernels of the adaptive convolution module, T j For the j The convolution weight tensor of the convolution kernels, λ j for T j The corresponding adaptive coefficient.

[0018] Furthermore, the calculation module is specifically used to: determine whether the main coordinate range of a dimension line in a dimension annotation and the main coordinate range of a line segment both belong to the horizontal coordinate range, or both belong to the vertical coordinate range; if not, the overlap between the main coordinate range of the dimension line in the dimension annotation and the main coordinate range of the line segment is 0; if yes, calculate the ratio of the intersection length and the union length of the main coordinate range of the dimension line in the dimension annotation and the main coordinate range of the line segment to obtain the overlap between the main coordinate range of the dimension line in the dimension annotation and the main coordinate range of the line segment; determine whether the main coordinate range of the dimension line in the dimension annotation and the main coordinate range of the line segment both belong to the horizontal coordinate range, or both belong to the vertical coordinate range; if not, the overlap between the main coordinate range of the dimension line in the dimension annotation and the main coordinate range of the line segment is 0; if yes, calculate the ratio of the intersection length and the union length of the main coordinate range of the dimension line in the dimension annotation and the main coordinate range of the line segment to obtain the overlap between the main coordinate range of the dimension line in the dimension annotation and the main coordinate range of the line segment Whether the degree of overlap between the main coordinate ranges of the segments is greater than a preset value; if not greater than the preset value, it is determined that the dimension annotation does not match the line segment; if greater than the preset value, it is determined whether there is at least one other dimension line in which the degree of overlap between the main coordinate range of the dimension line and the main coordinate range of the line segment is greater than the preset value; if not, it is determined that the dimension annotation matches the line segment, and the dimension number in the dimension annotation is used as the dimension of the line segment; if so, the dimension annotation with the largest degree of overlap is selected to match the line segment, and the dimension number in the dimension annotation with the largest degree of overlap is used as the dimension of the line segment.

[0019] Furthermore, the preprocessing module performs denoising and binarization processing on the hand-drawn drawing of the cabinet to be designed.

[0020] Beneficial effects of the present invention:

[0021] The present invention detects the dimension annotations in the hand-drawn drawing of the cabinet, obtains the cabinet lines, identifies the dimension numbers in the dimension annotations, and then matches the dimension annotations with the line segments according to the coordinate ranges of the dimension lines in the dimension annotations and the line segments in the cabinet lines, thereby obtaining the width and height of the cabinet, and finally generating a three-dimensional model of the cabinet in combination with the width, height and depth of the cabinet. Therefore, the corresponding three-dimensional model of the cabinet can be automatically generated according to the hand-drawn drawing of the cabinet conveniently and quickly, and the relative proportions of the cabinet lines in the hand-drawn drawing do not need to be strictly required, so as to meet the size design goals. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 A flowchart of a cabinet model generating method according to an embodiment of the present invention;

[0023] Figure 2 A hand-drawn drawing of a cabinet to be designed according to an embodiment of the present invention;

[0024] Figure 3 A three-dimensional model diagram of a cabinet to be designed according to an embodiment of the present invention;

[0025] Figure 4 It is a block diagram of a cabinet model generating system according to an embodiment of the present invention. DETAILED DESCRIPTION

[0026] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0027] like Figure 1 As shown, the cabinet model generation method of the embodiment of the present invention includes the following steps:

[0028] S1, obtaining a hand-drawn drawing of the cabinet to be designed.

[0029] In one embodiment of the present invention, Figure 2As shown, the hand-drawn drawing of the cabinet to be designed can be drawn on paper, including the cabinet lines and necessary dimension annotations of the cabinet to be designed. The cabinet lines are the cabinet elevation drawings without door panels, that is, the two-dimensional plan drawings under the main perspective, and each line in the cabinet lines represents the corresponding side of the plate. The dimension annotations include dimension lines with arrows and dimension numbers. If the dimension annotations are outside the overall outline of the cabinet lines, the dimension annotations may also include dimension limits. The dimension annotations are divided into horizontal width annotations and vertical height annotations. For the horizontal width annotations, the dimension numbers are on the upper side of the dimension lines; for the vertical height annotations, the dimension numbers are on the right side of the dimension lines. The above-mentioned cabinet lines contain the shape information of the cabinet to be designed in the width and height directions, and the dimension annotations contain the dimension information of the cabinet to be designed in the width and height directions.

[0030] It should be noted that the relative proportions of the various lines of the cabinet in the embodiment of the present invention do not need to be made according to the actual size, and it is only necessary to write the size numbers correctly.

[0031] S2, preprocessing the hand-drawn drawing of the cabinet to be designed to obtain a hand-drawn content drawing.

[0032] For the hand-drawn drawings of the cabinets to be designed drawn on paper, they can be photographed by a camera and input into a computer. In the computer, OpenCV (a cross-platform computer vision and machine learning software library) can be used to perform denoising and binarization on the hand-drawn drawings of the cabinets to be designed. The denoising process can remove stains on the paper or excess ink in the hand-drawn elements, and the binarization process further highlights the hand-drawn elements such as the cabinet lines and dimension markings in view of the obvious grayscale difference between the hand-drawn elements and the paper background. The image after the above pre-processing is called a hand-drawn content image in the embodiment of the present invention.

[0033] S3, inputting the hand-drawn content image into the dimension annotation detection model to detect the dimension annotation.

[0034] In one embodiment of the present invention, the dimension annotation detection model may be a YOLO model, and the YOLO network may be trained in advance with a large number of sample images containing hand-drawn dimension annotations to obtain a dimension annotation detection model. The dimension annotation detection model has the ability to detect dimension annotations in an input image.

[0035] In a further embodiment of the present invention, the YOLO model may include multiple adaptive convolution modules, that is, the existing YOLO model is improved, and the conventional convolution module is replaced by the adaptive convolution module. Preferably, all conventional convolution modules in the existing YOLO model can be replaced with adaptive convolution modules. When performing feature extraction, each adaptive convolution module performs the following operations on the input:

[0036] ,

[0037] in, F out is the output of the adaptive convolution module, F in is the input of the adaptive convolution module, represents the convolution operation, N is the total number of convolution kernels in the adaptive convolution module, T j For the j The convolution weight tensor of the convolution kernels, λ j for T j The corresponding adaptive coefficient.

[0038] The output of each adaptive convolution module can be called a feature map. F out The size is C out × H × W The feature map of C out is the number of output channels, H and W are the height and width of the feature map respectively. The convolution weight tensor of the convolution kernel is a four-dimensional tensor with size C in × C out × H × W ,in, C in is the number of input channels. The convolution weight tensor of the convolution kernel can be obtained by model training optimization. Before model training, the convolution weight tensor of the convolution kernel is an initialized parameter. In each adaptive convolution module, the operation result of each convolution kernel is affected by the operation results of all convolution kernels. Therefore, before the adaptive convolution module performs convolution operation, the embodiment of the present invention can first perform global average pooling operation on the input, then perform multi-layer perceptron operation on the pooling operation result, and then perform normalized exponential function operation on the multi-layer perceptron operation result to automatically generate λ 1 to λ N The vector λ is:

[0039] ,

[0040] in, P (·) represents the global average pooling operation, M (·) represents the multi-layer perceptron operation, that is, the output after inputting the multi-layer perceptron. S (·) represents the normalized exponential function operation.

[0041] It should be understood that in a hand-drawn drawing, the lengths of the dimension lines of each dimension annotation can vary greatly, while the size of the dimension numbers is approximately the same. Therefore, the target to be detected by the dimension annotation detection model has obvious differences in graphic features from the target in conventional target detection. The embodiment of the present invention designs the above-mentioned adaptive convolution module, and the convolution weights can be adaptively adjusted according to the input. Therefore, the YOLO model can optimize the feature map extraction according to the characteristics of the image itself, and can also have a more accurate detection effect when facing special targets such as dimension annotations.

[0042] S4, obtaining the cabinet lines according to the detected dimension markings and the hand-drawn content drawing.

[0043] It should be understood that the hand-drawn elements in the hand-drawn content map are the cabinet lines except for the dimension markings. Therefore, in one embodiment of the present invention, the cabinet lines can be obtained by setting the grayscale of the dimension markings to be the same as the paper background, for example, to 255, and then extracting pixels with a grayscale value of 0. Thus, the cabinet lines can be obtained by a simple grayscale setting, which can greatly reduce the amount of calculation compared to the related art of performing target detection on each target separately.

[0044] S5, inputting each dimension annotation into a digital recognition model to recognize the dimension number in each dimension annotation.

[0045] In one embodiment of the present invention, the digital recognition model may be an OCR model. By inputting the dimension annotations detected in step S3 into the OCR model respectively, the dimension numbers in the dimension annotations can be obtained.

[0046] S6, obtaining the horizontal coordinate range and the vertical coordinate range of the dimension line in each dimension annotation, and selecting the larger one of the horizontal coordinate range and the vertical coordinate range as the main coordinate range of the dimension line in the corresponding dimension annotation.

[0047] S7, obtaining the horizontal coordinate range and the vertical coordinate range of each line segment in the cabinet line, and selecting the larger one of the horizontal coordinate range and the vertical coordinate range as the main coordinate range of the corresponding line segment.

[0048] In the embodiment of the present invention, the line segments in the cabinet lines can have corner points as endpoints, and the corner points refer to the intersections of 2, 3, 4 or more lines, so that the cabinet lines can be divided into multiple line segments. It should be noted that in the embodiment of the present invention, every two endpoints and the lines between them can constitute a line segment, that is, overlap between line segments is allowed.

[0049] The coordinate system in the embodiment of the present invention is a pixel coordinate system, which can take the upper left corner of the hand-drawn content image as the coordinate origin, the horizontal direction as the horizontal coordinate axis, the rightward direction of the origin as the positive horizontal coordinate, the vertical direction as the vertical coordinate axis, the downward direction of the origin as the positive vertical coordinate, and the coordinate value is in pixels. After determining the above coordinate system, the coordinate values ​​of each pixel of the dimension line in each dimension annotation and the coordinate values ​​of each pixel of each line segment in the cabinet line can be obtained. Furthermore, the horizontal coordinate range and vertical coordinate range of each dimension line and the horizontal coordinate range and vertical coordinate range of each line segment can be obtained.

[0050] If the selected principal coordinate range is the horizontal coordinate range, the dimension line or line segment is actually parallel to the horizontal coordinate; if the selected principal coordinate range is the vertical coordinate range, the dimension line or line segment is actually parallel to the vertical coordinate. The reason why the non-principal coordinate range is also a range is caused by the width of the line or the error tilt of the hand-drawn line. By selecting the principal coordinate range, it is possible to avoid the subsequent calculation of the width or error tilt of the line as a line, which can not only reduce the amount of calculation, but also avoid the error of matching the dimension annotation with the line segment.

[0051] In addition, in special circumstances that may occur in theory, if the horizontal coordinate range and the vertical coordinate range are exactly equal, either coordinate range is used as the main coordinate range.

[0052] S8, calculating the overlap between the principal coordinate range of the dimension line in each dimension annotation and the principal coordinate range of each line segment, and matching the dimension annotation and the line segment according to the overlap to obtain the width information and height information of the cabinet to be designed.

[0053] The calculation process of the coincidence degree is as follows: determine whether the main coordinate range of the dimension line in a dimension annotation and the main coordinate range of a line segment both belong to the horizontal coordinate range, or both belong to the vertical coordinate range; if not, that is, one of them is the horizontal coordinate range and the other is the vertical coordinate range, then the coincidence degree between the main coordinate range of the dimension line in the dimension annotation and the main coordinate range of the line segment is 0; if yes, calculate the ratio of the intersection length and the union length of the main coordinate range of the dimension line in the dimension annotation and the main coordinate range of the line segment to obtain the coincidence degree between the main coordinate range of the dimension line in the dimension annotation and the main coordinate range of the line segment. For all dimension annotations and all line segments, they can be combined in pairs, and the coincidence degree between the main coordinate range of the dimension line in the dimension annotation and the main coordinate range of the line segment can be calculated according to the above process. For the ratio of the intersection length to the union length of two principal coordinate ranges, for example, if the principal coordinate range of the dimension line in a dimension annotation is [1260, 3380], and the principal coordinate range of a line segment is [1274, 3382], then the intersection length of the two is 3380-1274=2106, and the union length of the two is 3382-1260=2122, and the overlap between the two is 2106 / 2122*100%=99.25%.

[0054] The matching process of dimensioning and line segment is as follows: determine whether the overlap between the main coordinate range of the dimension line in the dimensioning and the main coordinate range of the line segment is greater than a preset value; if not greater than the preset value, determine that the dimensioning does not match the line segment; if greater than the preset value, determine whether there is at least one other dimensioning whose main coordinate range overlaps with the main coordinate range of the line segment greater than the preset value; if not, determine that the dimensioning matches the line segment, and use the dimension number in the dimensioning as the size of the line segment; if there is, that is, there is another or more dimensioning whose main coordinate range overlaps with the main coordinate range of the line segment greater than the preset value, then compare these overlaps greater than the preset value, and then select the dimensioning with the maximum overlap to match the line segment, and use the dimension number in the dimensioning with the maximum overlap as the size of the line segment. Among them, the preset value compared by the overlap can be set according to the degree of dimensional error during hand-drawing and the actual difference between the dimensionings in the designed cabinet. In a specific embodiment of the present invention, the preset value can be 98%.

[0055] Thus, the sizes of multiple line segments in the cabinet lines can be obtained. If the dimensions in the hand-drawn drawing of the cabinet to be designed are fully marked, the multiple line segments here will contain all the key and necessary width and height information of the cabinet to be designed.

[0056] S9, obtaining depth information of the cabinet to be designed.

[0057] S10, generating a three-dimensional model of the cabinet to be designed according to the width information, height information and depth information of the cabinet to be designed.

[0058] In step S8, the height and width of the cabinet to be designed are obtained. In step S9, the depth of the cabinet to be designed can be input, or the preset default depth can be retrieved. Thus, the three-dimensional model of the cabinet to be designed can be generated by combining the dimensions in the three-dimensional direction. Figure 2 The 3D model generated by the hand drawing of the cabinet to be designed is as follows Figure 3 shown.

[0059] In addition, it should be noted that since the cabinet lines in the hand-drawn drawing of the cabinet to be designed are the cabinet elevation drawings without door panels, the cabinet lines all correspond to the side of the board. In the 3D model, the board extends in the depth direction, and a back panel can be automatically generated at the deepest part ( Figure 3 is an embodiment in which the back panel is automatically generated. In other embodiments, the cabinet panel may not be automatically generated, but may be manually added by the user later). If the three-dimensional model of the cabinet that the user wants also needs to add door panels, drawers or hardware, etc., they can be automatically added according to the user's instructions, or manually added and adjusted by the user based on the three-dimensional model generated above. As for the thickness of the board itself, in the outer contour of the cabinet, the outer wall of the board can be used as the dimension boundary; in the cabinet, it can be deducted by the user when drawing the hand-drawn drawing of the cabinet to be designed. For example, when drawing multiple layers, the dimension between two adjacent layers can be marked with the distance between the lower surface of the upper plate and the upper surface of the lower plate, that is, the distance of the space, and the size of the side panel on one side of the multiple layers is the sum of the distances of the above-mentioned spaces plus the thickness of each layer here. For those that the user does not deduct by himself, the computer can also make adaptive fine adjustments. For example, the dimension marking of the side panel can be used as a reference, and the dimension marked between the two layers minus the thickness of the board can be used as the actual dimension between the two layers in the three-dimensional model. The addition of the above-mentioned door panels, etc., and the adjustment of the dimensions considering the thickness of the panels, etc., can be easily implemented by those skilled in the art using existing technologies or implementation experience in the field according to actual conditions, and the embodiments of the present invention will not list other possible situations and corresponding means one by one.

[0060] According to the cabinet model generation method of the embodiment of the present invention, the dimension annotations in the cabinet hand-drawn drawing are detected, and the cabinet lines are obtained, the dimension numbers in the dimension annotations are identified, and then the dimension annotations and the line segments are matched according to the coordinate ranges of the dimension lines in the dimension annotations and the line segments in the cabinet lines, so as to obtain the width and height of the cabinet, and finally the three-dimensional model of the cabinet is generated in combination with the width, height and depth of the cabinet. Therefore, the corresponding three-dimensional model of the cabinet can be automatically generated according to the cabinet hand-drawn drawing conveniently and quickly, and the relative proportions of the cabinet lines in the hand-drawn drawing do not need to be strictly required, so as to meet the size design goals.

[0061] Corresponding to the cabinet model generating method of the above embodiment, the present invention further provides a cabinet model generating system.

[0062] like Figure 4 As shown, the cabinet model generation system of the embodiment of the present invention includes a first acquisition module 1, a preprocessing module 2, a detection module 3, a second acquisition module 4, a recognition module 5, a third acquisition module 6, a fourth acquisition module 7, a calculation module 8, a fifth acquisition module 9 and a generation module 10. Among them, the first acquisition module 1 is used to obtain a hand-drawn drawing of the cabinet to be designed; the preprocessing module 2 is used to preprocess the hand-drawn drawing of the cabinet to be designed to obtain a hand-drawn content map; the detection module 3 is used to input the hand-drawn content map into a dimensioning detection model to detect the dimensioning; the second acquisition module 4 is used to obtain the cabinet lines according to the detected dimensioning and the hand-drawn content map; the recognition module 5 is used to input each dimensioning into a digital recognition model to recognize the dimension number in each dimensioning; the third acquisition module 6 is used to obtain the horizontal coordinate range and the vertical coordinate range of the dimension line in each dimensioning, and select the larger one of the horizontal coordinate range and the vertical coordinate range as the corresponding dimensioning. The fourth acquisition module 7 is used to obtain the abscissa range and ordinate range of each line segment in the cabinet line, and select the larger one of the abscissa range and the ordinate range as the principal coordinate range of the corresponding line segment; the calculation module 8 is used to calculate the overlap between the principal coordinate range of the dimension line in each dimension annotation and the principal coordinate range of each line segment, and match the dimension annotation and the line segment according to the overlap to obtain the width information and height information of the cabinet to be designed; the fifth acquisition module 9 is used to obtain the depth information of the cabinet to be designed; the generation module 10 is used to generate a three-dimensional model of the cabinet to be designed according to the width information, height information and depth information of the cabinet to be designed.

[0063] In one embodiment of the present invention, Figure 2 As shown, the hand-drawn drawing of the cabinet to be designed can be drawn on paper, including the cabinet lines and necessary dimension annotations of the cabinet to be designed. The cabinet lines are the cabinet elevation drawings without door panels, that is, the two-dimensional plan drawings under the main perspective, and each line in the cabinet lines represents the corresponding side of the plate. The dimension annotations include dimension lines with arrows and dimension numbers. If the dimension annotations are outside the overall outline of the cabinet lines, the dimension annotations may also include dimension limits. The dimension annotations are divided into horizontal width annotations and vertical height annotations. For the horizontal width annotations, the dimension numbers are on the upper side of the dimension lines; for the vertical height annotations, the dimension numbers are on the right side of the dimension lines. The above-mentioned cabinet lines contain the shape information of the cabinet to be designed in the width and height directions, and the dimension annotations contain the dimension information of the cabinet to be designed in the width and height directions.

[0064] It should be noted that the relative proportions of the various lines of the cabinet in the embodiment of the present invention do not need to be made according to the actual size, and it is only necessary to write the size numbers correctly.

[0065] The first acquisition module 1 may include a camera, which can be used to capture the hand-drawn drawing of the cabinet to be designed drawn on paper and input it into the preprocessing module 2. In the preprocessing module 2, OpenCV can be used to first perform denoising and binarization on the hand-drawn drawing of the cabinet to be designed. The denoising process can remove stains on the paper or excess ink in the hand-drawn elements, and the binarization process further highlights the hand-drawn elements such as cabinet lines and dimension markings in view of the obvious grayscale difference between the hand-drawn elements and the paper background. The image after the above preprocessing is called a hand-drawn content image in the embodiment of the present invention.

[0066] In one embodiment of the present invention, the dimension annotation detection model may be a YOLO model, and the YOLO network may be trained in advance with a large number of sample images containing hand-drawn dimension annotations to obtain a dimension annotation detection model. The dimension annotation detection model has the ability to detect dimension annotations in an input image.

[0067] In a further embodiment of the present invention, the YOLO model may include multiple adaptive convolution modules, that is, the existing YOLO model is improved, and the conventional convolution module is replaced by the adaptive convolution module. Preferably, all conventional convolution modules in the existing YOLO model can be replaced with adaptive convolution modules. When performing feature extraction, each adaptive convolution module performs the following operations on the input:

[0068] ,

[0069] in, F out is the output of the adaptive convolution module, F in is the input of the adaptive convolution module, represents the convolution operation, N is the total number of convolution kernels in the adaptive convolution module, T j For the j The convolution weight tensor of the convolution kernels, λ j for T j The corresponding adaptive coefficient.

[0070] The output of each adaptive convolution module can be called a feature map. F out The size is C out × H × W The feature map ofC out is the number of output channels, H and W are the height and width of the feature map respectively. The convolution weight tensor of the convolution kernel is a four-dimensional tensor with size C in × C out × H × W ,in, C in is the number of input channels. The convolution weight tensor of the convolution kernel can be obtained by model training optimization. Before model training, the convolution weight tensor of the convolution kernel is an initialized parameter. In each adaptive convolution module, the operation result of each convolution kernel is affected by the operation results of all convolution kernels. Therefore, before the adaptive convolution module performs convolution operation, the embodiment of the present invention can first perform global average pooling operation on the input, then perform multi-layer perceptron operation on the pooling operation result, and then perform normalized exponential function operation on the multi-layer perceptron operation result to automatically generate λ 1 to λ N The vector λ is:

[0071] ,

[0072] in, P (·) represents the global average pooling operation, M (·) represents the multi-layer perceptron operation, that is, the output after inputting the multi-layer perceptron. S (·) represents the normalized exponential function operation.

[0073] It should be understood that in a hand-drawn drawing, the lengths of the dimension lines of each dimension annotation can vary greatly, while the size of the dimension numbers is approximately the same. Therefore, the target to be detected by the dimension annotation detection model has obvious differences in graphic features from the target in conventional target detection. The embodiment of the present invention designs the above-mentioned adaptive convolution module, and the convolution weights can be adaptively adjusted according to the input. Therefore, the YOLO model can optimize the feature map extraction according to the characteristics of the image itself, and can also have a more accurate detection effect when facing special targets such as dimension annotations.

[0074] It should be understood that the hand-drawn elements in the hand-drawn content map are the cabinet lines except for the dimension markings. Therefore, in one embodiment of the present invention, the cabinet lines can be obtained by setting the grayscale of the dimension markings to be the same as the paper background, for example, to 255, and then extracting pixels with a grayscale value of 0. Thus, the cabinet lines can be obtained by a simple grayscale setting, which can greatly reduce the amount of calculation compared to the related art of performing target detection on each target separately.

[0075] In one embodiment of the present invention, the digital recognition model may be an OCR model. By inputting the dimension annotations detected by the detection module 3 into the OCR model respectively, the dimension numbers in the dimension annotations can be obtained.

[0076] In the embodiment of the present invention, the line segments in the cabinet lines can have corner points as endpoints, and the corner points refer to the intersections of 2, 3, 4 or more lines, so that the cabinet lines can be divided into multiple line segments. It should be noted that in the embodiment of the present invention, every two endpoints and the lines between them can constitute a line segment, that is, overlap between line segments is allowed.

[0077] The coordinate system in the embodiment of the present invention is a pixel coordinate system, which can take the upper left corner of the hand-drawn content image as the coordinate origin, the horizontal direction as the horizontal coordinate axis, the rightward direction of the origin as the positive horizontal coordinate, the vertical direction as the vertical coordinate axis, the downward direction of the origin as the positive vertical coordinate, and the coordinate value is in pixels. After determining the above coordinate system, the coordinate values ​​of each pixel of the dimension line in each dimension annotation and the coordinate values ​​of each pixel of each line segment in the cabinet line can be obtained. Furthermore, the horizontal coordinate range and vertical coordinate range of each dimension line and the horizontal coordinate range and vertical coordinate range of each line segment can be obtained.

[0078] If the selected principal coordinate range is the horizontal coordinate range, the dimension line or line segment is actually parallel to the horizontal coordinate; if the selected principal coordinate range is the vertical coordinate range, the dimension line or line segment is actually parallel to the vertical coordinate. The reason why the non-principal coordinate range is also a range is caused by the width of the line or the error tilt of the hand-drawn line. By selecting the principal coordinate range, it is possible to avoid the subsequent calculation of the width or error tilt of the line as a line, which can not only reduce the amount of calculation, but also avoid the error of matching the dimension annotation with the line segment.

[0079] In addition, in special circumstances that may occur in theory, if the horizontal coordinate range and the vertical coordinate range are exactly equal, either coordinate range is used as the main coordinate range.

[0080] In one embodiment of the present invention, the calculation module 8 is specifically used to: determine whether the main coordinate range of a dimension line in a dimension annotation and the main coordinate range of a line segment both belong to the horizontal coordinate range, or both belong to the vertical coordinate range; if not, the overlap between the main coordinate range of the dimension line in the dimension annotation and the main coordinate range of the line segment is 0; if so, calculate the ratio of the intersection length and the union length of the main coordinate range of the dimension line in the dimension annotation and the main coordinate range of the line segment to obtain the overlap between the main coordinate range of the dimension line in the dimension annotation and the main coordinate range of the line segment; determine the main coordinate range of the dimension line in the dimension annotation Whether the degree of overlap between the dimension annotation range and the main coordinate range of the line segment is greater than a preset value; if not, the dimension annotation is determined to be mismatched with the line segment; if greater than the preset value, whether there is at least one other dimension annotation whose main coordinate range and the main coordinate range of the line segment overlap greater than the preset value; if not, the dimension annotation is determined to match the line segment, and the dimension number in the dimension annotation is used as the dimension of the line segment; if so, the dimension annotation with the largest degree of overlap is selected to match the line segment, and the dimension number in the dimension annotation with the largest degree of overlap is used as the dimension of the line segment.

[0081] Thus, the sizes of multiple line segments in the cabinet lines can be obtained. If the dimensions in the hand-drawn drawing of the cabinet to be designed are fully marked, the multiple line segments here will contain all the key and necessary width and height information of the cabinet to be designed.

[0082] The calculation module 8 obtains the dimensions of the height and width of the cabinet to be designed, and the fifth acquisition module 9 can obtain the input or preset default depth dimension of the cabinet to be designed. Therefore, the generation module 10 can generate a three-dimensional model of the cabinet to be designed in combination with the dimensions in the three-dimensional direction. Figure 2 The 3D model generated by the hand drawing of the cabinet to be designed is as follows Figure 3 shown.

[0083] According to the cabinet model generation system of the embodiment of the present invention, the dimension annotations in the cabinet hand-drawn drawing are detected, and the cabinet lines and the dimension numbers in the dimension annotations are obtained. Then, the dimension annotations and the line segments are matched according to the coordinate ranges of the dimension lines in the dimension annotations and the line segments in the cabinet lines, so as to obtain the width and height of the cabinet. Finally, a three-dimensional model of the cabinet is generated in combination with the width, height and depth of the cabinet. Therefore, the corresponding three-dimensional model of the cabinet can be automatically generated according to the cabinet hand-drawn drawing conveniently and quickly, and the relative proportions of the cabinet lines in the hand-drawn drawing do not need to be strictly required, so as to meet the size design goals.

[0084] In the description of the present invention, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. "Multiple" means two or more, unless otherwise clearly and specifically defined.

[0085] In the present invention, unless otherwise clearly specified and limited, the terms "installed", "connected", "connected", "fixed" and the like should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, it can be the internal connection of two elements or the interaction relationship between two elements. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0086] In the present invention, unless otherwise clearly specified and limited, a first feature being "above" or "below" a second feature may mean that the first and second features are in direct contact, or the first and second features are in indirect contact through an intermediate medium. Moreover, a first feature being "above", "above" or "above" a second feature may mean that the first feature is directly above or obliquely above the second feature, or simply means that the first feature is higher in level than the second feature. A first feature being "below", "below" or "below" a second feature may mean that the first feature is directly below or obliquely below the second feature, or simply means that the first feature is lower in level than the second feature.

[0087] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, unless they are contradictory.

[0088] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code that includes one or more executable instructions for implementing the steps of a specific logical function or process, and the scope of the preferred embodiments of the present invention includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by those skilled in the art to which the embodiments of the present invention belong.

[0089] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples (non-exhaustive list) of computer-readable media include the following: an electrical connection with one or more wires (electronic device), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or processing in other suitable ways if necessary, and then stored in a computer memory.

[0090] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0091] A person skilled in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.

[0092] In addition, each functional unit in each embodiment of the present invention may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0093] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present invention. A person skilled in the art may change, modify, replace and vary the above embodiments within the scope of the present invention.

Claims

1. A cabinet model generation method, characterized in that: The following steps are involved: Obtain hand-drawn drawings of the cabinet to be designed; Preprocessing the hand-drawn drawing of the cabinet to be designed to obtain a hand-drawn content drawing; Inputting the hand-drawn content image into a dimensioning detection model to detect the dimensioning; Obtaining cabinet lines according to the detected dimension markings and the hand-drawn content diagram; Inputting each dimension annotation into a digital recognition model to recognize the dimension number in each dimension annotation; Obtaining the abscissa range and the ordinate range of the dimension line in each dimension annotation, and selecting the larger one of the abscissa range and the ordinate range as the principal coordinate range of the dimension line in the corresponding dimension annotation; Obtaining the horizontal coordinate range and the vertical coordinate range of each line segment in the cabinet line, and selecting the larger one of the horizontal coordinate range and the vertical coordinate range as the main coordinate range of the corresponding line segment; Calculating the degree of overlap between the principal coordinate range of the dimension line in each dimension annotation and the principal coordinate range of each line segment, and matching the dimension annotation and the line segment according to the degree of overlap to obtain the width information and height information of the cabinet to be designed; Obtaining depth information of the cabinet to be designed; Generate a three-dimensional model of the cabinet to be designed according to the width information, height information and depth information of the cabinet to be designed, Calculating the degree of overlap between the main coordinate range of the dimension line in each dimension annotation and the main coordinate range of each line segment, specifically comprising: determining whether the main coordinate range of the dimension line in a dimension annotation and the main coordinate range of a line segment both belong to the horizontal coordinate range, or both belong to the vertical coordinate range; if not, the degree of overlap between the main coordinate range of the dimension line in the dimension annotation and the main coordinate range of the line segment is 0; if yes, calculating the ratio of the intersection length to the union length of the main coordinate range of the dimension line in the dimension annotation and the main coordinate range of the line segment, and obtaining the degree of overlap between the main coordinate range of the dimension line in the dimension annotation and the main coordinate range of the line segment, The dimension annotation and the line segment are matched according to the overlap, specifically including: determining whether the overlap between the main coordinate range of the dimension line in the dimension annotation and the main coordinate range of the line segment is greater than a preset value; if not, determining that the dimension annotation does not match the line segment; if greater than the preset value, determining whether there is at least another dimension annotation in which the main coordinate range of the dimension line and the main coordinate range of the line segment overlap more than the preset value; if not, determining that the dimension annotation matches the line segment, and using the dimension number in the dimension annotation as the dimension of the line segment; if so, selecting the dimension annotation with the maximum overlap to match the line segment, and using the dimension number in the dimension annotation with the maximum overlap as the dimension of the line segment.

2. The cabinet model generation method according to claim 1, characterized in that: The dimensioning detection model is a YOLO model, and the number recognition model is an OCR model.

3. The cabinet model generation method according to claim 2, characterized in that: The YOLO model includes multiple adaptive convolution modules. When performing feature extraction, each of the adaptive convolution modules performs the following operations on the input: ， in, F out is the output of the adaptive convolution module, F in is the input of the adaptive convolution module, represents the convolution operation, N is the total number of convolution kernels of the adaptive convolution module, T j For the j The convolution weight tensor of the convolution kernels, λ j for T j The corresponding adaptive coefficient is Among them, λ 1 to λ N The vector λ is: ， in, P (·) represents the global average pooling operation, M (·) represents the multi-layer perceptron operation, S (·) represents the normalized exponential function operation.

4. The cabinet model generation method according to claim 1, characterized in that: The preprocessing of the hand-drawn picture of the cabinet to be designed includes performing denoising and binarization processing on the hand-drawn picture of the cabinet to be designed.

5. A cabinet model generation system, characterized in that: include: A first acquisition module, the first acquisition module is used to acquire a hand-drawn drawing of the cabinet to be designed; A preprocessing module, the preprocessing module is used to preprocess the hand-drawn drawing of the cabinet to be designed to obtain a hand-drawn content drawing; A detection module, the detection module is used to input the hand-drawn content image into a dimension annotation detection model to detect the dimension annotation; A second acquisition module, the second acquisition module is used to obtain the cabinet lines according to the detected dimension markings and the hand-drawn content diagram; A recognition module, the recognition module is used to input each dimension annotation into a digital recognition model to recognize the dimension number in each dimension annotation; A third acquisition module, the third acquisition module is used to acquire the horizontal coordinate range and the vertical coordinate range of the dimension line in each dimension annotation, and select the larger one of the horizontal coordinate range and the vertical coordinate range as the main coordinate range of the dimension line in the corresponding dimension annotation; A fourth acquisition module, the fourth acquisition module is used to obtain the horizontal coordinate range and the vertical coordinate range of each line segment in the cabinet line, and select the larger one of the horizontal coordinate range and the vertical coordinate range as the main coordinate range of the corresponding line segment; A calculation module, the calculation module is used to calculate the overlap between the main coordinate range of the dimension line in each dimension annotation and the main coordinate range of each line segment, and match the dimension annotation and the line segment according to the overlap to obtain the width information and height information of the cabinet to be designed; A fifth acquisition module, the fifth acquisition module is used to acquire the depth information of the cabinet to be designed; A generating module, the generating module is used to generate a three-dimensional model of the cabinet to be designed according to the width information, height information and depth information of the cabinet to be designed, The calculation module is specifically used to: determine whether the main coordinate range of a dimension line in a dimension annotation and the main coordinate range of a line segment both belong to the horizontal coordinate range, or both belong to the vertical coordinate range; if not, the overlap between the main coordinate range of the dimension line in the dimension annotation and the main coordinate range of the line segment is 0; if yes, calculate the ratio of the intersection length and the union length of the main coordinate range of the dimension line in the dimension annotation and the main coordinate range of the line segment to obtain the overlap between the main coordinate range of the dimension line in the dimension annotation and the main coordinate range of the line segment; determine whether the main coordinate range of the dimension line in the dimension annotation and the main coordinate range of the line segment Whether the degree of overlap between the coordinate ranges is greater than a preset value; if not, the dimension annotation is determined to be mismatched with the line segment; if greater than the preset value, whether there is at least one other dimension line in which the degree of overlap between the main coordinate range of the dimension line and the main coordinate range of the line segment is greater than the preset value; if not, the dimension annotation is determined to match the line segment, and the dimension number in the dimension annotation is used as the dimension of the line segment; if so, the dimension annotation with the largest degree of overlap is selected to match the line segment, and the dimension number in the dimension annotation with the largest degree of overlap is used as the dimension of the line segment.

6. The cabinet model generation system according to claim 5, characterized in that: The dimensioning detection model is a YOLO model, and the number recognition model is an OCR model.

7. The cabinet model generation system according to claim 6, characterized in that: The YOLO model includes multiple adaptive convolution modules. When performing feature extraction, each of the adaptive convolution modules performs the following operations on the input: ， in, F out is the output of the adaptive convolution module, F in is the input of the adaptive convolution module, represents the convolution operation, N is the total number of convolution kernels of the adaptive convolution module, T j For the j The convolution weight tensor of the convolution kernels, λ j for T j The corresponding adaptive coefficient is Among them, λ 1 to λ N The vector λ is: ， in, P (·) represents the global average pooling operation, M (·) represents the multi-layer perceptron operation, S (·) represents the normalized exponential function operation.

8. The cabinet model generation system according to claim 5, characterized in that: The preprocessing module performs denoising and binarization processing on the hand-drawn drawing of the cabinet to be designed.

Images