Method and device for identifying scale in image and electronic device

Abstract

The invention discloses a method and device for recognizing a scale in an image and an electronic device, and relates to the technical field of image processing, and the method comprises the steps: recognizing key points in the image through a key point recognition network, and obtaining a key point recognition result, the key point identification result comprises a plurality of groups of key point positions with connection directions, and the connection directions are opposite pairwise; according to the positions of the key points in the same group, determining the position of a scale endpoint; wherein the scale endpoint represents a scale end part in the image; associating the corresponding scale endpoints as scale line segments according to the positions of the scale endpoints with opposite connection directions; and according to the position of the scale line segment and a scale value identification result identified from the image, determining a corresponding relationship between the scale value and the scale line segment. According to the technical scheme of the embodiment of the invention, the accuracy and generalization of scale recognition in the image can be improved.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a method, device and electronic device for identifying a scale in an image. Background Art

[0002] House type recognition is a foundation in the field of home improvement, and ruler recognition is a key support for whether the house type can be truly restored.

[0003] Ruler recognition is mainly divided into two parts: ruler line segment recognition and ruler text recognition. Ruler line segment recognition refers to detecting the line segments representing the ruler in the floor plan. In the floor plan, the ruler line segments are often located in the four directions of the upper, lower, left and right sides of the main body of the house. In the prior art, since the ruler line segments are distributed around the main body of the house, the ruler line segments are divided into four categories according to the ruler line segments relative to the main body of the house, namely the upper, lower, left and right sides of the main body of the house, and then the pixels of the image are classified according to the classification of the ruler position, and finally the endpoints of the same category are merged into line segments.

[0004] In the prior art, the detection of ruler segments is achieved by classifying ruler endpoints. The classification of ruler endpoints requires regional segmentation of the image of the floor plan in order to determine the position of the ruler segments in the image. Based on the identified ruler endpoints, ruler endpoints belonging to the same category are merged into segments according to their positions. The four categories of endpoints need to be merged into segments one by one, which increases the difficulty of ruler recognition and the cost of merging ruler segments. Summary of the invention

[0005] The technical solution of the present application recognizes key points with connection direction and merges scale line segments using the connection direction information of the key points themselves, thereby fully increasing the generalization of the model.

[0006] According to one aspect of an embodiment of the present invention, a method for identifying a ruler in an image is provided, comprising:

[0007] Identify key points in the image through a key point recognition network to obtain key point recognition results, wherein the key point recognition results include multiple groups of key point positions with connection directions, and the connection directions are opposite to each other;

[0008] Determine the position of the ruler endpoint according to the position of the key point in the same group; wherein the ruler endpoint represents the end of the ruler in the image;

[0009] According to the positions of the ruler endpoints relative to the connection direction, the corresponding ruler endpoints are associated as ruler line segments;

[0010] The corresponding relationship between the scale value and the scale line segment is determined according to the position of the scale line segment and the scale value recognition result recognized from the image.

[0011] In one possible implementation, identifying key points in an image through a key point recognition network and obtaining key point recognition results includes:

[0012] Identify the key points in the image through a key point recognition network having four connection direction channels, and output pixel point recognition results in each connection direction channel; wherein the four connection direction channels are a channel with connection points downward, a channel with connection points to the right, a channel with connection points upward, and a channel with connection points to the left;

[0013] The pixel point recognition result is filtered according to the set threshold. If the pixel point recognition result is less than the threshold, the corresponding pixel point is a non-key point. If the pixel point recognition result is greater than or equal to the threshold, the corresponding pixel point is the key point.

[0014] In an implementable manner, determining the position of the ruler endpoint according to the key point positions in the same group includes:

[0015] Determine the key points within a distance threshold range in each group of key points as adjacent key points;

[0016] Make a minimum bounding rectangle for the adjacent key points, and use the minimum bounding rectangle as the key point connected area;

[0017] The center of the key point connected area is determined as the endpoint position of the ruler.

[0018] In an implementable manner, determining the center of the key point connected area as the ruler endpoint position includes:

[0019] Projecting the key point connected area onto the abscissa axis and the ordinate axis;

[0020] The coordinates of the midpoint of the projection line segment on the abscissa axis and the coordinates of the midpoint of the projection line segment on the ordinate axis are used as the coordinates of the endpoints of the ruler.

[0021] In an implementable manner, the associating the corresponding ruler endpoints as ruler line segments according to the positions of the ruler endpoints relative to the connection directions includes:

[0022] The ruler endpoints corresponding to the coordinate positions are searched in the ruler endpoint positions relative to the connection direction and connected to obtain the ruler line segment.

[0023] In one achievable manner, identifying a scale value recognition result from the image includes:

[0024] Detecting text in the image based on an optical character recognition (OCR) algorithm, and recording the position of the text as a scale value position;

[0025] The purely numerical text is retained as the ruler value.

[0026] In an implementable manner, determining the correspondence between the scale value and the scale segment according to the position of the scale segment and the scale value recognition result recognized from the image includes:

[0027] Determine the distance between the center position of each ruler line segment and the ruler value position;

[0028] The maximum weighted match between the scale value and the scale line segment is determined according to the distance to obtain the scale value corresponding to each scale line segment.

[0029] In an implementable manner, after determining the correspondence between the scale value and the scale segment according to the position of the scale segment and the scale value recognition result recognized from the image, the method further includes:

[0030] Determining the pixel density represented by the ruler line segment according to the ruler line segment and the corresponding ruler value;

[0031] The pixel density of the ruler line segment is detected for abnormal recognition results.

[0032] According to another aspect of an embodiment of the present invention, a device for identifying a scale in an image is provided, comprising:

[0033] A key point recognition module, used to recognize key points in an image through a key point recognition network, and obtain key point recognition results, wherein the key point recognition results include multiple groups of key point positions with connection directions, and the connection directions are opposite to each other;

[0034] A ruler endpoint determination module, used to determine the ruler endpoint position according to the key point position of the same group; wherein the ruler endpoint represents the end of the ruler in the image;

[0035] A ruler line segment determination module, used for associating the corresponding ruler endpoints as ruler line segments according to the positions of the ruler endpoints relative to the connection direction;

[0036] The ruler line segment and ruler value correspondence module is used to determine the correspondence between the ruler value and the ruler line segment according to the position of the ruler line segment and the ruler value recognition result recognized from the image.

[0037] According to another aspect of an embodiment of the present invention, there is provided an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor is configured to execute the method for recognizing a scale in an image provided in any embodiment through the computer program.

[0038] The embodiments of the present invention provide a method, device and electronic device for recognizing a ruler in an image. The method recognizes key points with connection direction and merges ruler line segments using the connection direction information of the key points themselves. This solves the problem that the ruler endpoints are classified and each type is merged into line segments separately, which increases the difficulty of ruler recognition and the high cost of merging ruler line segments. In addition, since the recognized key points themselves have certain connection direction information, this greatly reduces the number of times the endpoints are merged into line segments, thereby improving the recognition accuracy and generalization of use. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0040] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0041] Figure 1 is a schematic diagram of a hardware environment of a method for identifying a ruler in an image according to an embodiment of the present application;

[0042] Figure 2 is a flow chart of a method for identifying a scale in an image according to an embodiment of the present application;

[0043] Figure 3 is an example of a floor plan image according to an embodiment of the present application;

[0044] Figure 4 is a schematic diagram of a key point recognition result according to an embodiment of the present application;

[0045] Figure 5 is a schematic structural diagram of a device for identifying a scale in an image according to an embodiment of the present application;

[0046] Figure 6 It is a schematic diagram of the structure of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0047] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0048] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0049] According to one aspect of an embodiment of the present application, a method for identifying a ruler in an image is provided. The method for identifying a ruler in an image is widely used in smart home, smart home, smart home device ecology, smart residential (Intelligence House) ecology and other whole-house intelligent digital control application scenarios. Optionally, in this embodiment, the above-mentioned method for identifying a ruler in an image can be applied to Figure 1 In the hardware environment composed of the terminal device 102 and the server 104 shown in FIG. Figure 1 As shown, the server 104 is connected to the terminal device 102 via a network, and can be used to provide services (such as application services, etc.) for the terminal or a client installed on the terminal. A database can be set on the server or independently of the server to provide data storage services for the server 104. Cloud computing and / or edge computing services can be configured on the server or independently of the server to provide data computing services for the server 104.

[0050] The network may include but is not limited to at least one of the following: wired network, wireless network. The wired network may include but is not limited to at least one of the following: wide area network, metropolitan area network, local area network, and the wireless network may include but is not limited to at least one of the following: WIFI (Wireless Fidelity), Bluetooth. The terminal device 102 may be but is not limited to a PC, a mobile phone, a tablet computer, a smart air conditioner, a smart range hood, a smart refrigerator, a smart oven, a smart stove, a smart washing machine, a smart water heater, a smart washing device, a smart dishwasher, a smart projection device, a smart TV, a smart clothes drying rack, a smart curtain, a smart audio and video, a smart socket, a smart speaker, a smart fresh air device, a smart kitchen and bathroom device, a smart bathroom device, a smart sweeping robot, a smart window cleaning robot, a smart mopping robot, a smart air purification device, a smart steamer, a smart microwave oven, a smart kitchen treasure, a smart purifier, a smart water dispenser, a smart door lock, etc.

[0051] With the development of home improvement and deep learning technology, it is particularly important to use existing floor plans to build a floor plan vector database. To construct vector data, it is necessary to identify the ruler data in the floor plan to restore the real floor plan size. Without ruler recognition, it is impossible to make some home improvement and product recommendations for customers based on actual conditions. This ruler recognition method can help users restore the real size of the floor plan more intuitively. Users can upload their own floor plans through terminal devices, and use the ruler recognition method in the image in the embodiment of the present application to provide users with accurate on-site measurement recommendations. Users can upload images of floor plans on the corresponding application page through their own personal computers, smart phones, tablet computers and other terminal devices, for example, using the camera on the terminal device to take images of floor plans, or, in the images pre-stored locally on the terminal device, select images of floor plans and upload them to the application server.

[0052] According to one aspect of an embodiment of the present invention, Figure 2 The flowchart of a method for identifying a ruler in an image is shown. The method for identifying a ruler in an image can be used in a server. The method can specifically include the following steps:

[0053] Step S210: identifying key points in the image through a key point recognition network to obtain key point recognition results.

[0054] The floor plan is usually an image file, which is uploaded to the application server in an image format, such as a .jpg or .png file. Figure 3An example of a floor plan is shown in the figure. The main body of the floor plan is located in the middle of the floor plan, showing the floor plan layout, the layout and connection relationship of each room, and the direction of each room's door. The main body of the floor plan may also indicate the name and area of ​​the room. The method in the embodiment of the present application is mainly to identify the scale in the floor plan. Figure 3 In the example of the floor plan shown, ruler segments and ruler values ​​are arranged above, on the left, below, and on the right of the main body of the floor plan. The ruler referred to in the embodiment of the present application includes ruler segments and ruler values, and the ruler value usually defaults to millimeters. Therefore, some floor plans do not have units after the ruler values. For the embodiment of the present application, there is no limitation on whether the ruler values ​​of the floor plans have units. Figure 3 This is just an example of a floor plan. Figure 3 As can be seen in the figure, there are three rulers located above the main body of the apartment, including a longer ruler with a ruler value of 7908, and two shorter rulers with ruler values ​​of 5061 and 2727. If the pixel point in the lower left corner of the apartment image is taken as the origin, the width of the apartment image is taken as the horizontal direction, and the height of the apartment image is taken as the vertical direction, a plane rectangular coordinate system is established, and the rulers above the three main bodies of the apartment are arranged along the width direction of the image, which can be considered as horizontal rulers in the image. Then, taking one of the rulers with a value of 7908 as an example, a ruler line segment has two ruler endpoints, the left endpoint of one ruler line segment and the right endpoint of the other ruler line segment. For the left endpoint of the ruler line segment, it has a connection point connected to the right, that is, the right endpoint of the ruler line segment; similarly, for the right endpoint of the ruler line segment, it has a connection point connected to the left, that is, the left endpoint of the ruler line segment. Of course, it is understandable that the ruler below the main body of the house type and the ruler above the main body of the house type in the house type image can also be considered as horizontal rulers in the image, and their ruler segments also have left and right endpoints. There are four rulers located on the left side of the main body of the house type, namely a longer ruler with a ruler value of 7003, and three shorter rulers with ruler values ​​of 651, 1408 and 4823. The rulers on the left side of the four main bodies of the house type are arranged along the height direction of the image and can be considered as vertical rulers in the image. Then, taking one of the rulers of 7003 as an example, a ruler segment has two ruler endpoints, the upper endpoint of one ruler segment and the lower endpoint of the other ruler segment. For the upper endpoint of the ruler segment, it has a connection point connected downward, that is, the lower endpoint of the ruler segment; similarly, for the lower endpoint of the ruler segment, it has a connection point connected upward, that is, the upper endpoint of the ruler segment. Of course, it is understandable that the ruler on the right side of the main body of the floor plan and the ruler on the left side of the main body of the floor plan can also be considered as vertical rulers in the image, and their ruler segments also have upper and lower endpoints.

[0055] The key point recognition network can be a network model with multiple channels, which is used to recognize key points in the floor plan image. The key points are the endpoints of the ruler segments. The recognition results of the multiple channels outputted constitute the key point recognition results. The key points in the recognition results of each channel have a connection direction, and the connection direction indicates that the key point has a connectable key point in this direction. The key point recognition results include multiple groups of key points with connection directions, and the connection directions are opposite to each other, so that the connection directions are opposite to each other so as to be subsequently associated as ruler segments.

[0056] Step S230: Determine the position of the ruler endpoint according to the key point positions in the same group.

[0057] Among them, the ruler endpoint represents the end of the ruler in the image. Since the ruler line segment in the floor plan image is often not an ideal single row or single column of continuous pixel points, then the ruler endpoint of the ruler line segment may not be a single key point, that is, the key point output by the key point recognition network may have multiple key points corresponding to a ruler endpoint in the actual image. Therefore, for the same group of key points in the key point recognition result, it can be determined which key points correspond to a ruler endpoint based on the relative position between the key points, and multiple key points corresponding to a ruler endpoint are combined to determine the ruler endpoint position based on their positions. The ruler endpoint position can be determined based on the coordinate distribution of multiple key point positions to determine a position center of gravity as the ruler endpoint position, or, based on the coordinate distribution of multiple key point positions, select the center position as the ruler endpoint position. It can be understood that the determined ruler endpoint position has a connection direction in addition to its own coordinate position, and its connection direction is the connection direction of the group of key points.

[0058] Step S250: according to the positions of the ruler endpoints relative to the connection directions, the corresponding ruler endpoints are associated as ruler line segments.

[0059] Among them, only the ruler endpoints with opposite connection directions are likely to be associated. Figure 3In the description of the example, the connection directions of the two ruler endpoints of the same ruler line segment are relative. Therefore, the ruler endpoints with relative connection directions are selected, and the corresponding two ruler endpoints are determined to be associated as ruler line segments according to the positional relationship of the ruler endpoints. Usually, the two ruler endpoints of the longitudinal ruler line segment are basically consistent in the horizontal position, then, the ruler endpoint with a connection point in the downward connection direction is searched downward along its connection direction, and the ruler endpoint with the connection point in the upward connection direction that is closest to its lower longitudinal distance is a pair of ruler endpoints. Of course, the ruler endpoint with the connection point in the upward connection direction can also be used to search for the corresponding ruler endpoint upward along its connection direction. For the two ruler endpoints of the transverse ruler line segment, the longitudinal position is basically consistent, then, the ruler endpoint with a connection point in the right connection direction is searched to the right along its connection direction, and the ruler endpoint with the connection point in the left connection direction that is closest to its right horizontal distance is a pair of ruler endpoints. Of course, the ruler endpoint with the connection point in the left connection direction can also be used to search for the corresponding ruler endpoint along its connection direction to the left. It can be seen that when the ruler endpoints themselves carry connection direction information, matching ruler endpoints is very efficient. Compared with rulers classified by position, searching for matching key points in the same category may result in mismatches, for example, Figure 3 In the ruler above the main body of the medium-sized apartment, there are two shorter rulers with ruler values ​​of 5061 and 2727, and their ruler endpoints have the same longitudinal position and different lateral positions. However, in order to determine the matching relationship between the ruler endpoints, it is also necessary to use the ruler value to exclude the ruler line segments with incorrect associations, or it is necessary to count the position distribution of the four ruler endpoints to determine the matching ruler endpoints. In contrast, the embodiment of the present application associates the ruler endpoints into ruler line segments in this way, which is very fast.

[0060] Step S270: Determine the correspondence between the scale value and the scale segment according to the position of the scale segment and the scale value recognition result recognized from the image.

[0061] The scale value recognition result includes the scale value and the scale value position. The text in the floor plan image can be recognized by means of text recognition to find the text representing the scale value. The scale value and the scale line segment in the floor plan are usually in one-to-one correspondence. By determining the position of the scale line segment and the position of the scale value, the corresponding relationship between the scale line segment and the scale value can be determined by the relative position relationship between the scale line segment and the scale value.

[0062] In one possible implementation, identifying key points in an image through a key point recognition network and obtaining key point recognition results includes:

[0063] Identify the key points in the image through a key point recognition network having four connection direction channels, and output pixel point recognition results in each connection direction channel;

[0064] Among them, the four connection direction channels are respectively a channel with a connection point downward, a channel with a connection point to the right, a channel with a connection point upward, and a channel with a connection point to the left. The key point recognition network can be trained using samples with connection directions based on a stacked convolutional neural network. The key point recognition network training process generally includes: building a network model, providing data samples to train the model, and then the model produces results; the embodiment of the present application splits the ruler recognition into key point recognition, and obtains better estimation results with the help of the trained network; the key points are represented by channel data with connection direction information.

[0065] The pixel point recognition result is filtered according to the set threshold. If the pixel point recognition result is less than the threshold, the corresponding pixel point is a non-key point. If the pixel point recognition result is greater than or equal to the threshold, the corresponding pixel point is the key point.

[0066] Among them, the key points output by the key point recognition network after recognition can be represented in the form of channel data, for example, matrix data in a neural network. Figure 4 In the schematic diagram of the key point recognition results shown in the figure, the left side is the output result diagram of one channel (the example shows part of the content, and the rest of the recognition results are omitted), and the four channels are superimposed to obtain Figure 4 The channel diagram on the right. The recognition results of a channel are key points with the same connection direction. The output results of the same channel are regardless of the position of the ruler on the main body of the apartment. For example, the ruler above the main body of the apartment and the ruler below are both horizontal rulers. The ruler endpoints of these ruler segments have ruler endpoints in the left connection direction and ruler endpoints in the right connection direction. When identifying these ruler endpoints, the key point recognition network will identify the ruler endpoints and output the key point recognition results under the corresponding channel. Each pixel in the image is output as a numerical value after passing through the key point recognition network, and then the channel results are filtered according to the set threshold, and the channel results are binarized. If the pixel recognition result is less than the threshold, the corresponding pixel is a non-key point. If the pixel recognition result is greater than or equal to the threshold, the corresponding pixel is the key point. Figure 4 As shown in , the matrix output result of the single channel output on the left can use 1 to represent key points and 0 to represent non-key points.

[0067] In an implementable manner, determining the position of the ruler endpoint according to the key point positions in the same group includes:

[0068] Determine the key points within a distance threshold range in each group of key points as adjacent key points;

[0069] Make a minimum bounding rectangle for the adjacent key points, and use the minimum bounding rectangle as the key point connected area;

[0070] The center of the key point connected area is determined as the endpoint position of the ruler.

[0071] Among them, the key points predicted by the key point recognition network may contain noise. After denoising the key points predicted by the key point recognition network, the key points are merged into the ruler endpoints. In a set of key points in each channel, according to the pre-set distance threshold, all key points are traversed to find other key points within the distance threshold range from the key point and determine them as adjacent key points. In this way, the key points in the same group establish a proximity relationship due to the distance. This proximity relationship in distance can be associated one by one, for example, Figure 4 The number 1 represents a key point. The key point in the first row and third column is adjacent to the key point in the second row and third round, and the key point in the second row and third round is adjacent to the key point in the third row and second column. These three key points are considered to be adjacent key points. A connected area formed by adjacent key points can be defined. The connected area can be a minimum bounding rectangle of adjacent key points, and the minimum bounding rectangle is used as the key point connected area. Then the center of the key point connected area is determined as the endpoint position of the ruler. Figure 4 For the key point with the number 1, the rectangular area (dashed rectangle) circumscribed by adjacent key points is the key point connected area.

[0072] In an implementable manner, determining the center of the key point connected area as the ruler endpoint position includes:

[0073] Projecting the key point connected area onto the abscissa axis and the ordinate axis;

[0074] The coordinates of the midpoint of the projection line segment on the abscissa axis and the coordinates of the midpoint of the projection line segment on the ordinate axis are used as the coordinates of the endpoints of the ruler.

[0075] Among them, the center of the key point connected area can be determined by projecting the key point connected area onto the horizontal axis and the vertical axis. The projected key point connected area appears as a line segment on the two coordinate axes. The midpoint of the line segment is selected. The coordinates of the midpoint can represent the center coordinates of the key point connected area, and then serve as the coordinates of the endpoints of the ruler. It can be understood that the image is composed of pixels, so if the projected line segment is an even number of pixels, one of the two pixels in the middle position can be selected as the midpoint of the projected line segment. When selecting, you can choose according to the position distribution of the key points in the key point connected area, for example, Figure 4In the key point connected area shown in , after projection on the horizontal axis, because the pixel point on the right has two key points projected onto this point, the pixel point on the right can be selected as the coordinate of the midpoint of the projection line segment on the horizontal axis.

[0076] In an implementable manner, the associating the corresponding ruler endpoints as ruler line segments according to the positions of the ruler endpoints relative to the connection directions includes:

[0077] The ruler endpoints corresponding to the coordinate positions are searched in the ruler endpoint positions relative to the connection direction and connected to obtain the ruler line segment.

[0078] The connection directions of the two ruler endpoints of the same ruler line segment are relative. The ruler endpoints with relative connection directions are selected, and the first ruler endpoint with relative connection direction is found along its connection direction with one ruler endpoint as the starting point, and is determined as the corresponding ruler endpoint, and then the corresponding ruler endpoints are connected to obtain the ruler line segment.

[0079] In one achievable manner, identifying a scale value recognition result from the image includes:

[0080] Detecting text in the image based on an optical character recognition (OCR) algorithm, and recording the position of the text as a scale value position;

[0081] The purely numerical text is retained as the ruler value.

[0082] The floor plan image input by the user is subjected to text detection, and an optical character recognition (OCR) algorithm can be used to detect and recognize the location information of the scale value and the text of the scale value. The recognition result is text / number and the location of the text / number, and non-numeric text is filtered out. Figure 3 The character texts marked in the main body of the medium apartment, such as "living room and dining room", "bathroom", "kitchen", "master bedroom", etc., will be filtered out. In addition, the mixed character and numerical texts such as "Area: 22.57m 2 " will also be filtered out. This will leave only the scale value of the scale.

[0083] In an implementable manner, determining the correspondence between the scale value and the scale segment according to the position of the scale segment and the scale value recognition result recognized from the image includes:

[0084] Determine the distance between the center position of each ruler line segment and the ruler value position;

[0085] The maximum weighted match between the scale value and the scale line segment is determined according to the distance to obtain the scale value corresponding to each scale line segment.

[0086] Among them, the position of the area where each ruler value is located can be calculated as the center coordinate value, the distance from the center position of the ruler segment. Then the reciprocal of the distance from the ruler value position to the center position of the ruler segment is taken. If the distance from the ruler value position to the center position of the ruler segment is smaller, the weight of the ruler value and the ruler segment after the number is taken will be greater. Conversely, if the distance from the ruler value position to the center position of the ruler segment is larger, the weight of the ruler value and the ruler segment after the number is taken will be smaller. The ruler value corresponding to each ruler segment can be matched with the maximum weight.

[0087] In an implementable manner, after determining the correspondence between the scale value and the scale segment according to the position of the scale segment and the scale value recognition result recognized from the image, the method further includes:

[0088] Determining the pixel density represented by the ruler line segment according to the ruler line segment and the corresponding ruler value;

[0089] The pixel density of the ruler line segment is detected for abnormal recognition results.

[0090] Among them, the detection of abnormal recognition results is to find a more accurate pixel density as the proportion of the floor plan. The ruler value and the ruler segment have been matched before. There is a ratio between the ruler value and the pixel length of the corresponding ruler segment in the image. The ruler value can be divided by the pixel length of the ruler segment to obtain the pixel density represented by the ruler segment. Considering that there may be some misrecognition in the text recognition of ruler segments or ruler values, some anomaly detection algorithms are used for filtering to achieve more robust results. Specifically, the interquartile range anomaly detection can be used to detect abnormal recognition results. 1. Calculate the quartiles: the first quartile (Q1): the value at the lower 25% of the data set. The third quartile (Q3): the value at the upper 75% of the data set. 2. Calculate the interquartile range (IQR): IQR = Q3-Q1; IQR is the range of the middle 50% area of ​​the data distribution. 3. Determine the outlier threshold: Lower Bound = Q1-k*IQR; Upper Bound = Q3+k*IQR. K is usually 1.5, but can be adjusted as needed. 4. Identify outliers: Any data point below the lower bound or above the upper bound is considered an outlier.

[0091] According to another aspect of the embodiment of the present invention, Figure 5The schematic diagram of the structure of a device for identifying a ruler in an image is shown. The device for identifying a ruler in an image may be arranged on a server, and the device may specifically include:

[0092] The key point recognition module 510 is used to recognize key points in the image through a key point recognition network and obtain key point recognition results.

[0093] The key point recognition result includes multiple groups of key point positions with connection directions, and the connection directions are opposite to each other.

[0094] A ruler endpoint determination module 530, used to determine the ruler endpoint position according to the key point position of the same group;

[0095] Wherein, the ruler endpoint represents the end of the ruler in the image;

[0096] A ruler line segment determination module 550, configured to associate the corresponding ruler endpoints as ruler line segments according to the positions of the ruler endpoints relative to the connection direction;

[0097] The ruler line segment and ruler value corresponding module 570 is used to determine the corresponding relationship between the ruler value and the ruler line segment according to the position of the ruler line segment and the ruler value recognition result recognized from the image.

[0098] In one achievable manner, the key point identification module 510 is specifically configured to:

[0099] Identify the key points in the image through a key point recognition network having four connection direction channels, and output pixel point recognition results in each connection direction channel; wherein the four connection direction channels are a channel with connection points downward, a channel with connection points to the right, a channel with connection points upward, and a channel with connection points to the left;

[0100] The pixel point recognition result is filtered according to the set threshold. If the pixel point recognition result is less than the threshold, the corresponding pixel point is a non-key point. If the pixel point recognition result is greater than or equal to the threshold, the corresponding pixel point is the key point.

[0101] In one implementable manner, the ruler endpoint determination module 530 specifically includes:

[0102] An adjacent key point determination unit, used to determine the key points within a distance threshold range in each group of key points as adjacent key points;

[0103] A connected area determination unit, used for making a minimum bounding rectangle for the adjacent key points, and taking the minimum bounding rectangle as the key point connected area;

[0104] The ruler endpoint determination unit is used to determine the center of the key point connected area as the ruler endpoint position.

[0105] In an implementable manner, the ruler endpoint determination unit is specifically used to:

[0106] Projecting the key point connected area onto the abscissa axis and the ordinate axis;

[0107] The coordinates of the midpoint of the projection line segment on the abscissa axis and the coordinates of the midpoint of the projection line segment on the ordinate axis are used as the coordinates of the endpoints of the ruler.

[0108] In an implementable manner, the ruler line segment determination module 550 is specifically used to:

[0109] The ruler endpoints corresponding to the coordinate positions are searched in the ruler endpoint positions relative to the connection direction and connected to obtain the ruler line segment.

[0110] In one possible implementation, the device for identifying a scale in an image further includes:

[0111] A scale value recognition module, used to recognize a scale value recognition result from the image;

[0112] wherein, the text in the image is detected based on an optical character recognition (OCR) algorithm, and the position of the text is recorded as the scale value position;

[0113] The purely numerical text is retained as the ruler value.

[0114] In an implementable manner, the ruler line segment and ruler value correspondence module 570 is specifically used to:

[0115] Determine the distance between the center position of each ruler line segment and the ruler value position;

[0116] The maximum weighted match between the scale value and the scale line segment is determined according to the distance to obtain the scale value corresponding to each scale line segment.

[0117] In one possible implementation, the device for identifying a scale in an image further includes:

[0118] a pixel density determination module, configured to determine the pixel density represented by the scale line segment according to the scale line segment and the corresponding scale value after determining the correspondence between the scale value and the scale line segment according to the position of the scale line segment and the scale value recognition result recognized from the image;

[0119] The anomaly detection module is used to detect anomaly recognition results of the pixel density of the ruler line segment.

[0120] The device for recognizing a ruler in an image provided by an embodiment of the present invention can execute the method for recognizing a ruler in an image provided by any embodiment of the present invention, and has functional modules and beneficial effects corresponding to the execution method.

[0121] According to another aspect of the embodiments of the present invention, there is provided an electronic device, such as Figure 6 As shown, it includes a memory 610 and a processor 620, wherein the memory 610 stores a computer program, and the processor 620 is configured to execute the method for recognizing a scale in an image provided in any embodiment through the computer program.

[0122] The number of processors 620 in the electronic device may be one or more. Figure 6 In the figure, a processor 620 is used as an example; the electronic device may also include an input device 630 for receiving input digital or character information and generating signal input related to user settings and function control of the electronic device. The output device 640 may include a display device such as a display screen. The memory 610, processor 620, input device 630 and output device 640 in the electronic device may be connected via a bus or other means. Figure 6 The example of connecting through bus is taken in the following.

[0123] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A method for identifying a scale in an image, characterized in that: include: Identify key points in the image through a key point recognition network to obtain key point recognition results, wherein the key point recognition results include multiple groups of key point positions with connection directions, and the connection directions are opposite to each other; determine the position of the ruler endpoint according to the key point positions in the same group; wherein the ruler endpoint represents the end of the ruler in the image; According to the positions of the ruler endpoints relative to the connection direction, the corresponding ruler endpoints are associated as ruler line segments; The corresponding relationship between the scale value and the scale line segment is determined according to the position of the scale line segment and the scale value recognition result recognized from the image.

2. The method according to claim 1, characterized in that The step of identifying key points in an image through a key point recognition network and obtaining key point recognition results includes: Identify the key points in the image through a key point recognition network having four connection direction channels, and output pixel point recognition results in each connection direction channel; wherein the four connection direction channels are a channel with connection points downward, a channel with connection points to the right, a channel with connection points upward, and a channel with connection points to the left; The pixel point recognition result is filtered according to the set threshold. If the pixel point recognition result is less than the threshold, the corresponding pixel point is a non-key point. If the pixel point recognition result is greater than or equal to the threshold, the corresponding pixel point is the key point.

3. The method according to claim 2, characterized in that Determining the position of the ruler endpoint according to the key point position of the same group includes: Determine the key points within a distance threshold range in each group of key points as adjacent key points; Make a minimum bounding rectangle for the adjacent key points, and use the minimum bounding rectangle as the key point connected area; The center of the key point connected area is determined as the ruler endpoint position.

4. The method according to claim 3, characterized in that The step of determining the center of the key point connected area as the ruler endpoint position includes: Projecting the key point connected area onto the abscissa axis and the ordinate axis; The coordinates of the midpoint of the projection line segment on the abscissa axis and the coordinates of the midpoint of the projection line segment on the ordinate axis are used as the coordinates of the endpoints of the ruler.

5. The method according to claim 3, characterized in that: According to the position of the ruler endpoint relative to the connection direction, associating the corresponding ruler endpoint as a ruler line segment includes: The ruler endpoints corresponding to the coordinate positions are searched in the ruler endpoint positions relative to the connection direction and connected to obtain the ruler line segment.

6. The method according to claim 1, characterized in that Identifying a scale value recognition result from the image, including: Detecting text in the image based on an optical character recognition (OCR) algorithm, and recording the position of the text as a scale value position; The purely numerical text is retained as the ruler value.

7. The method according to claim 6, characterized in that The determining the correspondence between the scale value and the scale line segment according to the position of the scale line segment and the scale value recognition result recognized from the image includes: Determine the distance between the center position of each ruler line segment and the ruler value position; The maximum weighted match between the scale value and the scale line segment is determined according to the distance to obtain the scale value corresponding to each scale line segment.

8. The method according to claim 1, characterized in that After determining the correspondence between the scale value and the scale line segment according to the position of the scale line segment and the scale value recognition result recognized from the image, the method further includes: Determining the pixel density represented by the ruler line segment according to the ruler line segment and the corresponding ruler value; The pixel density of the ruler line segment is detected for abnormal recognition results.

9. A device for identifying a scale in an image, characterized in that: include: A key point recognition module, used to recognize key points in an image through a key point recognition network, and obtain key point recognition results, wherein the key point recognition results include multiple groups of key point positions with connection directions, and the connection directions are opposite to each other; A ruler endpoint determination module, used to determine the ruler endpoint position according to the key point position of the same group; Wherein, the ruler endpoint represents the end of the ruler in the image; A ruler line segment determination module, used for associating the corresponding ruler endpoints as ruler line segments according to the positions of the ruler endpoints relative to the connection direction; The ruler line segment and ruler value correspondence module is used to determine the correspondence between the ruler value and the ruler line segment according to the position of the ruler line segment and the ruler value recognition result recognized from the image.

10. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to execute the method according to any one of claims 1 to 8 through the computer program.

Images